A PROJECT REPORT On
To study and implement Digital Watermarking algorithm on Images using Scale Invariant Feature Transformation
submitted for partial fulfillment for the degree of Bachelor of Technology
In Department of Computer Engineering (2009-10)
Supervisor: Dr. Vijay Laxmi
Pritam Hinger (0609449) Vikas Sarda
(0609463)
Vishal Pareek (0609469)
MALAVIYA NATIONAL INSTITUTE of TECHNOLOGY
MALAVIYA NATIONAL INSTITUTE of TECHNOLOGY (Deemed University) JAIPUR(RAJASTHAN)-302017 DEPARTMENT of COMPUTER ENGINEERING
CERTIFICATE This is to certify that project report entitled “Study and implement Digital Watermarking algorithm on Images using Scale Invariant Feature Transformation” is submitted by Pritam Hinger (0609449) Vikas Sarda (0609463) Vishal Pareek (0609469) In partial fulfillment for the degree of bachelor of technology in computer engineering department is hereby approved for submission.
Dr. Vijay Laxmi Professor, Department of Computer Engineering MNIT, Jaipur
Contents
Chapter 1
INTRODUCTION ......................................................................................................... 6
Chapter 2
Overview ................................................................................................................ 8
2.1 History of Watermarking .................................................................................................................. 10 2.2 Requirements for watermarking algorithms: .................................................................................... 11 2.3 Importance of Digital Watermarking ................................................................................................ 12 2.4 Applications of Digital Watermarking .............................................................................................. 14 2.5 Attacks on Watermarked Work ........................................................................................................ 17
2.5.1 Scrambling Attacks: ..................................................................................................... 17 2.5.2 Pathological Distortions: .............................................................................................. 17 2.5.3 Copy Attacks: ............................................................................................................... 18 2.5.4 Ambiguity Attacks:....................................................................................................... 18 Chapter 3
Methodology ................................................................................................................. 19
3.1 Project Flow ...................................................................................................................................... 19
3.1.1 Insertion of Watermark Image into Cover Work: - ...................................................... 19 3.1.2 Extraction of Watermark Image from Watermarked Image: - ..................................... 19 3.1.3 Calculation of Detection ratio: - ................................................................................... 20 3.2 Step By Step procedure ..................................................................................................................... 20
3.2.1 Read Cover Image: ....................................................................................................... 20 3.2.2 SIFT: ............................................................................................................................. 20 3.2.3 Watermark Generation: ................................................................................................ 27 3.2.4 Watermark Insertion: .................................................................................................... 29 3.2.5 Watermark Detection:................................................................................................... 31 3.2.6 Detection ratio: ............................................................................................................. 33 Chapter 4
RESULTS ....................................................................................................................... 34
4.1 Test Images: ...................................................................................................................................... 34
4.1.1 Cover work: .................................................................................................................. 34 4.1.2 Position of Invariant patches in cover work: ................................................................ 34 4.1.3 Watermark image: ........................................................................................................ 35
4.1.4 Cover Image with watermark: ...................................................................................... 35 4.1.5 Retrieved Watermark:................................................................................................... 35 4.2 Performance of watermarking scheme and test results: .................................................................... 36 Chapter 5
Conclusion and Future Work.................................................................................... 37
Appendix A .................................................................................................................................................. 38 References .................................................................................................................................................. 40
ACKNOWLEDGEMENT We are deeply indebted to our guide Dr. Vijay Laxmi, professor in the department of computer engineering, MNIT Jaipur for their incomparable encouragement and valuable support in the project. We owe our deep regard to Dr. M. S. Gaur, professor in the department of computer engineering, MNIT Jaipur, whose noble suggestion and guidance as our project coordinator was instrumental in completing the project. We would like to acknowledge our sincere regards to all faculty members of department of Computer Engineering, MNIT and especially to Mrs. Reena Gunjan for her help and valuable suggestions in the project. We would like to thank our friend and classmate Roopesh Chuggani for his direct and indirect help in the project.
Pritam Hinger
Vikas Sarda
Vishal Pareek
Date: - 10th May, 2010 Place: - MNIT, Jaipur
ABSTRACT
With the advent of internet, creation and delivery of digital data (images, video and audio files, digital repositories and libraries, web publishing) has grown many fold. With this, issues like, protection of rights of the content and proving ownership, arises. Digital watermarking came as a technique and a tool to overcome shortcomings of current copyright laws for digital data. To prove ownership and protect right, a watermark is embedded in data but to save watermark from counterfeiters we need to find locations which are invariant to all kind of attacks (rotation, expansion, compression, cropping, filtering, and blurring). Every image has regions, also known as patches, which are invariant to attacks. These patches can be found by using Scale Invariant Feature Transform (SIFT) over image. As these patches are very stable and resistant to attacks so watermark is inserted in these patches and can also be successfully extracted with low error probability.
Chapter 1
INTRODUCTION
We are living in the era of information where billions of bits of data is created in every fraction of a second and with the advent of internet, creation and delivery of digital data (images, video and audio files, digital repositories and libraries, web publishing) has grown many fold. Since copying a digital data is very easy and fast too so, issues like, protection of rights of the content and proving ownership, arises. Digital watermarking came as a technique and a tool to overcome shortcomings of current copyright laws for digital data. The specialty of watermark is that it remains intact to the cover work even if it is copied. So to prove ownership or copyrights of data watermark is extracted and tested. It is very difficult for counterfeiters to remove or alter watermark. As such the real owner can always have his data safe and secure. Our aim was to study different watermarking techniques and implement the one which is most resistant to all types of attack, scalar or geometric. Counterfeiters try to degrade the quality of watermarked image by attacking an image (generally attacks are median and Gaussian filter, scaling, compression and rotation of watermarked image).By attacking watermarked image it become very difficult to recover watermark back from the watermarked image and even if it extracted one may no longer use it to prove the ownership and copyrights. So our main idea was to find such regions, also known as patches, in an image which are very stable and resistant to attacks. The report is divided mainly in 4 chapters coving literature on watermarking (chapter2), our methodology and step by step procedure (chapter3), test images and results (chapter4), conclusion and future work (chapter5). Chapter 2 gives full insight of digital watermarking, its history, requirements, application and possible attacks. The first subheading tells how, with information revolution, the need to have some technique to prevent piracy and illegal copying of data arises. This need give rise to a new technique, known as Digital Watermarking. While proposing any algorithm some parameters are needed to keep in mind on which the proposed algorithm must be consistent. These parameters are discussed in following section. Following sections are dedicated to watermarking application and attacks. A lot of work is going on for making watermarking techniques immune towards attack to retain the originality of watermark and assuring successful
extraction of watermark with low error probabilities so to sort out disputes, if any, over copyrights or ownership. Chapter 3 starts with a flowchart showcasing the complete flow of project. Flowchart also contains names of all the functions written to implement the proposed method. The next few sections explain the complete process followed by us in full detail, listing all the mathematical steps and explaining all the concepts used, like SIFT. Chapter 4 lists all the test images and test results. The proposed method is tested for over 30 random images (all 150x200 .jpeg images) covering over 7500 patches. Each image is attacked (13 different types of attacks) and tested. Results are listed in next section. Chapter 5 concludes the report with possible work which may be done in future. Since threats like piracy and counterfeiting are increasing day by day so a lot more work and research can be done. Appendix A contains code snippets and all functions’ signature.
Chapter 2
Overview
Hold a Rs100 note up or your offer letter up to light. What you will see is a picture of Mahatma Gandhi or company’s logo respectively. This is what is known as a watermark mainly used to prove the ownership (in case of offer letter, watermark prove that the document is official document of company meant for official work) or authenticity (in case of Rs 100, watermark rule out the forgery and authenticate the piece of paper of its worth).
The watermark on the Rs100 (Figure2.1), just like most paper watermarks today, has two properties. First, the watermark is hidden from view during normal use, only becoming visible as a result of a special viewing process (in this case, holding the bill up to the light). Second, the watermark carries information about the object in which it is hidden (in this case, the watermark indicates the authenticity of the bill).
Watermark
Fig 2.1 Image showing an INR 100 note having watermark at its left side which is considerably visible when note hold under light.
In addition to paper, watermarking can be applied to other physical objects and to electronic signals. Fabrics, garment labels, and product packaging are examples of physical objects that can be watermarked using special invisible dyes and inks Electronic representations of music, photographs, and video are common types of signals that can be watermarked. Thus, watermarking is defined as, “the practice of imperceptibly altering a Work to embed a message about that Work.”
Fig 2.2 A Generic Watermarking System
As is clear from the figure, digital watermarking model consist of an embedder and a detector. The embedder takes two inputs. One is the payload we want to embed (the watermark ), and the other is the cover work in which we want to embed the payload. The output of the embedder is typically transmitted or recorded. Later, that Work (or some other Work that has not been through the embedder) is presented as an input to the detector. Most detectors try to determine whether a payload is present, and if so, output the message encoded by it. The watermarking model is analogous to a communication model in which sender encode a message before transmitting it over communication channel and on receiving, receiver decode the encoded message.
2.1 History of Watermarking Although the art of papermaking was invented in China over one thousand years earlier, paper watermarks did not appear until about 1282, in Italy. The marks were made by adding thin wire patterns to the paper molds. The paper would be slightly thinner where the wire was and hence more transparent. The meaning and purpose of the earliest watermarks are uncertain. They may have been used for practical functions such as identifying the molds on which sheets of papers were made, or as trademarks to identify the paper maker. On the other hand, they may have represented mystical signs, or might simply have served as decoration. By the eighteenth
century,
watermarks
on paper
made
in Europe
and
America had become more clearly utilitarian. They were used as trademarks, to record the date the paper was manufactured, and to indicate the sizes of original sheets. It was also about this time that watermarks began to be used as anticounterfeiting measures on money and other documents. The
term watermark
seems
to have
been coined
near the
end of the eighteenth century and may have been derived from the German term wassermarke (though it could also be that the German word is derived from the English). The term is actually a misnomer, in that water is not especially important in the creation of the mark. It was probably given because the marks resemble the effects of water on paper. About the time the term watermark was coined, counterfeiters began developing methods of forging watermarks used to protect paper money. Counterfeiting prompted advances in watermarking technology. William Congreve, an Englishman, invented a technique for making color watermarks by inserting dyed material into the middle of the paper during papermaking. The resulting marks must have been extremely difficult to forge, because the Bank of England itself declined to use them on the grounds that they were too difficult to make. A more practical technology was invented by another Englishman, William Henry Smith. This replaced the fine wire patterns used to make earlier marks with a sort of shallow relief sculpture, pressed into the paper mold. The resulting variation on the surface of the mold produced beautiful watermarks with varying shades of gray. This is the basic technique used today for the face of President Jackson on the $20 bill. Four hundred years later, in 1954, Emil Hembrooke of the Muzak Corporation filed a patent for “watermarking” musical Works. An identification code was inserted in music by
intermittently applying a narrow notch filter centered at 1 kHz. The absence of energy at this frequency indicated that the notch filter had been applied and the duration of the absence used to code either a dot or a dash. The identification signal used Morse code. It is difficult to determine when digital watermarking was first discussed. In 1979, Szepanski described a machine-detectable pattern that could be placed on documents for anti-counterfeiting purposes.
Nine years later, Holt described a method for embedding an
identification code in an audio signal. However, it was Komatsu and Tominaga, in 1988, which appear to have
first used the term digital watermark. Still, it was probably not until the early
1990s that the term digital watermarking really came into vogue. About 1995, interest in digital watermarking began to mushroom. In addition, about this time, several organizations began considering watermarking technology for inclusion in various standards. The Copy Protection Technical Working Group (CPTWG) tested watermarking systems for protection of video on DVD disks. The Secure Digital Music Initiative (SDMI) made watermarking a central component of their system for protecting music. Two projects sponsored by the European Union, VIVA [110] and Talisman, tested watermarking for broadcast monitoring. The International Organization for Standardization (ISO) took an interest in the technology in the context of designing advanced MPEG standards. In the late 1990s several companies were established to market watermarking products. Technology from the Verance Corporation was adopted into the first phase of SDMI and was used by Internet music distributors such as Liquid Audio.
In
the
area
of
image
watermarking,
Digimarc
bundled
its
watermark
embedder and detectors with Adobe’s Photoshop. More recently, a number of companies have used watermarking technologies for a variety of applications.
2.2 Requirements for watermarking algorithms: A watermarking algorithm should be consistent over following properties and parameters:
Transparency: The most fundamental requirement for any Watermarking method shall be such that it is transparent to the end user. The watermarked content should be consumable at the intended user device without giving annoyance to the user. Watermark only shows up at the watermark-detector device.
Security: Watermark information shall only be accessible to the authorized parties. Only authorized parties shall be able to alter the Watermark content. Encryption can be used to prevent unauthorized access of the watermarked data
Ease of embedding and retrieval: Ideally, Watermarking on digital media should be possible to be performed “on the fly”. The computation need for the selected algorithm should be minimum.
Robustness: Watermarking must be robust enough to withstand all kinds for signal processing operations, “attacks” or unauthorized access. Any attempt, whether intentional or not, that has a potential to alter the data content is considered as an attack. Robustness against attack is a key requirement for Watermarking and the success of this technology for copyright protection depends on this.
Effect on bandwidth: Watermarking should be done in such a way that it doesn’t increase the bandwidth required for transmission. If Watermarking becomes a burden for the available bandwidth, the method will be rejected.
Interoperability: Digitally watermarked content shall still be interoperable so that it can be seamlessly accessed through heterogeneous networks and can be played on various playout devices that may be watermark aware or unaware.
2.3 Importance of Digital Watermarking The sudden increase in watermarking interest is most likely due to the increase in concern over copyright protection of content. The Internet had become user friendly with the introduction of Marc Andreessen’s Mosaic web browser in November 1993, and it quickly became clear that people wanted to download pictures, music, and videos. The Internet is an excellent distribution system for digital media because it is inexpensive, eliminates warehousing and stock, and delivery is almost instantaneous. However, content owners (especially large Hollywood studios and music labels) also see a high risk of piracy. This risk of piracy is exacerbated by the proliferation of high-capacity digital recording devices. When the only way the average customer could record a song or a movie was on analog tape, pirated copies were usually of a lower quality than the originals, and the quality of second-generation pirated
copies (i.e., copies of a copy) was generally very poor. However, with digital recording devices, songs and movies can be recorded with little, if any, degradation in quality. Using these recording devices and using the Internet for distribution, would-be pirates can easily record and distribute copyright-protected material without appropriate compensation being paid to the actual copyright owners. Thus, content owners are eagerly seeking technologies that promise to protect their rights. The first technology content owners turn to is cryptography. Cryptography is probably the most common method of protecting digital content. It is certainly one of the best developed as a science. The content is encrypted prior to delivery, and a decryption key is provided only to those who have purchased legitimate copies of the content. The encrypted file can then be made available via the Internet, but would be useless to a pirate without an appropriate key. Unfortunately, encryption cannot help the seller monitor how a legitimate customer handles the content after decryption. A pirate can actually purchase the product, use the decryption key to obtain an unprotected copy of the content, and then proceed to distribute illegal copies. In other words, cryptography can protect content in transit, but once decrypted, the content has no further protection. Thus, there is a strong need for an alternative or complement to cryptography: a technology that can protect content even after it is decrypted. Watermarking has the potential to fulfill this need because it places information within the content where it is never removed during normal usage. Decryption, reencryption, compression, digital-to-analog conversion, and file format changes—a watermark can be designed to survive all of these processes. Watermarking has been considered for many copy prevention and copyright protection applications. In copy prevention, the watermark may be used to inform software or hardware devices that copying should be restricted. In copyright protection applications, the watermark may be used to identify the copyright holder and ensure proper payment of royalties. Although copy prevention and copyright protection have been major driving forces behind research in the watermarking field, there is a number of other applications for which watermarking has been used or suggested. These include broadcast monitoring, transaction tracking, authentication (with direct analogy to our Rs100 example), copy control, and device control.
2.4 Applications of Digital Watermarking Digital Watermarks are potentially useful in many applications, including:
Ownership assertion:
Watermarks can be used for ownership assertion. To assert
ownership of an image, Alice can generate a watermarking signal using a secret private key, and then embed it into the original image. She can then make the watermarked image publicly available. Later, when Bob contends the ownership of an image derived from this public image, Alice can produce the unmarked original image and also demonstrate the presence of her watermark in Bob’s image. Since Alice’s original image is unavailable to Bob, he cannot do the same. For such a scheme to work, the watermark has to survive image processing operations aimed at malicious removal. In addition, the watermark should be inserted in such a manner that it cannot be forged as Alice would not want to be held accountable for an image that she does not own.
Fingerprinting: In applications where multimedia content is electronically distributed over a network, the content owner would like to discourage unauthorized duplication and distribution by embedding a distinct watermark (or a fingerprint) in each copy of the data. If, at a later point in time, unauthorized copies of the data are found, then the origin of the copy can be determined by retrieving the fingerprint. In this application the watermark needs to be invisible and must also be invulnerable to deliberate attempts to forge, remove or invalidate. Furthermore, and unlike the ownership assertion application, the watermark should be resistant to collusion. That is, a group of k users with the same image but containing different fingerprints should not be able to collude and invalidate any fingerprint or create a copy without any fingerprint.
Copy prevention or control. Watermarks can also be used for copy prevention and control. For example, in a closed system where the multimedia content needs special hardware for copying and/or viewing, a digital watermark can be inserted indicating the number of copies that are permitted. Every time a copy is made the watermark can be modified by the hardware and after a point the hardware would not create further copies
of the data. An example of such a system is the Digital Versatile Disc (DVD). In fact, a copy protection mechanism that includes digital watermarking at its core is currently being considered for standardization and second generation DVD players may well include the ability to read watermarks and act based on their presence or absence. Another example is in digital cinema, where information can be embedded as a watermark in every frame or a sequence of frames to help investigators locate the scene of the piracy more quickly and point out weaknesses in security in the movie’s distribution. The information could include data such as the name of the theater and the date and time of the screening. The technology would be most useful in fighting a form of piracy that’s surprisingly common, i.e., when someone uses a camcorder to record the movie as it’s shown in a theater, then duplicates it onto optical disks or VHS tapes for distribution.
Fraud and tamper detection. When multimedia content is used for legal purposes, medical applications, news reporting, and commercial transactions, it is important to ensure that the content was originated from a specific source and that it had not been changed, manipulated or falsified. This can be achieved by embedding a watermark in the data. Subsequently, when the photo is checked, the watermark is extracted using a unique key associated with the source, and the integrity of the data is verified through the integrity of the extracted watermark. The watermark can also include information from the original image that can aid in undoing any modification and recovering the original. Clearly a watermark used for authentication purposes should not affect the quality of an image and should be resistant to forgeries. Robustness is not critical as removal of the watermark renders the content inauthentic and hence of no value.
ID card security. Information in a passport or ID (e.g., passport number, person’s name, etc.) can also be included in the person’s photo that appears on the ID. By extracting the embedded information and comparing it to the written text, the ID card can be verified. The inclusion of the watermark provides an additional level of security in this application. For example, if the ID card is stolen and the picture is replaced by a forged copy, the failure in extracting the watermark will invalidate the ID card. The above
represent a few example applications where digital watermarks could potentially be of use. In addition there are many other applications in rights management and protection like tracking use of content, binding content to specific players, automatic billing for viewing content, broadcast monitoring etc. From the variety of potential applications exemplified above it is clear that a digital watermarking technique needs to satisfy a number of requirements. Since the specific requirements vary with the application, watermarking techniques need to be designed within the context of the entire system in which they are to be employed. Each application imposes different requirements and would require different types of invisible or visible watermarking schemes or a combination thereof. In the remaining sections of this chapter we describe some general principles and techniques for invisible watermarking. Our aim is to give the reader a better understanding of the basic principles, inherent trade-offs, strengths, and weakness, of digital watermarking. We will focus on image watermarking in our discussions and examples. However as we mentioned earlier, the concepts involved are general in nature and can be applied to other forms of content such as video and audio.
Broadcasting Monitoring: Commercials are aired by broadcasting channels and stations. For this advertising firm purchase airtime from broadcasting channel. There are several organizations and individuals interested in broadcasting monitoring, viz. advertiser, who want to ensure if his commercial is broadcasted for all of his purchased airtime, performers, who want to ensure that they get the royalties due to them from advertising firm and owners of copyrighted works, who want to ensure that their property is not illegally rebroadcasted by pirate stations. One solution to the problem is human observers watching the broadcasting which is neither a feasible nor practically possible solution. The other solution is to match the signal with the signals present in databases to ascertain advertisers that messages are broadcasted. But matching signals from databases is very complex process and require large amount of time and money. The last solution is using watermarking techniques. It has advantage of existing within content itself, rather than exploiting a particular segment of the broadcast signal, and is therefore completely compatible with the installed base of broadcast equipment, including both digital and analog transmission.
2.5 Attacks on Watermarked Work Below are some significant known attacks: 2.5.1 Scrambling Attacks: A scrambling attack is a system-level attack in which the samples of a Work are scrambled prior to presentation to a watermark detector and then subsequently descrambled. The type of scrambling can be a simple sample permutation or a more sophisticated pseudo-random scrambling of the sample values. The degree of scrambling necessary depends on the detection strategy. A well-known scrambling attack is the mosaic attack, in which an image is broken into many small rectangular patches, each too small for reliable watermark detection. These image segments are then displayed in a table such that the segment edges are adjacent. The resulting table of small images is perceptually identical to the image prior to subdivision. This technique can be used in a web application to evade a web-crawling detector. The scrambling is simply the subdivision of the image into sub images, and the descrambling is accomplished by the web browser itself.
2.5.2 Pathological Distortions: For a watermark to be secure against unauthorized removal, it must be robust to any process that maintains the fidelity of the Work. This process may be a normal process, in which case we are requiring that a secure watermark be robust. However, it may also be a process unlikely to occur during the normal processing of the Work. Any process that maintains the fidelity of the Work could be used by an adversary to circumvent the detector by masking or eliminating the watermark. The two most common categories of such pathological distortions, geometric/temporal distortions (attacks on synchronization) and noise removal distortions 2.4.2.1 Synchronization Attacks: Many watermarking techniques are sensitive to synchronization. By disturbing this synchronization, an adversary attempts to mask the watermark signal. Examples of simple synchronization distortions include delay and time scaling for audio and video, and rotation, scaling, and translation for images and video. These simple distortions can be implemented such that they vary over time
or space. More complex distortions include pitch-preserving scaling and sample removal in audio, and shearing, horizontal reflection, and column or line removal in images. Even more complex distortions are possible, such as nonlinear warping of images. 2.4.2.2 Linear filtering and noise removal attack: Linear filtering also can be used by an adversary in an attempt to remove a watermark. For example, a watermark with significant energy in the high frequencies might be degraded by the application of a low-pass filter. In addition, any watermarking system for which the added pattern is “noiselike” is susceptible to noise-removal techniques.
2.5.3 Copy Attacks: A copy attack occurs when an adversary copies a watermark from one Work to another. As such, it is a form of unauthorized embedding. The copy attack attempts to thwart the effectiveness of such systems by estimating the watermark given in an originally watermarked piece of media, and then adding that watermark to an unwatermarked piece. In the first scenario listed above, this would allow an attacker to have an inauthentic image be declared authentic, since it contains a watermark. In the second scenario, an attacker could flood the market with content which ordinarily would allow a user to manipulate it as he saw fit, but due to the presence of the watermark, limitations would be imposed. In this way, schemes which sought to limit use of watermarked media may prove to be too unpopular for wide distribution.
2.5.4 Ambiguity Attacks: Ambiguity attacks create the appearance that a watermark has been embedded in a Work when in fact no such embedding has taken place. An adversary can use this attack to claim ownership of a distributed Work. He or she may even be able to make an ownership claim on the original Work. As such, ambiguity attacks can be considered a form of unauthorized embedding. However, they are usually considered system attacks.
Chapter 3
Methodology
3.1 Project Flow The following diagram depicts the complete flow of project with the respective modules or function names: 3.1.1 Insertion of Watermark Image into Cover Work: -
Fig 3.1 Flowchart showing Insertion of Watermark into Cover Work.
3.1.2 Extraction of Watermark Image from Watermarked Image: -
Fig 3.2 Flowchart showing extraction of watermark from watermarked image
3.1.3 Calculation of Detection ratio: -
Fig 3.3 Flowchart showing steps to calculate detection ratio
3.2 Step By Step procedure The following is the step by step procedure with detailed description is as follows:
3.2.1 Read Cover Image: The first step is to read the cover image (in our case we used a .jpeg image) and process the image so that SIFT can be applied over the cover image. The .jpeg image, which is originally RGB, is converted to gray image (pixel values belong to the set [0, 255].Further the image matrix is divided by 256 so as to bring the values in range of [0, 1].After these steps the image is ready for the SIFT to be operated over it to find all the invariant patches.
3.2.2 SIFT: Scale-invariant feature transform (or SIFT) is an algorithm in computer vision to detect and describe local features in images. The algorithm was published by David Lowe in 1999. For any object in an image, interesting points on the object can be extracted to provide a "feature description" of the object. This description, extracted
from a training image, can then be used to identify the object when attempting to locate the object in a test image containing many other objects. It is important that the set of features extracted from the training image is robust to changes in image scale, noise, illumination, and local geometric distortion to perform reliable recognition. Lowe's patented method can robustly identify objects even among clutter and under partial occlusion, because his SIFT feature descriptor is invariant to scale, orientation, and affine distortion, and partially invariant to illumination changes. The following is an image (Figure 3.1(a)) on which sift is applied to find keypoints. Initially (Figure 3.1(b)) 832 keypoints locations at maxima and minima of the difference-of-Gaussian function. After applying a threshold on minimum contrasts, 729 keypoints remain (Figure 3.1(c)). Figure3.1 (d) shows image with 536 keypoints that remained following an additional threshold.
Fig 3.4. This figure shows the stages of keypoints selection. (a) The 233x189 pixel original image. (b) The initial 832 keypoints locations at maxima and minima of the DoG function. (c) After applying a threshold on minimum contrast, 729 keypoints remain. (d) The final 536 keypoints that remain following an additional threshold.
The selection of features is important for robust watermarking in content-based synchronization methods. We believe that local image characteristics are more useful than global ones. The scale-invariant feature transform, SIFT, extracts features by considering local image properties and is invariant to rotation, scaling, translation, and partial illumination changes. We implemented a proposed watermarking method, using the SIFT, that is robust to geometric distortions. Using the SIFT, we generate circular patches that are invariant to translation and scaling distortions. The watermark is inserted into the circular patches in an additive way in the spatial domain. Rotation invariance is achieved using the translation property of the polar-mapped circular patches. We have performed an intensive simulation to show the robustness of the proposed method with 25 test images. The simulation results confirm that our method is robust against geometric distortion attacks as well as signal-processing attacks.
3.2.2.1 The General Mathematical model of SIFT: Considering local image characteristics, the SIFT descriptor extracts features and their properties, such as the location (t1, t2), the scale s, and the orientation. The SIFT was proposed by Lowe and has proved to be invariant to image rotation, scaling, translation, partial illumination changes, and projective transformations. Considering local image characteristics, the SIFT descriptor extracts features and their properties, such as the location (t1, t2), the scale s, and the orientation. The basic idea of the SIFT is to extract features through a staged filtering that identifies stable points in the scale space: 1. Select candidates for features by searching for peaks in the scale space of the difference-of-Gaussians (DoG) function. 2. Localize each feature using measures of its stability. 3. Assign orientations based on local image gradient directions.
In order to extract candidate locations for features, the scale space D(x, y,) is computed using a DoG function. As shown in Fig. 3.2, they successively
smooth original images with a variable-scale (1, 2, and 3) Gaussian function and calculate the scale-space images by subtracting two successive smoothed images. The parameter is a variance (called a scale) of the Gaussian function. The scale of the scale-space images is determined by the nearby scale (1, 2, or 3) of the Gaussian-smoothed image. In these scale-space images, they retrieve all local maxima and minima by checking the closest eight neighbors in the same scale and nine neighbors in the scales above and below. These extrema determine the location (t1, t2) and the scale s of the SIFT features, which are invariant to the scale and orientation change of images. In our experiment, to generate the scalespace images, we apply scales of the Gaussian function from 1.0 to 32.0 and increase the scale by multiplying by a constant factor 2.
Fig 3.5 Scale space by using Difference-of-Gaussian (DoG) function and neighbor of a pixel.
After candidate locations have been found, a detailed model is fitted by a 3-D quadratic function to determine accurately the location (t1, t2) and scale s of each feature. In addition, candidate locations that have a low contrast or are poorly localized along edges are removed by measuring the stability of each feature using a 2-by-2 Hessian matrix H as follows:
Here e is the ratio of the largest to the smallest Eigen value and is used to control stability. They use e=10. The quantities Dxx, Dxy, and Dyy are the derivatives of the scale space images. In order to achieve invariance to image rotation, they assign a consistent orientation to each feature. In the Gaussian-smoothed image with the scale of the extracted SIFT features, they calculate gradient orientations of all sample points within a circular window about a feature location and form an orientation histogram. The peak in this histogram corresponds to the dominant direction of that feature.
3.2.2.2 The Modification in Mathematical model of SIFT for Watermarking: The local features from the SIFT descriptor are not directly applicable to watermarking, because the number and distribution of the features are dependent on image contents and textures. Moreover, the SIFT descriptor was originally devised for image-matching applications, so it extracts many features that have dense distribution over the whole image. Therefore, we adjust the number, distribution, and scale of the features and remove those features that are susceptible to watermark attacks. The SIFT descriptor extracts features with such properties as their location (t1, t2), scale s, and orientation. In practice, the orientation property of the SIFT descriptor of the original image and distorted images do not match precisely. Hence, we make a circular patch by using only the location (t1, t2) and scale s of extracted SIFT features, as follows:
where k is a magnification factor to control the radius of the circular patches. The way in which this factor is determined is explained in the fourth paragraph of this subsection. These patches are invariant to image scaling and translation as well as spatial modifications. By applying a prefilter, such as a Gaussian filter, before feature extraction, we can reduce the interference of noise and increase the robustness of extracted circular patches. The scale of features derived from the SIFT descriptor is related to the scaling factor of the Gaussian function in the scale space. In our analysis, features whose scale is small have a low probability of being redetected, because they disappear easily when image contents are modified. Features whose scale is large also have a low probability of being redetected in distorted images, because they move easily to other locations. Moreover, using large-scale features means that patches will overlap each other, which will result in degradation of the perceptual quality of watermarked images. Therefore, we remove features whose scale is below a or above b. In our experiments, we set a and b at 2.0 and 10.0, respectively. The SIFT descriptor considers image contents so that extracted SIFT features have different properties in the scale s, depending on image contents. Watermark insertion and detection necessarily require interpolation to transform the rectangular watermark to match the shape of patches, and vice versa. In order to minimize the distortion of the watermark through interpolation, the size (radius) of the patches must be similar to, or larger than, the size of the watermark. The scale s of extracted SIFT features varies from 2.0 to 10.0. Therefore, we divide the scale of features into two ranges and apply different magnification factors k1 and k2, which are determined empirically on the assumption that the size of the watermarked images will not be changed excessively. Although the features whose size is near to the boundary of the range may be susceptible to scaling attacks, there are a number of circular patches in an image, so that the effect of these features on the watermarking is small. The distribution of local features is related to the performance of watermarking systems. In other words, the distance between adjacent features must be determined carefully. If the distance is small, patches will overlap in large areas, and if the distance is large, the number of patches will not be
sufficient for the effective insertion of the watermark. To control the distribution of extracted features, we apply a circular neighborhood constraint in which the features whose strength is the largest are used to generate circular patches. The value from the DoG function is used to measure the strength of each feature. The distance D between adjacent features depends on the dimensions of the image and is quantized by the r value as follows:
The width and height of the image are denoted by w and h, respectively. The r value is a constant to control the distance between adjacent features and is set at 16 and 32 in the insertion and detection processes, respectively. Figure 2 shows a circular patch from our proposed synchronization method in spatial filters, additive uniform noise, rotation, and scaling of the image. For convenience of identification, we represent only one patch and find that the patch is formulated robustly, even when the image is distorted.
Fig 3.6Circular patch from our proposed method in (a) the original image, (b) the mean-filtered image, (c) the median-filtered image, (d) the additive uniform noise image, (e) the 10-deg rotated image, and (f) the 1.2x scaled image.
3.2.3 Watermark Generation: We generate a 2-D rectangular watermark that follows a Gaussian distribution, using a random number generator. To be inserted into circular patches, this watermark should be transformed so that its shape is circular. We consider the rectangular watermark to be a polar-mapped watermark and inversely polar-map it to assign the insertion location of the circular patches. In this way, a rotation attack is mapped as a translation of the rectangular watermark, and the watermark still can be detected using the correlation detector. Note that the size of circular patches differs, so we should generate a separate circular watermark for each patch.
Let the x and y dimensions of the rectangular watermark be denoted by M and N, respectively. Let r be the radius (size) of a circular patch. As shown in Fig. 3, we divide a circular patch into homocentric regions.
Fig 3.7 Polar mapping between rectangular watermark and the circular watermark.
To generate the circular watermark, the x- and the y-axes of the rectangular watermark are inversely polar-mapped into the radius and angle directions of the patch. The relation between the coordinates of the rectangular watermark and the circular watermark is represented as follows:
where x and y are the rectangular watermark coordinates, ri and are the coordinates of the circular watermark, rM is equal to the radius of the patch, and r0 is a fixed fraction of rM. In our project, we set r0 to rM /4. For effective transformation, r0 should be larger than M /, and the difference between rM and r0 should be larger than N. If these constraints are not satisfied, the rectangular watermark must be sampled. As a result, it is difficult to transform efficiently. To increase the robustness and invisibility of the inserted watermark, we transform the rectangular watermark to be mapped to only the upper half of the patch, i.e., the y-axis of the rectangular watermark is scaled by the angle of a half circle (), not the angle of a full circle (2 ). The lower half of the patch is set symmetrically with respect to the upper half (see Fig. 3). In aspect of the image, watermarks constitute a kind of noise. When noise of similar strength gathers together, we can perceive it. In our scheme, a pixel in the rectangular watermark is mapped to adjacent several pixels in the circular watermark during polar mapping. In other words, the same noise is inserted into the homocentric region of a circular patch. Therefore, if the size of the homocentric region is large, the inserted watermark is visible (as an embossing effect). Through symmetrical mapping, we can make the size of the homocentric region small and thus render the watermark
invisible. Moreover, we can increase the likelihood that the watermark will survive attacks such as cropping.
3.2.4 Watermark Insertion: The first step in watermark insertion is to analyze image contents to extract the patches. Then, the watermark is inserted repeatedly into all patches. Our watermark insertion process is shown in Fig. 4(A). Step a: To extract circular patches, we use the SIFT descriptor, as explained in Sec.3.2.2. A single image may contain a number of patches. We insert the watermark into all patches to increase the robustness of our scheme. Step b.1.We generates a circular watermark dependent on the radius (size) of each patch, using the method described in Sec.3.2.3. We have endeavored to construct the patches so that their radius is similar to, or larger than, the x and y sizes of the rectangular watermark; thus, during extraction of the patches, a pixel w in the rectangular watermark is mapped to several pixels wc in the circular watermark. This compensates for errors in alignment of the circular patches regarding location and scale during watermark detection. Step b.2. The insertion of the watermark must not affect the perceptual quality of images. This constraint has a bearing on the insertion strength of the watermark, inasmuch as it must be imperceptible to the human eye. We apply the perceptual mask as follows:
where is the lower bound of visibility in flat and smooth regions and is the upper bound in edged and texture regions. The noise visibility function is calculated as follows:
where 2x(ij) and 2 x max denote the local variance and maximum of neighboring pixels within five pixels, and D is a scaling constant.
Fig 3.8 Diagrammatic representation of Watermark insertion scheme
Step b.3. Finally, we insert this circular watermark additively into the spatial domain. The insertion of the watermark is represented as the spatial addition between the pixels of images and the pixels of the circular watermark as follows:
Here vi and wci denote the pixels of images and of the circular watermark, respectively, and denotes the perceptual mask that controls the insertion strength of the watermark.
3.2.5 Watermark Detection: Similarly to watermark insertion, the first step for watermark detection is analyzing the image contents to extract patches. The watermark is then detected from the patches. If the watermark is detected correctly from at least one patch, we can prove ownership successfully. Our watermark detection process is shown in Fig. 4(B). Step a. To extract circular patches, we use the SIFT descriptor, as described in Sec. 2. There are several patches in an image, and we try to detect the watermark from all patches. Step b.1. The additive watermarking method in the spatial domain inserts the watermark into the image contents as noise. Therefore, we first apply a Wiener filter to extract this noise by calculating the difference between the watermarked image and its Wienerfiltered image, and then regard that difference as the retrieved watermark.6 As with the watermark insertion process, we compensate for the modification by perceptual masks, but such compensation does not greatly affect the performance of watermark detection. Step b.2. To measure the similarity between the reference watermark generated during watermark insertion and the retrieved watermark, the retrieved circular watermark should be converted into a rectangular watermark by applying the polar-mapping introduced in Sec3.2.4. Considering the fact that the watermark is inserted symmetrically, we take the mean value from the two semicircular areas. By this mapping, the rotation of circular patches is represented as a translation, and hence we achieve rotation invariance for our watermarking scheme.
Fig 3.9 Diagrammatic representation of Watermark extraction scheme
Step b.3. We apply circular convolution to the reference watermark and the retrieved watermark. The degree of similarity between the two, called the response of the watermark detector, is represented by the maximum value of circular convolution as follows:
where w is the reference watermark and w* is the retrieved watermark. The range of similarity values is from −1.0 to 1.0. We can identify the rotation angle (/ r) of the patches by finding the r with the maximum value. If the similarity exceeds a predefined threshold, we can be satisfied that the reference watermark has been inserted. The method of determining the threshold is described in the following section.
Step c. As mentioned, there are several circular patches in an image. Therefore, if the watermark is detected from at least one patch, ownership is proved, and not otherwise. The fact that we insert the watermark into several circular patches, rather than just one, makes it highly likely that the proposed scheme will detect the watermark, even after image distortions. Our watermarking scheme is robust against geometric distortion attacks as well as signal-processing attacks. Scaling and translation invariance is achieved by extracting circular patches from the SIFT descriptor. Rotation invariance is achieved by using the translation property of the polar mapped circular patches.
3.2.6 Detection ratio: This module compares the patches in both original image as well as attacked image. During watermark detection, the redetection of patches that have been extracted during watermark insertion is important for robustness. In order to mea sure the redetection ratio, we first extracted circular patches from both the original image and the attacked images, and then compared the locations and radii of the patches from the original image with those of the patches from the attacked images. Prior to the comparison, we transformed the locations and radii of circular patches from images subjected to geometric attack to those of patches in the original image. If the difference between circular patches from the original image and those from attacked images was below 2 pixels, we regarded the patches as having been redetected correctly. We applied various attacks: median filter [2x2], [3x3], and [4x4], JPEG compression quality factor 40, 60, 90, Gaussian filtering [3x3], and scaling 0.7x, 0.8x, 0.9x, 1.1x, 1.2x and 1.3x. Detection ratio refers to the ratio of the number of extracted patches from attacked images to the number of correctly redetected patches from original image. Detection failure refers to the number of images in which no patch is redetected.
Chapter 4
RESULTS
4.1 Test Images: 4.1.1 Cover work:
Fig 4.1 Original Cover Image
4.1.2 Position of Invariant patches in cover work:
Fig 4.2 Invariant patches present in original cover work
4.1.3 Watermark image:
Fig 4.3 Watermark image (72x72 pixels)
4.1.4 Cover Image with watermark:
Fig 4.4 Watermarked Image (Invisible)
4.1.5 Retrieved Watermark:
Fig 4.5 Retrieved Watermark Image
4.2 Performance of watermarking scheme and test results: We tested the performance of our watermarking scheme. The proposed scheme is tested for 30 random images for over 7500 patches. Each image used for testing is a .jpeg image of size 150x200 pixels. The size of rectangular watermark we took for testing the scheme is 72x72 pixels and the weighing factors and of noise visibility function was set to 5.0 and 1.0 respectively. We were able to get a detection ratio as high as approx. 85% under various type of attacks listed in Stirmark 3.1: median filter, JPEG compression, Gaussian filtering, scaling. The simulation results under attack are shown in Table 4.1. We take into consideration a pixel error of 2 pixels while redetecting patches, that is if the position of any patch is shifted by +/- 2 pixels and radius of patch is same we consider patch to be redetected correctly. Detection ratio refer to the ratio of the number of patches correctly redetected from watermarked image to total number of patches in which watermark inserted. Among 30 images, detection failure refers to the number of images where inserted watermark could not be detected with minimum similarity to prove ownership and hence we fail to prove ownership in those images. Similarity is the average of similarity values from correctly detected watermark patches. 4.2.2 Test results Table 4.1 Fraction of correctly detected watermark patches, number of failure images, and similarity under common signal-processing and geometric attacks
Signal Processing and Geometric Attacks Watermarked image(no attack) Median Filter 2x2 Median Filter 3x3 Median Filter 4x4 Gaussian Filter Scaling 1.1x Scaling 1.2x Scaling 1.3x Scaling .9x Scaling .8x Scaling .7x JPEG Compression 90 JPEG Compression 60
Detection Ratio
Detection Failure
Similarity
95.231%
0
0.635
94.917% 81.943% 56.837% 86.871% 97.823% 96.363% 95.155% 66.231% 34.582% 16.750% 92.661% 79.905%
0 0 0 0 0 0 0 0 1 2 0 2
0.631 0.616 0.595 0.603 0.614 0.609 0.583 0.603 0.508 0.417 0.589 0.497
Chapter 5
Conclusion and Future Work
The result set as shown in Table 4-1, clearly depicts the inserted watermark is successfully detected even after using additive watermarking method in spatial domain, which is very unlikely as in case of attacks pixel values may change abnormally and it may be difficult to recover the original pixel value and thus watermark may be lost. Our major contribution is that we have proposed a robust watermarking scheme that uses local invariant features. In order to resist geometric distortions, we extracted circular patches using the SIFT descriptor, which is invariant to translation and scaling distortion. These patches were watermarked additively in the spatial domain. Rotation invariance was achieved using the translation property of the polar-mapped circular patches. We performed an intensive simulation, and the results showed that our method would be robust against geometric distortion attacks as well as signal-processing attacks. We believe that the consideration of local features is important for the design of robust watermarking schemes, and our method is a solution that uses such features Drawbacks of the proposed watermarking scheme are related to its vulnerability to large distortion of the aspect ratio. In addition, due to the computation time for the SIFT descriptor and for the compensation of alignment errors, our scheme cannot be used effectively in real-time applications. Future work will focus on eliminating those drawbacks. In future this work can also be extended for inserting watermark on videos. A Video is a sequence of images, called as frames, and these frames are more or less same with slight changes and thus position of patches may not vary considerably in adjacent patches. This property may be exploited to derive a new technique for watermarking on videos. The time complexity may be an issue as a lot of computation needs to be done and computational time for SIFT descriptor is already very large.
Appendix A The following are the signature and brief description of functions used during implementation of the proposed scheme: 1. processWatermark:
Input parameters: o watermarkFileName o radius
Output parameters: o W: processed watermark image matrix o P: radius matrix of circular watermark o Q: theta matrix of circular watermark
Description: This module performs the polar mapping of rectangular watermark to circular watermark.
2. insertWatremark: Input parameters: o I: cover work o W: processed Watermark o X: x-co-ordinate o Y: y-co-ordinate Output parameters: o I: watermarked image Description: This module inserts watermark into image. 3. extractWatermark:
Input parameters: o I: watermarked Image o I1: original cover work o X: x-co-ordinate o Y: y-co-ordinate o [a,b]:size of watermark
Output parameters: o RW: retrieved watermark
Description: This module extract watermark from watermarked image.
4. compareWatermark:
Input parameters: o W: reference watermark o RW: retrieved watermark o Diff: Pixel Error
Output parameters: o Ratio: similarity ratio.
Description: This module compares the retrieved watermark with reference watermark and gives the similarity as a ratio.
5. calculateDetectionratio:
Input parameters: o frame: sift frame matrix for original cover work o f1: sift frame matrix for attacked image o diff: pixel error
Output parameters: o ratio: detection ratio
Description: This module calculates the detection ratio for the attacked image.
References [1] H.Y. lee, H.S. Kim and H.K. Lee, “Robust image watermarking using local invariant features” optical engineering vol45 (3)(2006)037002(page 1-11) [2] D.G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints” International Journal of Computer Vision 60(2) (2004) page 91-110. [3] Leida LI, Xiaoping YUAN, Zhaolin LU, Jeng-Shyang PAN “Rotation Invariant Watermark Embedding Based on Scale-Adapted Characteristic Regions” Informational Sciences (November 2009) page 1-22. [4] Xia-mu Niu, Zhe-ming Lu, Sheng-ho Sun, “Digital Watermarking of Still Images with GrayLevel Digital Watermarks” IEEE transaction on Consumer Electronics, Vol. 46, No.1 (February 2000) page 137-145. [5]Patrick Bas, Jean-Marc Chassery, Benoit Macq, “Geometrically Invariant Watermarking Using Feature points” IEEE Transactions on Image Processing, VOL.11, NO.9 (September 2002) page 1014-1028. [6]Yanqun Zhang, “Digital Watermarking Technology: A Review”2009 ETP International Conference on Future Computer and Communication page- 250-252. [7]Yiwei Wang, John F. Doherty, Robert E. Van Dyck, “A Wavelet-Based Watermarking Algorithm for Ownership Verification of Digital Images” IEEE Transactions on Image Processing, Vol.11, NO.2 (February 2002) page 77-86. [8]A. Nikolaidis and I. Pitas, “Region-based Image Watermarking” IEEE Transactions on Image Processing, VOL.10, NO.11 (2001) page 1726-1740. [9] Book on Digital Watermarking and Steganography by Ingemar J. Cox and Matthew L. Miller, The Morgan Kaufmann Series in Multimedia Information and Systems, Second Edition.
