How to Detect Image Manipulations? Part II

The Clone Detection Tool

Another tool available on the website “Forensically” (https://29a.ch/photo-forensics/#clone-detection) helps with the detection of clones. Like the Error Analysis Tool, which was discussed earlier on this blog, the Clone Detection Tool includes a set of levels that enable the user to identify manipulated areas in images.

 

Before discussing this tool, we need to have a closer look at another instrument – the “clone stamp” in Adobe Photoshop. This instrument makes it easy to significantly alter images in a way that the average viewer tends not to recognize. The clone stamp makes it possible to reproduce and to overwrite the “natural” texture of a certain area in an image. Detecting image manipulations in all kinds of images (e.g., artistic or scientific) not only requires a basic understanding of the tools that are designed for detection, but of the tools and practices that made the manipulation possible. In other words, whoever wants to detect manipulations must be able to produce them.

This is why I include three examples of how to work with the clone stamp:

 

Erasing Artifacts with the Clone Stamp in Adobe Photoshop

 

Example 1:

 

Flower_Orig

 

Fig. 1: Blossom and Bee

The idea of the first experiment is to make this picture a little more dramatic by erasing elements surrounding the blossom on the left and on the right.

 

Example 2:

 

Schaffhausen_Orig

 

Fig. 2: Schaffhausen Boat Scene

The second image shows a boat near the Rheinfall in Schaffhausen. I decided to let the boat disappear completely using the clone tool.

 

Example 3:

 

Shadow_orig

 

Fig. 3: Outdoor Scene with Shadows

The third image manipulation is a little more complicated, since it is not trivial to reproduce consistent color flows. The plan is to hide two of the shadows in the foreground (keeping only the shadow in the middle), and to erase the figure in the background.

 

Before I present and discuss the final results of the manipulations, here are some screenshots that document how I altered the images.

 

 

Overview of manipulations:

 

1_Blossom and Bee

 

step1

 

 

Fig. 4: Erasing of background texture

 

step2_detail

 

 

Fig. 5: Reproducing flower petals.

 

 

2_Schaffhausen Boat Scene

 

Schaff_Step1

 

Fig. 6: Cloning the water surface gives the impression as if the boat were sinking.

 

 

Schaff_Step2

 

Fig. 7: Half of the boat vanished.

 

 

3_Outdoor Scene

 

Shadow_Step3

 

Fig. 8: Image with replaced shadows in the foreground.

 

 

 

Shadow_Step4

 

Shadow_Step5

 

Shadow_Step6

 

Figs. 9-11: Cloning the background figure. Because the figure is located in front of different background textures (like the tree, the fence, the rocks and the path), it is crucial to copy information from each of these elements to produce a somewhat natural impression.

 

 

THE CLONE DETECTION TOOL

 

Now, here are the three images after the manipulations:

 

Flower_Manip_Clone

 

Fig. 12: The image has a little less background texture, and duplicating the petals highlights the blossom.

 

 

Schaffhausen_Clone_Result

 

Fig. 13: The boat is gone and the manipulation not easy to trace.

 

 

Shadow_Manip_Result

 

Fig 14: A closer look makes it possible to see the manipulations in the foreground. The lighting on the right appears unnatural and there are visible traces of the clone tool. The erasing of the figure in the background seems less easy to trace.

 

 

Let us now see whether the Clone Detection Analysis Tool can reveal these manipulations. For the purpose of comparison, I analyzed both the cloned and the original image to give an impression of how the results differ.

 

 

Example 1: Blossom and Bee

 

Flower_manip_clonedetect

 

Fig. 15: It turns out that the clone detection tool covers most of the manipulations. The tool highlights the duplicated/cloned flower petals and it reveals some other similarities in background structures (not all of them represent manipulated areas). It does not, however, highlight any of the rather drastic background manipulations (I erased parts of the background on the left and enlarged the dark area on the right – best compared with the original below).

 

 

Here is how the Clone Detection Tool analyzes the unaltered version of the image:

 

Flower_orig_clonedetect

 

Fig 16: Although the tool traces some similarities, the difference between original and manipulated image remains clearly visible. Overall, the tool helps revealing many clones – except for those in the background texture.

 

 

Example 2: Schaffhausen Boat Scene

 

schaff_manip_clonedetect

 

Fig. 17: In analyzing the second image, the analysis highlights certain areas of the image, but since the texture of the water surface is naturally repetitive, it is not easy to tell the manipulated from the untouched parts.

 

 

schaff_orig_clonedetect

 

Fig 18: The comparison is useful in this case, because it shows that the manipulated areas are highlighted in different ways than other highlighted areas. One possible conclusion is: whenever the detection tool highlights a section strongly, this may be interpreted as evidence for cloned areas.

 

 

Example 3: Outdoor Scene with Shadows

 

shadow_clone_detect

 

Fig. 19: The last example shows some of the tool’s limitations. The tool introduces a paradox: it highlights areas in the image that have not been touched, and it does not identify areas that the human eye can easily identify as suspicious.

 

 

shadow_orig_clonedetect

 

Fig. 20: Most of the textures and elements in the image are highly repetitive, which may be a reason why the algorithm does not detect the cloned areas. What is obvious: the clone detection tool does not work well in this case.

 

 

Summary:

 

The help section of the “Forensically” website says: “The clone detector highlights copied regions within an image. These can be a good indicator that a picture has been manipulated.” (https://29a.ch/photo-forensics/#help) As the above experiments have shown, there are clear limitations to the capacities of the tool. Yet none of the tools on “Forensically” claim to reveal all manipulations under all circumstances. They rather promise to make it easier to identify where to look closer.

 

All in all, out of the three examples discussed in this blog post, there is only one in which the tool clearly highlighted repetitive areas (Example 1), another one in which the tool indicated areas within the image that the algorithm marked as suspicious (Example 2), and one in which the algorithm remained oblivious, if not misleading (Example 3).

 

More tools will be evaluated soon – visit headt.eu for upcoming posts.

 

HEADT CENTRE 2017