On Wednesday, July 12 the HEADT Centre opened its doors for the first HEADT Centre Summer Reception. About thirty invited guests from different fields and with various professional backgrounds joined the event, which was organized to offer an opportunity for discussion and exchange with regard to the topic of scholarly integrity.
Among our guests, we welcomed representatives from integrity commissions at universities in Berlin, from agencies of oversight, from science intense industries as well as publishing houses and of course many friends and colleagues from institutes at our university.
The summer reception was the first of a series of events that will be held to establish networks and support collaborations among researchers and professionals in the Berlin region and beyond.
This new video introduces to aspects of measurement when it comes to research integrity violations.
Here is the HEADT Centre’s new YouTube infomercial that introduces to aspects of plagiarism.
Elsevier and CWTS published a report on data sharing with the title “73% of Academics Say Access to Research Data Helps Them in Their Work; 34% Do Not Publish Their Data”. The article is available here.
Prof. Dr. Michael Seadle led a panel discussion on “Examining Research Integrity” at the International Symposium of Information Science (ISI) 2017 in Berlin. Among the panelists were Dr. Thorsten Beck (HEADT Centre), Prof. Dr. Gerhard Dannemann (Humboldt-Universität zu Berlin), Prof. Dr. Wolfram Horstmann (Niedersächsische Staats- und Universitätsbibliothek Göttingen), and Prof. Dr. Debora Weber-Wulff (Hochschule für Technik und Wirtschaft Berlin).
Photo - From left to right: Prof. Dr. Michael Seadle, Dr. Thorsten Beck, Prof. Dr. Gerhard Dannemann
Watch our new infotorial on image manipulation and research integrity:
ImageJ is a public domain image-processing program developed under the auspices of the United States National Institutes of Health and especially popular among biologists. It was first published in 1997 and is designed to support a large range of image processing and analysis tasks and the detection of image manipulations in many different types of images, such as „three-dimensional live-cell imaging to radiological image processing, multiple imaging system data (and) comparisons to automated hematology systems.“ (Wikipedia: https://en.wikipedia.org/wiki/ImageJ)
One of its advantages is that ImageJ is able to read many different image file formats, such as TIFF, PNG, JPEG, BMP and many more. The program supports operations like edge detection, contrast manipulation, the measuring of distances and angles, sharpening, smoothing, rotation and many other (for more details see https://imagej.nih.gov/ij) and it allows the processing of stacks.
Downloading and installing the program and the associated InspectJ plugin is fairly simple. Just follow the instructions on the ImageJ website or watch one of the many tutorials on Youtube, for example a series provided by the Zentrum für Molekulare Biologie at Universität Heidelberg:
All you then have to do is to drag and drop an image for analysis onto the ImageJ tool bar in order to start the InspectJ plugin and follow the instructions of the program. ImageJ asks the user to decide whether an image has a dark or light background and then automatically produces an analysis image for each of the three RGB color schemes (green, blue, red). The user is free to decide which one of the layers should be analyzed – or whether all of the layers should be included in the analysis.
Fig. 1: Analysis images for each of the three RGB color schemes (red, blue, green) are generated. In this example most of the information appears to be found in the red and green schemes (the analysis of the blue scheme does not generate any useful results).
Once a layer for analysis is chosen, the program starts to process a set of predefined actions and a video sequence can be watched, which may reveal some of the otherwise hidden features of an image (for example edges).
Example: Gel Electrophoresis Analysis
For this evaluation, I chose an electrophoresis image that I retrieved from Wikimedia Commons (https://commons.wikimedia.org/wiki/File:Gel_electrophoresis_2.jpg).
This is the original (still unaltered) image:
Fig. 2: Gel electrophoresis: 6 “DNA-tracks”. In the first row (left), DNA with known fragment sizes was used as a reference. Different bands indicate different fragment sizes (the smaller, the faster it travels, the lower it is in the image); different intensities indicate different concentrations (the brighter, the more DNA). DNA was made visible using ethidium bromide and ultraviolet light. Author: Mnolf, Innsbruck, Austria, 2006
Creative Commons Attribution-ShareAlike 3.0 Unported https://creativecommons.org/licenses/by-sa/3.0/
For the purpose of testing I decided to copy and paste some areas of the bands and to erase some unwanted areas:
Fig. 3: Manipulated electrophoresis image.
Figure 4 specifies the details of the manipulations:
Fig. 4: Copied and pasted areas (rectangular shapes) and erased areas (oval shapes).
Here is a short video in which the subsequent steps of the analysis procedure are shown:
This is what the result of the analysis looks like once it is completed:
Fig. 5: Result of analysis (red color scheme): The ImageJ program with InspectJ plugin produces an analysis image with highlighted ROI’s (regions of interest), a log chart that presents the analysis results (in this case for example „identical areas“) and a ROI manager that allows to inspect each one of the results separately.
Discussion of results (Red Color Scheme):
The software specified regions of interest that are suspect of inappropriate image manipulation. The red color scheme analysis correctly reveals identical items at area 1 and 4 (with a 2-5 % tolerance setting) – which are actually those areas which were copied and moved, but it does not reveal the similarities between 1 and 3 or 3 and 4. Moreover, with a 10% tolerance setting it indicates a similarity between area 7 and 11, which was not part of the carried out manipulations.
Fig. 6: Result Analysis: Red Scheme
Fig. 7: Result Analysis: Green Scheme
Overview: Analysis results at different tolerance levels for the analysis of red and green color schemes:
Red Color Scheme Analysis:
Green Color Scheme Analysis:
Here is an example for how identical areas in an image are summarized in the log file:
Fig. 8: Log File for Red Scheme Analysis: Identical Items found at 1 & 4; 7 & 11.
This evaluation revealed how ImageJ helps with the identification of identical regions within an image. Still, some areas that have been highlighted in the process of analysis do not appear to be identical, but the program has found and indicated most of the copied regions – which might be due to inappropriate tolerance values.
It should be mentioned that the results of the green and red scheme analysis are not fully identical and that both of them have to be evaluated in order to derive a correct conclusion. Moreover, processing the JPEG image file with the InspectJ plugin indicated identical regions, but did not identify the erased areas (the oval forms, see Fig. 4). In one of the next blog posts, I am going to test the ImageJ solution for detecting erased areas.
More tools will be evaluated soon – please visit HEADT.EU for upcoming posts.
© HEADT CENTRE 2017
We set up a YouTube channel for the HEADT Centre where we make videos available that we produce to introduce our research as well as to share our research progress. We published our very first video yesterday: An Introduction to Research Integrity.
Our team working on Research Integrity at the HEADT Centre is currently producing short videos to introduce the core concepts that play a role in this field of research. Michael Seadle, Thorsten Beck, and I visited the recording studio at the Erwin-Schrödinger-Zentrum in Berlin-Adlershof, which belongs to Humboldt-Universität zu Berlin (HU). Both HU’s natural science library and the Computer and Media Service (CMS) are located there.
Michael Seadle (left) and Melanie Rügenhagen (right) at the recording studio of HU's Erwin-Schrödinger-Zentrum.
The CMS has professional staff dedicated to assist with the entire recording procedure that is done in addition to providing voice. They helped us a lot in getting the sound right, detecting reading mistakes and redo passages that sounded unclear. We spent about half a day reading the scripts we had written, edited and proofread in advance. The first results should be ready soon, and we will publish the videos on our website along with a note on this blog.
The US Office of Research Integrity (ORI) in the United States has been occupied with research integrity related issues for well over two decades. The office was established in 1992 by consolidating the „Office of Scientific Integrity“ and the „Office of Scientific Integrity Review.“ Today the Office of Research Integrity oversees many of the „Public Health Service (PHS) research integrity activities on behalf of the Secretary of Health and Human Services.“
For more information on the history and agenda of ORI, please visit: https://ori.hhs.gov/
Among its many aims are the development of “policies, procedures and regulations related to the detection, investigation, and prevention of research misconduct and the responsible conduct of research.” Over the years the office developed a wide spectrum of activities and services, like interactive videos, case studies, web modules, and many other forms of outreach and tools.
One of the resources the office provides for the wider academic audience are the so-called “Forensic Tools” for image analysis and manipulation detection, which provide customized standard operations and recordings of standard routines in Adobe Photoshop. (https://ori.hhs.gov/forensic-tools)
Forensic Droplets and Actions
ORI started with the development of such tools for Photoshop CS2-CS3, which leads to the fact that many of these routines can not be assessed with later versions of Photoshop. The office offers droplets and actions: droplets are „small desktop applications (…) that automatically process image files that are dragged onto their icon” whereas an action “is the sequence of steps that was pre-recorded in Photoshop” and is started from inside the program.
The “Forensic Tools” package comprises routines for the enhancement of certain image qualities and features, like droplets for the lighting up of areas or the adjustment of gradients. Other droplets allow for the comparing of two images, or provide overlays for images with dark or light backgrounds. Basically, the aim is to enhance features to support visibility, so that hidden details in an image (like obscured background information, artifacts or edges from copy-paste operations) may be revealed.
Other than the droplets the actions for Photoshop are claimed to be compatible with later versions of the program. However, the office provides many operations – too many to deal with all of them in detail in this blog post – therefore I am first going to first focus on the “Advanced Gradient Map” and thereby demonstrate some of the underlying logics of working with such actions.
For the purpose of testing I composed two different forms of manipulations of an image of a wallpaper to demonstrate the capacities or limitations of such standard operations.
Fig. 1: The original image: Wallpaper with decorative floral design. Photo was taken at Mirow Castle, Mecklenburg, Western Pomerania with a Sony Alpha 6000 ILS Camera.
Manipulation 1: Copy-Move Forgery
Fig. 2: Image detail of the original image without manipulation.
Fig. 3: Details obscured through background cloning.
Fig. 4: Copying and moving floral element with magic wand tool.
Fig. 5: Resulting manipulated image with copied floral elements (highlighted).
Manipulation 2: Cut & Paste Manipulation
The second manipulation is less subtle and easier to spot even with the naked eye.
Fig. 6: Copied and pasted area (highlighted).
Fig. 7: Detail of copied and pasted area. The cutting edge is clearly visible even without further enhancements.
Image Analysis with Forensic Photoshop Action: Advanced Gradient Map
Processing an image with the Advanced Gradient Map action simply requires to drag and drop the image onto the action item. This automatically initiates a number of steps: first the color image is reduced to grey scales, then the user is free to manually adjust curves and finally the action offers a set of color tables that may be helpful for further enhancing contrasts.
Analysis: Manipulation 1
Here are some screenshots I took while executing the action:
Fig. 8: Adjusting curves.
Fig. 9: Choosing a suitable color scheme for contrast enhancement
This is how the resulting image looks like:
Fig. 10: Result of Analysis
Result: The Advanced Gradient Map action allows for new perspectives when analyzing image content. Still, the action did not help to reveal the subtle changes that were carried out with the clone stamp and the magic wand – they simply remain invisible. Note: the result could have looked completely different with another color scheme. There is not just one option.
Fig. 11: Detail from resulting image: No clear evidence of a manipulation.
Analysis: Manipulation 2
Now let’s have a look at the second manipulated image in which the pasted region appears to be easier detectable. Again, the analysis is carried out applying the automatized steps of the action, but with different curve adjustments and color scheme. This is how the resulting image looks like:
Fig. 12: Advanced Gradient Map action: result with cut and paste manipulation
Clearly in this case the Advanced Gradient Map supports the identification of the manipulation. The edges around the pasted area are enhanced through the “light rainbow” effect and thus clearly visible.
Fig. 13: Detail from Advanced Gradient Map action
As has been shown in this quick evaluation the ORI action palette opens up a wide range of opportunities when it comes to image manipulation detection – respectively the visual examination of images or the visualization of hidden features. However, it should be mentioned that the ORI customized the action palette for analyzing specific images, especially for Western blots and other images from the field of biomedicine. Nevertheless, the manipulations carried out on the wallpaper images above are in some way comparable to manipulations of blot images. Duplicating image areas or obscuring certain aspects are a serious concern in many biomedicine research integrity cases.
Other than the “Forensically” tools that we have discussed earlier on this blog Photoshop actions support the enhancement of certain features within an image, but can not be understood as a way of automatically detecting image manipulations. On the contrary, applying the actions requires a “trial and error”-attitude and the user has to invest time and effort to produce satisfying results. As it seems there are manipulations that cannot easily be traced with the Advanced Gradient Map action tool, but which require other tools – tools which will be discussed on this blog soon.
The HEADT Centre 2017