Transcript Detecting False Captioning Using Common

Summary
Marie Yarbrough
 Introduction
 History of Image Forgery
 Method
 Segmentation
 Classification
 Common-Sense Reasoning
 Conclusion
 Images have been a powerful media of delivering and
communicating information ever since their inception.
 The act of distorting and changing images has been
around for the same amount of time.
 Detecting these image manipulations images has
become and important problem.
 Even more so now that we have entered the digital age.
 This paper produces a way of detecting these falsified
images.
 People have been doing image manipulation since the
beginning and these forgeries have been put to many
uses. Such as
 Journalists who want to make up their own stories
 Photojournalists who want dramatic scenes
 Scientists who forge or repeat images in academic
papers
 Politian's who try to direct public opinion by
exaggerating or falsifying political events
 There are 4 manipulation techniques that are used on
images.
 Deletion of details: removing scene elements
 Insertion of detail: adding scene elements
 Photomontage: combining multiple images
 False Captioning: misrepresenting image content.
History of Image
Forgeries
A series of photos of
the “Devil’s Den”
sniper
photographed after
the battle of
Gettysburg. The first
three photos show
the soldier where he
fell in battle. The
fourth shows him
“Posed” for dramatic
effect.
 The author proposes using Artificial Intelligence (AI)
techniques of common-sense reasoning to detect
duplicated and anomalous elements in a set of images.
 The basic premise: If they can detect a set of key
elements in an image then they can detect if they have
been moved, added, or deleted from a scene prior to
image created.
 The most important step of this process it to split the
images into Regions of Importance (ROI).
 They use mean-shift image segmentation to
decompose an image into homogeneous regions.
 Routine has inputs:
 Spatial radius: hs
 Color radius: hr
 Minimum number of pixels: M
 They then over segment the image using low values of
hr and M and merge adjacent regions.
Segmentation
(Top)Results of mean-shift
segmentation with
parameters hs = 7, hr = 6,
and M=50.
(Bottom) Results of region
merging.
 After they segment the image the next step is to
classify the ROI.
 They propose a segment based classification scheme.
 Brute force pixel comparisons take too long to do.
Segment-wise classification reduces the size of the
problem space significantly.
 Comparing the relationship of ROI across a corpus of
images gives the ability to determine if a scene has been
manipulated during the photo recording process.
 They set about this classification by computing an
importance map that assigns a scalar value to each
pixel estimating the importance of that image location
based on an attention model.
 They use measures of visual salience, image regions
likely to be interesting to the low-level vision system,
and high-level detectors for specific objects that are
likely to be important.
(Top) Importance
map. Saliency
regions are outlined
in magenta, face
regions in cyan.
(Bottom) Regions of
importance.
 They use two different approaches to assist digital
forensics.
 To resolve local classification ambiguities within images,
they query a knowledge base to resolve the proper
relation
 A common-sense knowledge base such as Cyc and
OpenMind is well suited for this task.
 For example, given two large horizontal blue regions many
classifiers cannot distinguish which is ‘sky’ and which is
‘water’. A common-sense knowledge base can be queried to
find the answer.
 Then, they reason across a larger corpus of images to
find unique or missing elements during an
investigation.
 In many cases, the single image might not tell the complete
story.
 A collection of photos, however, does show a narrative of a
larger story.
 For example, a man-made object such as a plane in a field of
grass should raise suspicion. Unless, all the photos in a
corpus have a similar qualitative structure.
Common-Sense
Reasoning
They suggest a
software system
based on a
combination of
existing tools to
identify common
objects across a
corpus of images. By
visualizing such
objects, as in this
figure, even a
layperson can quickly
determine whether a
given image isfalsely
captioned.
 Through a series of segmentation, classification, and
common sense reasoning they can find parts of images
that might have been manipulated.
 These methods are limited by the performance of the
components they use though.