Transcript A Grid Infrastructure for Scientific Film Studies

A Grid Infrastructure for
Scientific Film Studies
Dominik Seiler and Thilo Stadelmann
DFG Research Project
„Methods and Tools for Computer-Assisted Media Analysis “
Research Center „Media Upheavals“ (SFB/FK615)
University of Siegen & University of Marburg
Content
• MT Project
• Scientific Film Studies
• Videana
– Use Case Examples
• Videana Grid
– Architecture
– Usage Scenario
– Web and Grid Service Browser
MT Project
• Project MT:
– „ Methods and Tools for Computer-Assisted Media
Analysis“
• Research Center „Media Upheavals“ (SFB/FK615)
• Goals:
– Computer-based methods for scientific analysis of
images and videos
– Development of a Grid-based infrastructure for
• easy data access
• parallel execution of compute-intensive algorithms
Scientific Film Studies
• Gaining insights and knowledge about production,
perception and aesthetics of movies and TV casts
• Relevant aspects of film:
– Shot composition (cut, gradual transition type: dissolve etc.)
– Camera motion, camera distance (close-up, medium shot,
panorama)
– Presence of actors and people
– Superimposed text
– Audio information (speech, music, background noise)
– Motion, tempo
– ...
Videana
• Software toolkit for scientific film studies
• Integrated algorithms for:
–
–
–
–
–
–
–
Shot boundary detection
Camera motion estimation
Face detection and clustering/recognition
Speaker detection and clustering/recognition
Text detection und segmentation (for OCR)
Semantic Analysis (concepts, „high-level features“)
Video shot retrieval
• New analysis algorithms can be easily added as plug-in
• Metadata are saved and handled according to MPEG-7 standard
Videana
• Graphical User Interface:
– Video player, access particular frames
– analyze videos automatically
– visualizations of cut frequency, luminance,
luminance variance
– insert annotations
– correct detection results
Videana Use Case Example 1
• Semantical analysis of computer games
– Automatic detection of following categories in egoshooter:
Inactive
Preparation
Search
Violence
Experimental Results
[%]
Recall
Precision
F1-measure
Preparation
92.2
96.0
94.1
Search
94.9
90.0
92.4
Violence
55.3
66.0
60.2
Inactive
92.0
97.6
94.8
Videana Use Case Example 2
• Structural analysis of early films
• Possible Question:
1. Difference in cut frequency related to producing
country?
2. Difference in structural film montage patterns?
Results
1. Analysis of cut frequency related to
producing country and period
origin
average #cut/minute
USA
5.6
France
3.4
origin
average #cut/minute
1907-1909
2.6
1911-1913
5.3
Results
2. Quick detection of montage patterns
– e.g. usage of text
Text
Text
Text
Text
Text
Motivation for Grid
• Analysis tasks are computationally intensive
– Long running tasks
– Batch jobs can keep a system buys for days
• Resources of desktop computer / laptop are often low
– Conflict with other activities on the system
• Solution:
– outsourcing of computational tasks
• Grids provide access to needed computational resources
– Cluster
– Desktop Pool
Videana Grid
• Grid Computing extension of Videana
• Based on Globus Toolkit 4 (GT4)
– open source toolkit for building grids
• Access external data sources
• Support large binary data transport
– GridFTP
– Flex-SwA
Videana Point of View
Videana Point of View
Videana Point of View
Videana Point of View
Videana Grid Architecture
Videana Grid Usage Scenario
Videana Grid Usage Scenario
Videana Grid Usage Scenario
Videana Grid Usage Scenario
Videana Grid Usage Scenario
Videana Grid Usage Scenario
Web and Grid Service Browser
• Exposition of detection algorithms and Videana core
tasks as Grid Service
– MPEG decoding, Face Detection, Text Localization, OCR, …
• Ease usages and testing of Web and Grid Services
• Firefox-based Plug-In
– browse service description files in analogy to html
• Automatic client generation
• Direct Flex-SwA support for transport of large binary
Data
Service Description
• Web Service
Description
Langue (WSDL)
– XML-based
language
– Technical Service
description
Web and Grid Service Browser
Result Presentation
Thank You!
Questions?
Outlook (Future Work)
• Mash-up engine and editor
– Ease access to Web (2.0) content
– Ease process of building content analysis
workflows
• Utilization of virtualization
Mediana
• Frontend integrating heterogeneous databases
• Early version exists, including
– 1000+ mediografies, linked films and related data (A4)
– 1000+ literature files and related data (B6)
– 10000-50000 broadcast related data sets (SFB 240)
• Currently not publicly published due to open questions
– Who are potential users
– Problems with permissions and copyright issues
• Plans to revise within archiving activities in next phase