Transcript VIDOS - TERENA Networking Conference 2002

TERENA
Networking Conference
2000
Limerick, June 3-6, 2000
The development of VIDOS:
a Web-based video editing,
customization and repurposing service
David Shotton, Thomas Boudier, John Pybus, Danny Torbica
and Jamie Shotton
Image Bioinformatics Laboratory
Department of Zoology, University of Oxford
E-mail: [email protected]
Personal computing at present







Desktop-centred
Requires multiple individual copies of all applications, and
multiple backup facilities
Hardware and software become rapidly outdated and are
expensive to replace
Problems with application and version incompatibility
between colleagues
Limited use of the network for co-operative
computing over long distances
Network bandwidth inadequate for real-time viewing of
videos or high quality video conferencing
Use of mass-produced standardized media entities
The new paradigm for personal computing
CENTRAL
NETWORKED
SERVERS




Network-centred
Single copy of most recent
version of application on
powerful central server
Simple and cost-effective
Enhances inter-personal
co-operation




Requires advanced infrastructure
with high bandwidth network
Permits effective use of central
databases and archives
Real-time video viewing possible
Permits customization of personal
versions of media entities
Two examples of network-centred computing


The BioImage Database, an
on-line database for
multi-dimensional images
and videos of biological
specimens
www.bioimage.org
VIDOS, a Web-based video
editing, customization and
repurposing service
www.vidos.ac.uk
Outline of my presentation

The VIDOS prototype

Current VIDOS developments

A VIDOS demonstration

The future of VIDOS:
 Improving the user interface and spatial editing – the J-Video player
 Integration with IVANA for semantic content analysis
 Continuing the VIDOS service as part of VideoWorks for the Grid
 Integration with video databases
 The SONGBIRD project for semantics and ontology generation
The VIDOS prototype
VIDOS SERVER
VIDOS CLIENT
VIDOS video
retrieval
program
VIDOS video
conversion
program
DISTANT VIDEO
DATABASE
Original
videos
KEY
Software
libaries
Java
program
Internal
bus
Software
libraries
Internet
Video
filestore
Video filestore
Schematic
diagram of the
prototype VIDOS
system
VIDOS
a Web-based video customization system

Edit a digital video spatially and temporally over the Web

Adapts the video content, size, format and compression quality to
its intended purpose, or to match the capabilities of one’s PC

The video may be in a VIDOS-enabled database or anywhere
else on the Web, including the user’s own Web server

VIDOS gives the user basic video editing capabilities without the
need to purchase expensive video editing software

By reducing the size of digital video files, VIDOS can improve
efficiency by speeding network transfers, reducing
bandwidth bottlenecks, and minimizing disc storage
Conference presentation
Terena Networking
Conference
Lisbon, 22-25 May 2000
Session 6A, ‘Coming soon to a monitor near you’
Chair: Egon Verharen
VIDOS - a system for editing and customizing
videos over the Web
David M. Shotton and Thomas Boudier
Department of Zoology
University of Oxford, UK
Journal publication
Boudier and
Shotton (2000)
Computer
Networks
34: 931-944
The current development of VIDOS
The limitations of the VIDOS prototype

Underlying lack of flexibility in the software implementation

Limited video format and codec choice, determined by the
outdated underlying Silicon Graphics digital media library
employed for video format trans-coding

User interface showed only first and last frames, making
temporal editing difficult on videos with scene changes

Inability to splice together different video clips
The VIDOS second phase - objectives

To rewrite VIDOS from scratch, to provide a more efficient and
extendable software implementation and overcome the
limitations of the prototype
 To enhance VIDOS functionality by expanding the choice of video output
formats, and by updating the choice of codecs to include those using the
most modern compression algorithms
 To enhance the user interface for temporal editing
 To permit the splicing together of separate video clips to create a single new
output video file

To launch a VIDOS service for academia via the VIDOS web site
The VIDOS Home Page
The VIDOS second phase –architecture
The VIDOS hardware configuration:

A VIDOS Web server – VIDOS slave processor farm topology
The VIDOS Web server:

To undertake user authentication

To serve Web pages and Java applets, and receive parameters

To commission jobs for the slave processors to implement

To control the video filestore and send customized videos to users
An expandable farm of slave processors:

To undertake individual video conversion tasks
Current VIDOS architecture
VIDOS WEB SERVER
VIDOS CLIENTS
VIDOS client
interface
VIDOS SLAVE
PROCESSORS
Client thread
VIDOS server
VIDOS client
interface
Client thread
Conversion thread
Conversion thread
DISTANT VIDEO
DATABASES
Video files
VIDOS slave
conversion
processor
VIDOS slave
conversion
processor
KEY
Java
program
Retrieval thread
PostgreSQL
database
PostgreSQL
database
Video
filestore
Internal
threads
Video files
Retrieval thread
Local video
filestore
Internet
VIDOS functionality
The GALLOP test video
Original video details:
QuickTime Format
M-JPEG compressed
Compression quality 0.6
One stride of 15 frames
repeated six times.giving
90 frames at 30 fps
File size: 3,731 Kb
Preview video details:
Highly compressed AVI preview
Zoom factor 50%
Compressed file size: 420 Kb
Credits
Photographer:
Racehorse:
Jockey:
Eaduard Muybridge, 1878
Annie, a Californian thoroughbred
unknown
The VIDOS input video selection interface
The VIDOS output format selection interface
The VIDOS spatio-temporal editing interface
The VIDOS spatio-temporal editing interface
The VIDOS conversion request interface
The VIDOS conversion request interface
The VIDOS conversion request interface
The final VIDOS- customized video
Original video:
QuickTime format, M-JPEG
compressed at quality 0.6, 90
frames (6 strides) at 30 fps
File size 3,731 Kbytes
Customized video:
Selected area: Jockey and horse’s head and shoulders only, 50% zoom
Selected time: first 45 frames only (three strides) at 10 fps
Selected output format: AVI format, Indeo compressed at quality 0.3
File size 91 Kbytes
Compressed file approximately 41 times smaller than original
Let’s try a demo !
Start VIDOS
The benefits of VIDOS

For individuals and academic institutions, VIDOS provides:
 enhanced download times and reduced video storage requirements,
 free video editing, customization and format conversion facilities,
 reduced hardware and software costs, and
 enhanced personal and corporate efficiency

For distributed organizations (e.g. a university with regional campuses):
 a system for distributed client-based video editing from a centralized server

In education and training:
 the ability to select and repurpose video clips from a central resource for inclusion in
live lectures or computer-based training programs

For academic video databases and video content providers:
 added value by enabling users to edit and customize selected videos
 particularly powerful if coupled with a Query-by-Content system
Registration of VIDOS Users

We are now seeking VIDOS Users who, as well as using VIDOS for their
own productive work, can evaluate VIDOS and advise us on functionality
enhancements

If you would like to register as a VIDOS User, please log on to
www.vidos.ac.uk, and complete the Registration Form that you will find
under the Registration button.

You will then be sent a password that will give you unrestricted access to
VIDOS from any computer
Functional developments of VIDOS
Further developments of VIDOS functionality will require a break from our
previous philosophy of enabling VIDOS to run on any modern browser
platform, and will instead require that users first install from the Sun Web site
the latest versions of Java and Java Media Framework.
This will enable us:

To further enhance the functionality of VIDOS for temporal editing of videos,
by incorporating
 J-Video, our new Java video player that uses Java Media Framework and
permits the splicing together of separate video clips
 IVANA (Interactive Video ANalysis Application) for semantic content analysis

To integrate VIDOS with our other video tools
CTL-mediated target cell death
The J-Video java video player
The J-Video java video player
The J-Video java video player
IVANA for semantic content analysis
Semantic content analysis classes
Video
Segment
Frame
A temporal sequence of scenes
A distinct sequence of video frames
A single video frame
Character
Object
Background
Animate characters (e.g. cells, racing drivers)
Inanimate objects (e.g. pipettes, racing cars)
Region of image not occupied by objects, etc.
Event
Sound
An instantaneous or extended happening
involving one or more characters or objects
An item on the video soundtrack
Area of interest
Annotation
Insert
A geometric area enclosing items of interest
A textual label relating to a character or object
A separate image, animation or movie insert
Query by Content using VANQUIS

We now plan to combine
 J-Video for client video manipulation,
 IVANA (Interactive Video ANalysis Application) for interactive video
content analysis, and our alternative
 VAMPORT (Video Analysis and Metadata Production by Object
Recognition and Tracking) system for automated analysis of simple
videos, with
 VideoStore, our semantic content metadata database, and a
 Query by Content interface, to create
 VANQUIS (Video Analysis and QUery Interface System)

This will enables users to make such queries as:
 “Find me a sequence showing Lord Jenkins”
 “Find me the times of all target cell deaths that occur in the
upper left corner of the screen”
 “Which CTL are serial killers? Show me them at work!”
The future of VIDOS as part of
VideoWorks for the Grid,
an Oxord University e-Science
Centre testbed project
The VIDEOWORKS partnership

VideoWorks for the Grid is a new umbrella project for
these video e-services, launched in the context of the
current UK Grid initiatives

It is the first testbed project of the Oxford e-Science Centre

Our VideoWorks industrial partners are
 IBM
 Virage
 Telestream
 Square Box Systems, and
 The International DOI Foundation
Functionality of VideoWorks

Central to the video e-services within VideoWorks will be the on-line video
editing and customization services of VIDOS

The same Java software structure and server-slave protocols as presently
developed for VIDOS will be expanded for VideoWorks

We will use VANQUIS for interactive video semantic content analysis

VANQUIS semantic metadata will be stored a VideoStore database modelled
upon the BioImage Database

We will use
 Telestream’s FlipFactory for efficient video transcoding
 Virage’s VideoLogger for automated structural video analysis
 IBM’s Video Charger for streaming preview videos for editing and analysis
VideoWorks server
Multiple concurrent users
Video logging workstation
Web Web
server
VideoWorks
server
Browser interfaces
XML and X-Query
Java applets
VideoWorks middleware
Browser interfaces
User authentication
User authentication
LDAP server
Transaction
Transaction
logging logging
Contentand
Manager
Job distribution
load balancing
DV video
Camera
DV video
tape deck
VideoWorks
VideoWorks applications layer
applications layer
Academic video databases
Telestream
ClipExpress
BUFVC database
Video files
and metadata
Local video files
Analogue
VCR
BioImage database
Image files
and metadata
Slave processor farm
Key
HTTP and Grid protocols,
video file transfers, and
compressed broadcast quality
(2-6 mbps) video streams
over high band width networks
(Gigabit ethernet and SuperJANET4)
Slave processor 1
Slave
processor
IEEE 1394
Analogue video
V W
Slave processor 3
Slave
processor
VIDOS editing
Slave processor 4
Slave
processor
VIDOS editing
The
VideoWorks
for
the
Grid
system
Links from VideoWorks to video databases

Links will be made to major academic video databases such as
BioImage and the British Film and Video Council’s MAAS Media Online

User of these databases can upload videos of interest to VideoWorks
either for semantic content analysis using VANQUIS
or for editing and customization using VIDOS, before downloading

After semantic content analysis, the semantic content metadata stored
within VideoStore will be available for subsequent query by content
using the VANQUIS query interface. Videos located by QbC may
also be customized using VIDOS before downloading
From image to knowledge:
the state of the art of image bioinformatics
Shotton, D. M. (Nov 2000) Microscopy and Analysis 80: 23-25.
In that paper I said that harvesting knowledge from digital videos
would require five components:





a searchable image database - The BioImage Database
an automatic video content analysis system for simple videos - VAMPORT
an interactive video analysis system - IVANA
a query by content system – VANQUIS
a web-based video customization system - VIDOS
Combined together, these form a unique and powerful combination to
facilitate the conversion of raw video data into knowledge.
VideoWorks is the realization of that vision.
Thomas Boudier
John Pybus
Jamie Shotton
Danny Torbica
My research
assistant
1997-1999
VIDOS Research
and Development
Programmer
Computer Science
Student and VIDOS
Java Programmer
VIDOS Project
Development
Manager
Author of the
VIDOS prototype
Has written
VIDOS 3.0
Has written our Java
video player and
semantic content
analysis module
Mac expert and
Webmaster