Transcript JMF-CTS-2006 - Community Grids Lab

Building Scalable and High Efficient
Java Multimedia Collaboration
Wenjun Wu, Tao Huang, Geoffrey Fox
Community Grids Computing Laboratory, Indiana
University, USA
Global-MMCS & XGSP

XGSP
• A common AV signaling protocol
• A core conference control mechanism
• Collaboration Web-Services

Global Multimedia Collaboration System (GlobalMMCS)
• to support scalable web-service based interoperable
collaborations
• integrates various services including videoconferencing, instant
messaging and streaming, and supports multiple
videoconferencing technologies such as H.323, SIP and Access
Grid clients
Global-MMCS & XGSP
Use Multiple Media servers to scale to many codecs and many
versions of audio/video mixing
Session Server
XGSP-based Control
NaradaBrokering
All Messaging
NB Scales as
distributed
Admire
Web
Services
SIP
H323
Media Servers
Filters
High Performance (RTP)
and XML/SOAP and ..
Access Grid
Gateways convert to uniform XGSP Messaging
NaradaBrokering
Native XGSP
Java Media Framework

Java Media Framework (JMF)
• A unified framework for managing the acquisition, processing,
and delivery of time-based media data
• A common cross-platform Java API for accessing underlying
media frameworks.
• “Write Once, Run Anywhere” to multimedia developers
JMF for multimedia collaboration

JMF is the key building block for the implementation
of Global-MMCS
For both client-side and server-side
Building multimedia services such as video, audio mixing,
snapshot generation
Implementing Global-MMCS AVportlet, a native XGSP
audiovisual client

Performance Issues in JMF
the performance optimization of Java Media Framework has to be
made especially for the conferencing application
Multimedia Programming Frameworks
Video Capture
Windows
VFW or WDM
Video
Rendering
Direct Draw
Linux
Video-for-Linux
XLib
Mac OS
Sequence Grabber Quick Draw
Audio
ALSA
Direct Sound
Core Audio
Portability vs Performance :
Platform-dependent multimedia devices and codec management
a portable multimedia framework must define a good platform-independent
part to cover the heterogeneous native multimedia platforms.
Multimedia Programming Frameworks



the similar multimedia filter pipeline architecture
Filter & Pipe design pattern
cascaded filter chain
Support high efficient media processing
• High-volume multimedia data is typically generated by
a source filter objects and piped through one or more
filter objects. Eventually, the media reaches a sink filter
object and is consumed.
Multimedia Programming Frameworks



Tcl/Tk based
CMT [7], VIC [8], VuSystem [9]
low-overhead control functionality implemented in a
scripting language like Tcl
performance-critical data handling implemented in a
compiled language like C/C++
Windows DirectShow
C++ COM API
only enables Windows applications for multimedia
programming
JMF
Network
RTP Video RX
Session
Manager
RTP
DataSource
RTP Audio RX
Session
Manager
RTP
DataSource
RTP Video TX
Session
Manager
Video
Encoded
DataSource
Video
Encoder
Raw Video
DataSource
RTP Audio RX
Session
Manager
Audio
Encoded
DataSource
Audio
Encoder
Raw Audio
DataSource
Video
Decoder
Video
Renderer
Monitor
Audio
Decoder
Audio
Renderer
Network
JMF Filter Architecture
Access Grid
Access Grid : a large scale
audio/videoconference based
on a multicast network
 provides the group-to-group
collaborations among 150
nodes connected to Internet 2
world wide.
 Use improved MBONE
audiovisual tools VIC and
RAT
 Depends upon high-speed
network ( each node needs
20Mbps )
Issues for JMF in collaboration



Very good cross-platform framework
No optimization for conferencing application
Compared to fully performance tuned Tcl/Tk package
( Access Grid )
No implementation for Mac OS
Enhanced Java Media Framework



Improving JMF performance by exposing the
interfaces of the JMF filters and rewriting a faster one
Adding the state-of-art codecs in JMF and screen
capture
Extending the JMF to the Mac-OS platform to extend
its portability
Video Rendering



JMF filter pipeline is composed of filter Modules and
linking Connectors
streaming protocols between a OutputConnector and
InputConnector:
Safe protocol
Push protocol
JMF render implementation:
safe protocol , blit the whole video frame

Our optimization approach:
“Direct Write Through” approach
Direct-Write-Through optimization in JMF rendering
Output
Connector
Circular
Buffer
Input
Connector
Direct
Draw
Renderer
Video
Decoder
Mark
Data
OffScreen
YUV Surface
BLT
Primary
YUV Surface
Video Capturing



a video data source to abstract the real capture
services which usually have different APIs in modern
operating systems.
added Mac video capturing into JMF package
Added the desktop capture as a video data source
to support remote desktop sharing
Add New Codec

JMF 2.0 package only supports H.261 decoder, H263
and JPEG. We added H.261 encoder and MPEG-4
video codec (DivX) based on JMF codec interface. In
addition, new RTP format and payload for MPEG-4
video are also added through JMF RTPManager.
Global-MMCS AVPortlet
Video / Audio
Video
/ Audio
Processor
Processor
Video / Audio
Video Player
/ Audio
RTP/RTCP
RTP/RTCP
Session
Session
Narada
Connector
Narada
Connector
RTPEvent
Transport
RTPEvent
Transport
Player
Narada
Bridge
XGSP
AV Session
Controller
XGSP
AV Session
Protocol Stack
NaradaBroker
JMS
Transport
Global-MMCS AVPortlet



GlobalMMCS AVPortlet, to fully make use of the
services provided by Global-MMCS. The tool which
can run on multiple desktop platforms such as windows,
Linux and Mac OSX, integrates audio and video
collaboration together
Extending JMF RTP Transport over Publish/Subscribe
Overlay
XGSP Audiovisual Session Management
Performance Evaluation

Compare the video rendering performance between JMF and
VIC
Testing video streams are captured from the same desktop of the
sending machine, and multicasted to the receiver machine
which runs Access Grid VIC, two JMF clients based on Sun’s
JMF implementation and our improved version.

two test scenarios with different source streams
•
the CIF-size still image of the desktop area is sent to the clients. Each
stream is encoded in H.261, and has average bandwidth 20kbps.
• the CIF-size video sequence from a 30-second movie with a lot of motions
is streamed to the clients. Each stream is encoded in H.261, and has
average bandwidth 500kbps.
Performance Evaluation
60
70
50
60
40
AG VIC
30
SunJMF
20
FastJMF
50
40
SunJMF
30
FastJMF
20
10
10
0
0
1 2 3 4 5 6 7 8
AG VIC
1 2 3 4 5 6 7 8
Video Rendering performance (left: still desktop, right: movie sequence)
Q&A