Transcript Adaptation of Applets and Live videos to Mobile devices

ADAPTATION OF APPLETS AND LIVE
VIDEOS TO MOBILE DEVICES
J Manoj Kumar (M.Tech IIT Bombay)
Guided by: Prof. Sridhar Iyer
Problem Definition
Adapt CDEEP Live Videos to Mobile Devices.
1.
 Adapt to low bandwidth networks.
 Problems in video streaming to mobile devices
 Limited Memory in Mobile devices.
 Low Bandwidth of Mobile Network like GPRS.
Adapt Java Applets to Mobile Devices
2.

Java Applets are built with J2SE technology that do not work on
mobile devices.

Limited Memory in Mobile devices.
Mobile devices work on J2ME technology.

Adaptation Of CDEEP Live videos to Mobile Devices
Existing System - Study Element Based System
 Works only for “stored CDEEP videos”
Study Element Based System
 Three types of Study Elements are categorized
i.
Presentation Element -- portion of video that shows one
slide of presentation.
ii.
White paper Element – portion of video that shows white
paper.
iii.
Instructor Element – portion of video that shows
instructor.
Classifying Images
 Classification -- Feature based Tagging
 Feature -- a unique property
Type of Element
Feature Used
Instructor Element
Face Recognition
White paper Element
CDEEP Logo Image
Presentation Element
Heading Color
Proposed System
Live Video Adaptation Module
 Capture sets of 300 images from Live stream.
 Classify each set of images study elements.
 Copy images and audio to server.
Image Extraction Methodology
 Approach 1:
 Using Video LAN(VLC)
 Save or Record for five minutes.
 Extract Images from saved video.
 Disadvantages
 High CPU utilization because of Video transcoding.
 Delay between actual video and seeing video increases for each
iteration.
 Convert to MJPEG Stream
 MJPEG - Motion JPEG ( Images are shown one after the
other). Ex: surveillance camera.
 Images are separated by delimiter.
 Disadvantages
 High CPU Utilization because of Video Transcoding.
 Unable to display images.
 Approach 2:
 Using FFmpeg
 Images extracted directly from live stream.
 ffmpeg -i url -an -r 1 -f image -s 320x240 video%d.jpg
 Group images to sets of 300.
 Using Mplayer:
 Images extracted and grouped directly from live stream.
mplayer -nosound -vf framestep=25 -vo jpeg:subdirs=img:maxfiles=300 url
 Audio Extraction
 Using FFmpeg
 Audio is saved in the form of mp3 chunks.
 ffmpeg -i url -t 00:00:20 -acodec libmp3lame -ab 24 -ar 8000 audio.mp3
 Generate XML
System Implementation
Sequence Diagram
Design Considerations
 Port blocking in GSM networks
 Direct TCP connections not possible due to port blocking.
 Allows only HTTP connection to ports 80 and 8080.
 Audio Streaming
 Audio streaming to mobile device is not supported.
 Hence, need to do progressive download.
Usage
1.
Enter URL
2. select video
3. slide show
Comparison to Existing System
 Supports automatic creation of offline copy of video.
 Live video can be viewed as soon as it starts.
 Misclassifications of images cannot be corrected.
Adaptation of Applets to Mobile Devices
Project OSCAR
 Project OSCAR – Open Source Courseware Animations
Repository.
 Repository for web-based, interactive animations for various
teaching concepts.
 Animation is typically an Java Applet with brief description of
concept.
Overview of J2ME
 Java Micro Edition J2ME – small devices like mobiles, PDA’s
with low RAM and speed.
 J2ME has 3 principles
 Configuration – JVM + set of API’s
 Profile – set of API’s designed for specific configuration
 Optional package –technology specific API’s to extend the
capabilities of JVM
Architecture of J2ME
J2ME
 Configuration: specifies JVM and core set of API’s for a specific
family of devices.
 Two types of configuration
 Connected Device Configuration CDC – minimum of 512KB ROM,
256KB of RAM, full JVM must be supported.
 High end PDA’s, TV setup boxes, car navigation systems.
 Connected Limited Device Configuration CLDC – minimum of 160KB
of ROM, 32KB RAM, KVM
 Mobiles, pagers, PDA’s.
 Cannot add native methods.
J2SE Applets to J2ME Conversion Difficulties
1.
Incompatibility of Architectures of J2SE and J2ME


MIDlet architecture
Applet Architecture
Only one form per screen in MIDlet, Multiple panels in applet.
Difficult to convert from J2ME architecture to J2SE
architecture.
1.
.
2.
Canvas size and Co-ordinates of Image Object


Designed for Desktop systems
Co-ordinates have to be set manually – cannot be automated
Sample OSCAR paint code
public void paint(Graphics G){
g.drawImage(Image img1, 480, 600,this);
g.drawImage(Image img2, 530, 400,this);
g,drawImage(Image img3, 60, 370, this);
}
Amount of Text on Screen
1.
Designed for Desktop Resolutions – cannot be fit mobile
screen, preplanning cannot be automated.
Sample OSCAR animation with text field.

public void init(){
ta = new TextArea("", 31, 47);
ta.setText("\n\t Energy Transaction in Biological Systems\n"+
"\n In this animation you will learn the concepts of ATP
Metabolism……….
……………… 10 lines };
Other General Differences
 No floating point support in J2ME.
 No Java Native Interface (JNI) in J2ME.
 No finalize method in J2ME.
 No Thread Groups.
J2SE to J2ME – Conversion Possibilities
1.
Copy of Paint Method
J2ME uses J2SE’s Graphics class with minor changes -- enables
copy paint method as it is during conversion
Sample applet with paint method
public void paint( Graphics g){
g.drawString("Sending",150,75);
g.drawRect("40,50,10,20);
g.drawImg(Image img,20,30,center);
g.drawArc(30,40,10,30,60,120);}
g.drawString, g.drawRect, g.drawImg, g.drawArc are common in both j2se and
j2me
 Buttons to Key mapping
 J2ME doesn’t display buttons, it uses mobile device keys take
input
 OSCAR animations -- buttons like start, stop.
 Buttons to key mapping is possible.
 LWUIT
 Light Weight User Interface Toolkit is a UI library developed by
Sun.
 Offers advanced UI capabilities and API inspired by Swing.
 LWUIT allows multiple panels on screen.
Overview of Flash and Action Script
 Flash – multimedia platform language for creating animations
 Action Script can be embedded. – ECMA script.
 AS is Object Oriented similar to Java Script.
 AS + Flash – more interactive animations.
 Used to build Rich Internet Applications.
 Flash Lite – highly optimized flash runtime for mobiles.
 Provides same functionality similar to desktop.
Applet to Flash – Conversion Difficulties
 No Multi-Threading in Flash
 Multi-Threading – two or more parts in a program to run
concurrently ( Multitasking ).
 Essential ability to have code do something while UI is doing
something else.
 Single Thread – Having heavy computation, cannot update UI.
 Program becomes un-responsive and crashes after 60 seconds.
Sample OSCAR program with thread
class Timer extends Thread {
//code//
void Settime();
void sleep(); }
Conversion difficulties
 No method(function) overloading
 Difficult for multiple methods – if constructor overloading
 Sample java program
public void defaultSetter(int iValue) {
setter(iValue, false); }
public void setter(int iValue, boolean bForce) {
// whatever }
In action script
public function setter(iValue:int, bForce:Boolean = false):void {
// whatever }
Other General Difficulties and Possibilities
1. Limited Data Structures
 Action script supports only Arrays and Dictionary.
 No queues, no linked List, no stack compared to Java.
 Need to re-implement data structures which are wrapped.
No Blocking
2.

Next code of line is supposed to run, user cannot prevent it
from run – Alert.show(); does not stop exexctuing next line.
 Possibilities
•
•
Java and Action Script have same syntax with a little change.
Syntax change automated by Java to Action script converter
(J2AS3)
Conclusion
 Successfully integrated to existing Study element based
adaptation.
 Live video transmission of CDEEP videos can be adapted
to networks with low bandwidth.
 Automated porting Java applets to mobile devices as
J2ME MIDlets and flash files is difficult and infeasible.
Future Work
1.
Decrease Misclassifications of images


2.
Synchronization between audio and images

3.
Study element based system is prone to errors.
Better feature based classification.
Synchronizing audio with the image slideshow.
Text Improvement

Identify text in study elements and improve text quality.
References
1.
Fabrice Bellard. Ffmpeg codec manual. http://www.ffmpeg.org/ffmpegdoc.html .
2.
Free Software Foundation. Mplayer manual.
http://www.mplayerhq.hu/DOCS/HTML/en/index.html.
3.
G.N. Murthy and S. Iyer. Study element based adaptation of lecture videos to
mobile devices. In Communications (NCC), 2010 National Conférence on,
pages 1 - 5, 29-31 2010.
4.
Herbert Schildt . Java 2ME the complete reference. Osborne/McGraw-Hill,
fourth edition, 2001.
Questions?