Transcript Internet and Multimedia - Florida Atlantic University

Video and Image Internet
Broadcasting Systems
Research Projects
in Multimedia Laboratory at
Florida Atlantic University
Projects and Their Applications
IP Simulcast - An Innovative
Broadcasting Technique Over the
Internet
XYZ Video Encoding - A New Video
Compression Technique
A Fast Content-Based Multimedia
Retrieval Technique
Interactive Progressive Encoding
System
Internet Broadcasting or
Webcasting
Broadcasting multimedia data
(audio and video) over the Internet from a server (sender) to a large
number of clients (receivers)
Applications include:
radio and television broadcasting
real-time broadcasting of critical data
distance learning
videoconferencing
database replication
electronic software distribution
Webcast Audio Examples
 AudioNet - has mainstream content from
Major League Baseball to live jazz :
www.audionet.com
 JamTV - provides daily commercial
music news, chat, and online music
store: www.jamtv.com
 LiveConcerts - music concerts and
archives old shows. www.liveconcerts.com
Webcast Video Examples
C-Span Online - Live video of House
and Senate events
www.c-span.org
Fox News - www.foxnews.com
WebActive - news station
www.webactive.com
Technologies Needed for
Internet Broadcasting
Techniques and protocols for
efficient transmission of multimedia
data over the Internet.
Real-time audio and video
compression techniques that
provide high compression ratios
and high quality.
Present Approaches in Data
Broadcasting Over the Internet
IP Unicast - transmits data from a
sender to a single receiver
IP Broadcast - transmits data from a
sender to an entire subnetwork
IP Multicast - transmits data from a
sender to a set of receivers that are
members of a multicast group in
various scattered subnetworks
IP Unicast
 Many current radio and television broadcast
applications use unicast data transmission
 Connection-oriented stream transports are used
to distribute data to each receiver individually.
Clients
Sender
Network
IP Multicast
 Sender sends a copy of data to multiple
receivers, which are members of the
same multicast group.
Client
SUBNETS
SUBNET
Multicast
Router
SERVER
Client
Client
SUBNET
Client
IP Simulcast - An Innovative Technique
for Internet Broadcasting
IP Simulcast reduces the server (or
sender) overhead by distributing the
load to each client (or receiver)
Each receiver becomes a repeater,
which rebroadcasts its received
content to two child receivers
The needed network bandwidth for
the server is significantly reduced
Broadcast Pyramid Applied
in IP Simulcast
SERVER
Client
1
Client
3
Client
2
Client
4
Client
5
Client
6
Client
2
Client
7
2
Client
8
Client
9
Client
10
Client
11
Client
12
Client
13
Client
14
IP Simulcast - An Innovative Technique
for Internet Broadcasting
IP Simulcast reduces the server (or
sender) overhead by distributing the
load to each client (or receiver)
Each receiver becomes a repeater,
which rebroadcasts its received
content to two child receivers
The needed network bandwidth for
the server is significantly reduced
Characteristics of IP Simulcast
 It is a radically different model of digital
broadcast, referred to as repeater-server
model
 The server manages and controls the
interconnection of repeaters
 Each repeater not only plays back the
data stream, but also transmits the data
to two other repeaters
 IP Simulcast provides guaranteed
delivery of packets, which is not the case
with IP Multicast
Comparison With
Other Approaches
FEATURES
UP UNICAST
UP MULTICAST
UP SIMULCAST
Server Bandwidth
162 Mbps
1.62 Mbps
16.2 Kbps
Bandwidth Cost
$100,000 per month
$20,000 per month
$100 per month
Error Recovery
By server
By server
By client
Initial Server Cost
$53,000
$8,000
$5,000
Client
Reachability
Implementation
Issues
Any IP address
Only clients in
proprietary network
Any IP address
Cannot scale to serve
increasing number of
clients
Requires all intermediate IP
Multicast routers
Requires special network
card and software which
supports IP Multicast
Easy to implement
Does not require any
special cards or routers
Pipe Dream’s Radio Player
XYZ - New Video
Compression Technique
The XYZ video compression
algorithm is based on 3D Discrete
Cosine Transform (DCT)
It provides very high compression
ratios and excellent video quality
It is very suitable for real-time video
compression
Forming Video Cube for
XYZ Compression
Frames
0
1
8
8
2
8
8
3
8
8
4
8
8
5
8
8
8X8X8
video
cube
6
8
8
7
8
8
8
8
Block Diagram of
the XYZ Compression
Quantizing
Tables
Video
Cube
Forward
3-D
DCT
Huffman
Table
Quantizer
Entropy
Coder
Compressed
video
sequence
Key Encoding Equations
 Both encoder and decoder are symmetrical, which makes
the algorithm suitable for VLSI implementation
7
7
7
F (u, v , w)  C u C v  C w    f  x , y , z
x 0 y 0 z 0


cos 2 x  1 u  cos 2 y  1v cos 2 z  1 w 
16
16
16
 F u, v, w 
Fq u, v, w  



Q
u
,
v
,
w


(4.1)
(4.3)
XYZ Versus MPEG
Video
Compression
Technique
XYZ
Compression
Ratio
Normalized
RMS
Error
Execution
Time [min]
(8 frames, 320x240)
34.5
0.079
6.45
57.7
0.097
6.45
70.8
0.105
6.45
101.7
0.120
6.45
128.1
0.130
6.45
11.0
0.080
21.35
15.6
0.080
163.0
27.0
0.140
21.35
32.9
0.125
163.0
(QT1)
XYZ
(QT2)
XYZ
(QT3)
XYZ
(QT4)
XYZ
(QT5)
MPEG
Logarithmic Search and
Error Correction
MPEG
Exhaustive Search and
Error Correction
MPEG
Logarithmic Search and
No Error Correction
MPEG
Exhaustive Search and
No Error Correction
Complexity of Video
Compression Techniques
Compression
Algorithm
H.261/H.263
MPEG
No B Frames
MPEG
70% B Frames
XYZ
Encoder
Complexity
Decoder
Complexity
Total
Complexity
970
750
200
100
1,170
850
1,120
120
1,240
240
240
480
XYZ Versus MPEG
MPEG
Cr=11,NRMSE=0.08
MPEG, Motion Est. only
Cr=27, NRMSE=0.14
Original
XYZ
Cr=45, NRMSE=0.079
Examples of XYZ Compression
Original
XYZ-compressed
Cr=51
Examples of XYZ Compression
Original
XYZ-compressed
Cr=110
Characteristics of
XYZ Video Compression
XYZ gives significantly better
compression ratios than MPEG for
the same quality of video
For similar compression ratios, XYZ
gives much better quality than
MPEG
XYZ is faster than MPEG (lower
complexity)
XYZ is simple for implementation
Applications of the XYZ
TV phone
Video broadcasting on the Internet
Video-on-demand applications
Interactive TV
Videoconferencing
TV Phone
 Videophone is a box on the top of TV with a
small camera, modem, and video/audio
codec.
Conventional
telephone network
Videophone
Videophone
TV set
TV set
Design of the TV Phone
Camera
VIDEOPHONE
Video/audio capture
and compression
Video-in
Video/audio out
VLSI chips
implement XYZ
algorithm
TV
Video/audio out
Modem
Video/audio
decompression
and conversion
to TV format
User's
control panel
Telephone
lines
Video/audio in
A Fast Content-Based
Multimedia Retrieval Technique
Two main approaches in indexing
and retrieval of images and videos
Keyword-based indexing and
retrieval
Content-based indexing and
retrieval
Keyword-Based
Retrieval and Indexing
 Uses keywords or descriptive text, which
is stored together with images and
videos in the database
 Retrieval is performed by matching the
query, given in the form of keywords,
with the stored keywords
 This approach is not satisfactory - the
text-based description is incomplete,
imprecise, and inconsistent in specifying
visual information
New Algorithm for SimilarityBased Retrieval of Images
 Images in the database are stored as JPEGcompressed images
 The user submits a request for search-bysimilarity by presenting the desired image.
 The algorithm calculates the DC coefficients of
this image and creates the histogram of DC
coefficients.
 The algorithm compares the DC histogram of
the submitted image with the DC histograms
of the stored images.
Histogram of DC Coefficients
for the Image “Elephant”
Comparison of Histograms
of DC Coefficients
Example of Similarity-Based
Retrieval Using the DC Histograms
Similarity-Based Retrieval of
Compressed Video
Partitioning video into clips - video
segmentation
Key frame extraction
Indexing and retrieval of key frames
DC Histogram Technique
Applied for Video Partitioning
40
35
30
NSD x 100 [%]
25
20
15
10
5
0
1
2
3
4
5
6
7
8
9
10
11
12
13
Frame number
14
15
16
17
18
19
20
21
22
23
24
Example of Similarity-Based Retrieval
of Key Frames Using DC Histograms
Interactive Progressive
Encoding System
 Users submit requests for imagery to the
image database via a graphical user
interface
 Upon an initial request, a DCT image
(version of the image based on DC
coefficients only) is transmitted and
reconstructed at the user site.
 The user can then isolate specific
regions of interests within the image and
request additional levels of details.
Band Transmission in Interactive JPEG
System Based on Spectral Selection
All blocks of the image
Amplitude of
Coefficients
Blocks of the selected region of the image
Band-1
DC
Transmission
Band-2
AC1
AC2
Band-3
AC3 …… AC6
Band-4
AC7 ………. AC63
Prototype System - IPES and
Experimental Results
Original image “Airport”
Interactive Progressive
Transmission in Four Scans
Selection of Two Regions
Cumulative transmitted bits [Kbits]
Cumulative Number of
Transmitted Bits
2000
1800
1600
1400
Whole Image
1200
1000
800
1 Region
2 Regions
600
400
200
0
1
2
3
Scans
4
Extracted Images From
a Group of Images
Applications
 Retrieval and transmission of complex
images over low bandwidth
communication channels (image
transmission over the Internet, real-time
transmission of medical images)
 Archiving and browsing visually lossless
image databases (medical imaginary,
space exploration and military
applications)