Transcript PPT - Communications and Multimedia Laboratory

Real Time Media Synchronization
on H.323 Multimedia Conference
System
H.323多媒體會議系統之即時媒體同步機制
Amy, Mei-Hsuan Lu
CML/CSIE/NTU
April 16, 1999
Outline
Introduction to H.323
H.323 Media Synchronization
Mechanism
System Architecture
Conclusion
1999/4/16
Amy, Mei-Hsuan Lu
CML/CSIE/NTU
2
H.323 Protocol Stack
Control
Data
Audio Video A/V Cntl
G.7xx
H.225.0 H.245
Gatekeeper
H.26x
RTCP
T.120
RTP
TCP
Control
Reg,
Adm,
Status
(RAS)
UDP
IP
1999/4/16
Amy, Mei-Hsuan Lu
CML/CSIE/NTU
3
H.323 Direct Call Model
RAS
RAS
RAS
Q.931
H.245
GK1
GK2
PC1
RTP
RTCP
RTP
RTCP
PC2
 We use timestamp value in RTP/RTCP header and
NTP value in RTCP header to perform intra-stream
and inter-stream synchronization
1999/4/16
Amy, Mei-Hsuan Lu
CML/CSIE/NTU
4
Terminology
Timestamp
Master/Slave Streams
Initial Delay
Eligible Time
Receive Path Delay
1999/4/16
Amy, Mei-Hsuan Lu
CML/CSIE/NTU
5
H.323 Terminal Equipment
Scope of Recommendation H.323
Video I/O equipment
Video Codec
H.261, H.263
Receive
Path
Delay
Audio I/O equipment
Audio Codec
G.711, G.722,
G.723, G.728,
G.729
H.225.0
Layer
Local Area
Network
Interface
User Data Applications
T.120, etc.
System Control
H.245 Control
System Control
User Interface
Call Control
H.225.0
RAS Control
H.225.0
1999/4/16
Amy, Mei-Hsuan Lu
CML/CSIE/NTU
6
Notation (1/2)
 (i, j)
 ET(i, j)
 ETM(i, j)
 A(i, j)
D
 TS(i, j)
1999/4/16
the i-th packet in stream j
eligible time of packet (i, j)
eligible time of the head-of-line packet
in i’s master stream M
head-of-line arrival time of packet (i, j)
System Initial Delay
source timestamp of packet(i, j)
Amy, Mei-Hsuan Lu
CML/CSIE/NTU
7
Notation (2/2)
 ETh(j)
 Ah(j)
 TSh(j)
 Ps(i, m)
B
1999/4/16
eligible time of the head-of-line packet
in stream j
A(i, j) of head-of-line packet i in stream j
TS(i, j) of head-of-line packet i in stream j
Receive path delay added to packet (i, m)
for inter-stream synchronization between
master stream m and salve stream s
WaveOut device buffer occupancy level
Amy, Mei-Hsuan Lu
CML/CSIE/NTU
8
Media Sync. Mechanism
Design 1
ET(i, M) := MAX(D + TS(i, M) - TS(1, M), A(i, M))
ET(i, S) := ETM(S) + TS(i, S) - TSM(S)
 When Delay is controlled by D, Synchronization is
perfect
 System may drop late packets of slave stream
 When slave stream falls behind master stream too
much, synchronization is impossible
1999/4/16
Amy, Mei-Hsuan Lu
CML/CSIE/NTU
9
Media Sync. Mechanism
Design 2
ET(i, M) := MAX(D + TS(i, M) - TS(1, M), A(i, M) + PS(i, M))
ET(i, S) := ETM(S) + TS(i, S) - TSM(S)
wherePS(i, M) = (Ah(S) - A(i, M)) - (TSh(S) - TS(i, M))
 Receive path delay forces master stream to “wait
for” slave stream
 When Ps is larger than some threshold, for
interactive communication, system can still play
master stream directly
1999/4/16
Amy, Mei-Hsuan Lu
CML/CSIE/NTU
10
Media Sync. Mechanism
Design 3
ET(i, M) := MAX(D + TS(i, M) - TS(1, M), A(i, M) + PS(i, M))
ET(i, S) := MAX(ETM(S), B*30 + A(i, S)) + TS(i, S) - TSM(S)
 Since eligible time is not the real playback time for
audio media, we modified Design 2 accordingly as
above
 We have to handle the problem of audio
accumulative delay !
1999/4/16
Amy, Mei-Hsuan Lu
CML/CSIE/NTU
11
Accumulative Delay
 Drop continuos silence frames upon
reception
 Drop data frames when real-time property is
seriously destroyed
Drop Data Packets for
Real-time Constraint
Threshold for Real-time Control
H pkts
1999/4/16
Audio
Buffer
Audio
Decoder
Video
Buffer
Video
Decoder
0
Audio
Device
WaveOut Buffer
Amy, Mei-Hsuan Lu
CML/CSIE/NTU
Display Device
12
System Architecture
Audio I/O
Equipment
User Interface
System Control Unit
Call Control
H.225.0
RAS Control
H.225.0
Q.931
Video I/O
Equipment
Media Control Unit
H.245
Control
Audio Codec
G.723.1
Video Codec
H.263
H.225.0 Layer
Local Area Network Itnerface
1999/4/16
Amy, Mei-Hsuan Lu
CML/CSIE/NTU
13
Conclusion
 We propose a media synchronization mechanism for
H.323 systems
 We formulize the concept of receive-path-delay in
H.323, and resolve the problem of late slave streams
 By reference to “real” playback time, we refine our
mechanism to achieve more precise
synchronization
 With WaveOut buffer control, We can avoid audio
accumulative delay and maintain real-time property
1999/4/16
Amy, Mei-Hsuan Lu
CML/CSIE/NTU
14