Transcript T - Web3D 2011

MPEG-4-based adaptive remote
rendering for video games
Authors: N. Tizon, C.Moreno, M.Cernea, M.Preda
Dept. ARTEMIS: http://www-artemis.it-sudparis.eu/
Contact: [email protected]
Outline
 Remote
rendering approaches
 The
Kusanagi framework
• The Kusanagi plug-in
• The lobby server
• The MPEG-4 client
 Experimentation
and optimizations
• User experience
• Latency measurement and adaptation
page 1
direction ou services
<pied de page>
Remote rendering approaches
page 2
direction ou services
<pied de page>
Video streaming based solution ?
 Six
main families of solutions:
Constraint
Approach
Limited
Bandwidth
~
Constant
Bitrate
High
Quality
Thin Client
Graphic Commands
+
-
+
-
2D Pixels
-
+
+
+
2D Primitives
+
+
-
+
3D Vectors
+
+
-
+
Simple 3D Object
+
+
+
-
Multiples 3D Objects
+
+
+
-
page 3
direction ou services
<pied de page>
The Kusanagi framework
I.
II.
III.
page 4
direction ou services
The Kusanagi plug-in
The lobby server
The MPEG-4 client
<pied de page>
Architecture overview
page 5
direction ou services
<pied de page>
Kusanagi plugin
 From
the 3D graphic application to the remote
controlled software:
• Image & Audio Capture
• Interactivity commands between the client and the server
Net-input HTTP server
• Audio and video data from server to client.
RTSP
server
Send data to:
IP_client
Game +
Stream Lib.
Send data to:
localhost
page 6
direction ou services
Client
Open: rtsp://IP_server/sdp_file
<pied de page>
The lobby server

Interface between:
• the users connected through the
network,
• the 3D graphic application,
• the hardware components and/or
third applications.
 Resource
manager:
• the network interface for the user
connections and data transport,
• the CPU and the graphic card for
the rendering and the encoding,
• the audio card for the audio mixing and capture.
page 7
direction ou services
<pied de page>
The MPEG-4 client
 Full
multimedia player with networking, media,user
interactions management:
• MPEG-4 scene description language
• H.264/AVC and RTP/RTSP
 Standard
page 8
solution => Cross platform approach
direction ou services
<pied de page>
Experimentation and
optimizations
I.
II.
page 9
direction ou services
User experience
Latency measurement and adaptation
<pied de page>
User experience
«
Beta tests » approach (subjective evaluation)
• Area: Parisian region, 53 users,1 month.
• Point&click game => low constraint for the latency.
• TRA (Theory of Reasoned Action, Fishbein and Ajen): 92%
of positive attitude towards the Kusanagi service.
 Improvements
• Availability on different devices (TV, mobile phone).
• People are sensitive to image quality and latency => adaptive
video streaming.
page 10
direction ou services
<pied de page>
Latency measurment
 RTT(t)
estimated from RTCP receiver reports
 Round Trip Time (RTT) estimation:
RTT (t )  RTTi  RTTc (t )
Intrinsic component, provided at the
beginning by the lobby server (ping)
Network infrastructure based
parameter (routers, transport channel..)
 RTTi
Congestion based component
Depends on users activity.
=> prone to adaptation
update:
• RTT relative standard deviation: RSD = σ/µ
• If RTT variability is low (RSD< RSDth) => RTT(t) ~ RTTi
page 11
direction ou services
<pied de page>
Bitrate adaptation
RTTi update
Qp increase
Qp decrease
page 12
direction ou services
<pied de page>
Qp factor adaptation when
congestion is detected
Expirmental results: background
 Testing
conditions
• Video encoding: VGA, 30fps, H.264 low latency encoding (no Bframes, CAVLC, no rate control buffering), Qp0  20 (~5Mbps, 50dB)
• Network: bandwidth and delay(RTT) monitoring
• Algorithm parameters: α=0.1, β=1 and θ=0.9

RTT measurments without adaptation (RTTi=20ms, 5Mbps)
Peak bitrates > 5Mbps
=> Congestion
=> Playability not acceptable
page 13
direction ou services
<pied de page>
Intrinsic delay estimation
 Measured
RTT and PSNR (RTTi = 20ms and 80ms,
bandwidth:5Mbps)
70s to discover the
new RTTi value
page 14
direction ou services
<pied de page>
Bitrate adaptation
 Measured
RTT and PSNR
(RTTi = 20ms, bandwidth:2.5Mbps)
30s to reach a stable
state: RTT=RTTi
page 15
direction ou services
<pied de page>
Conclusions

A complete framework for remote rendering applied on video
games.

Three main modules: the Kusanagi plug-in, the lobby server, the
MPEG-4 client.

Beta tests: need of more adaptability in order to improve the user
QoE.

An efficient method to detect congestion based latency and to
adapt the video bitrate adequately.

Futur works:
• To optimize the resource sharing
• To optimize video encoding with side information from the 3D engine.
page 16
direction ou services
<pied de page>