Transcript Test - Ultra-Videoconferencing

History of Internet Audio
Experiments at McGill (and elsewhere)
Jeremy R. Cooperstock
Centre for Intelligent Machines
Centre for Interdisciplinary Research
in Music Media and Technology
McGill University
ANET Workshop
Software available from:
http://ultravideo.mcgill.edu
Research supported by:
ANET Workshop
Some Early (Pre-)History
Eve Schooler: Distributed Music: A Foray into Net Performance (Sept. 1993)


synchronized three real-time streams from different hosts; delays in the order of 200 ms
difficult for performers to be listeners
Paul Hoffert: CyberSoiree (Feb. 1996)


ATM-based technology for audio and video streaming of a four-way jazz performance
delay >0.5s delay but musicians earned to compensate through extensive practice
Dimitri Konstantas: Distributed Musical Rehearsal Studio (May 1996)


ATM based distributed rehearsal with conductor at different location from musicians
80 ms one-way delay for audio-video synch; echo resulted in "extreme confusion"
Seiji Ozawa: Opening Ceremony Nagano Winter Olympics (1998)


conduct choruses on 5 continents: singers in Sydney, New York, Beijing, Berlin, False Bay
time lag adjustor used to eliminate satellite delay
ANET Workshop
World's first Internet AC-3 stream
Montreal-New York for the AES 107th Convention, Sept. 26, 1999
ANET Workshop
System Diagram
AC-3 (640 Kbps)
(5.1 channels,16
bits @ 48kHz)
coded in AES/EBU
SGI Indy
Dolby encoder
SGI Indy
Cisco IP/TV
PC Server
McGill University
Montreal, Canada
ANET Workshop
Network
PC Client
NYU Cantor Hall
New York, USA
In action
Sender
Receiver
Read
Send
Audio
Queue
ANET Workshop
Receive,check,
playback
Transmit
Receive
Audio
Queue
Recording Studio that Spanned a
Continent
Montreal-Los Angeles for the AES 109th Convention, Sept. 23, 2000
12 channels,
96kHz, 24
bits/channel
USC Norris Theatre, LA
McGill Redpath Hall, Montreal
ANET Workshop
Microphone Configuration
ANET Workshop
3 Mbps
ANET Workshop
27.6 Mbps
ANET Workshop
Some Modern History
Internet2 Initiative: World's First Remote Barbershop Quartet (Nov. 2000)


multi-location quartet; each of the 4 singers in different cities, conductor in 5th
network delay variances prevented singers from hearing each other or conductor
Internet2 Initiative: Music Video Recording via Internet2 (Nov. 2000)


multi-location music video recording session using real-time streaming video
musicians simultaneously connected via timing tracks to a mixing board
Chris Chafe: QoS Enabled Audio Teleportation (Nov. 2000)


CD quality sound (750 kbps) of 2 separated musicians in Dallas streamed to Stanford
musicians played "together" in same space (Stanford) but delay was severe
John Wawrzynek: Network Musical Performance (May 2001)


gestural coding (e.g. MIDI) used to manage data for distributed musical performance
musicians at Berkeley and CalTech, playing on MIDI keyboards; local feedback only
ANET Workshop
"Frères Jacques" experiment
Montreal, RISQ 2000 Conference
"low-latency" MPEG-2 videoconferencing
ANET Workshop
"Playing Together" experiment
New York-Ottawa, Remote Masters class, Dec. 8, 2000
Pinchas
Zuckerman,
Columbia U, NYC
Wu Ji,
Canarie ARDNOC,
Ottawa
used Litton MPEG-2 codec @ 10 Mbps
ANET Workshop
Quality vs. Latency
Bandwidth (Mbps)
 near-equivalent quality of
encoded video at greatly
reduced bandwidth
 but... compression takes
time
300
250
200
150
100
50
0
uncompressed
SDI
ANET Workshop
M-JPEG/DV
MPEG-4
Tolerated Synchronicity
musical
activity
small
ensemble
large
ensemble
lip
synch
compress
acquire
transmit
MPEG*
0
networks
10
speed of
light: Mtl-LA
20
30
40
50
60
70
80ms
net latency
Mtl-LA
* This figure is a theoretical “best-case” scenario. Actual performance may vary based on local hardware. The
manufacturer makes no warranties, either express or implied, and will not be held liable for damage resulting
from use of the product. Certain restrictions may apply. Offer void where prohibited by law. Batteries not
included.
ANET Workshop
or put another way...
Latency:
The Interaction-Killer
Interactive Success!
Chris Chafe: SoundWIRE (August 2001)
 streaming pro audio from remote locations for musical collaboration
 UDP/IP streaming: good results between Armonk, NY and Stanford
 round trip delay 125ms, musicians able to "catch-up" during pauses
ANET Workshop
The future according to Sympatico?
ANET Workshop
For those with DSL at home...
yeah, right!
(tu n'es pas sérieux!)
ANET Workshop
So what does it take?
 much bigger network pipes ("abusive bandwidth")
 need Fast Ether (100Mbps) for audio
 GigE (1000Mbps) for video
 more efficient network topologies – hops add delay
ANET Workshop
So what does it take?
 much bigger network pipes ("abusive bandwidth")
 need Fast Ether (100Mbps) for audio
 GigE (1000Mbps) for video
 more efficient network topologies – hops add delay
 more powerful hardware
 CPU speed (fast process switching, especially for mixed media)
 pro-audio interfaces with small buffers
ANET Workshop
So what does it take?
 much bigger network pipes ("abusive bandwidth")
 need Fast Ether (100Mbps) for audio
 GigE (1000Mbps) for video
 more efficient network topologies – hops add delay
 more powerful hardware
 CPU speed (fast process switching, especially for mixed media)
 pro-audio interfaces with small buffers
 more masochistic researchers
ANET Workshop
Sometimes things work...
ANET Workshop
Sometimes things work...
note: look of
extreme anguish
and sometimes they don't
ANET Workshop
Distributed Violin Duet
McGill-Université de Montréal for RISQnet Conference, Nov. 21, 2001
 connected two
Montreal universities
via IP over 1 Gbps
fiber link
 uncompressed audio
and video
 < 30ms latency
ANET Workshop
"The medium is the message"
Marshall McLuhan
Cooperstock's Corollary:
"The medium must be faster than the message."
ANET Workshop
Size does matter
Montreal-Ottawa (NRC), Distance Teaching Trial, Feb. 20, 2002
 connected McGill to
National Research
Council (Ottawa) with
SDI video and
multichannel
96kHz/24bit audio
 display on 50" plasma
screen (near life-size)
ANET Workshop
Distributed Jazz Jam
Montreal-Stanford (CCRMA), Research Demo, Oct. 24, 2002
 uncompressed audio and
video @ < 50ms latency
ANET Workshop
Remote Video Interpretation (Sign Language)
Montreal-Fredericton (UNB), research trials, 2003
 DV (patient, physician)
exchanged with remote
sign-language
interpreter
physician's
setup
patient's setup
ANET Workshop
Vibrosensory Transmission
VRQ demo, March 8, 2004
ANET Workshop
Coming next: DSD Transmission
AES demo, October 31, 2004
ANET Workshop
Milestones Ahead




multicast
traffic shaping
region-of-interest client selection
multichannel echo-supression/cancellation
ANET Workshop
Self-Signal Isolation: Echo-suppression
 the big problem in videoconferencing
 same issue for video and vibrosensory data
achieving high-fidelity distributed interaction, at
levels of presence and latency that support the
most demanding applications and doing so in spite
of sensor and bandwidth limitations
http://ultravideo.mcgill.edu