Transcript 2007-10-IROS-Sharecam

ShareCam:
Interface, System Architecture, and Implementation of a
Collaboratively Controlled Robotic Webcam
Dezhen Song (TAMU) and Ken Goldberg (UC Berkeley)
IEEE/RSJ International Conference on Intelligent Robots and Systems
Finalist for New Technology Foundation (NTF) Award for
Entertainment Robots and Systems
Teleoperation: Related Work
•
•
•
•
•
•
•
•
•
•
•
•
Tesla, 1898
Goertz, ‘54
Mosher, ‘64
Tomovic, ‘69
Salisbury,Bejczy, ‘85
Ballard, ’86
Volz, ’87Sheridan, ‘92
Sato, ’94
Goldberg, ’94Presence Journal ‘92O. Khatib, et al. ’96
Internet
networked
robot:
Taxonomy (Tanie, Matsuhira, Chong 00)
Single Operator, Single Robot (SOSR):
Single Operator, Multiple Robot (MOSR):
Multiple Operator, Single Robot (MOSR):
sharecam
Pan, Tilt, Zoom
robotic video
camera
Entertainment Applications
Playing Games
Related Work
• Networked robots
–
–
–
–
–
–
–
Tanie, K., Chong, N. et al(01)
Jia, S. and K. Takase (01)
Hu, H., Yu, L., Tsui, P., Zhou, Q (01)
Safaric, R. et al. (01)
Goldberg and Siegwart (02)
Coppin, P. and Wagner, M.D. (02)
Konukseven, I., Erkmen, A. et al (02)
• SOSR
–
–
–
–
–
Siegwart, R. and Saucy P. (99)
Paulos, E. and Canny, J. (99)
Tanie, K., Arai, H. et al. (00)
Lynch, K. and Liu, C. (00)
Fong, T., Thorpe, C., et al(01)
Related Work
• SOMR
– Hu, Yu, Tsui, Zhou (01)
– Jia, Takase (01)
• MOMR
– Fukuda, Xi, Liu, Elhajj et al. (00,02)
– Tanie, Chong, et al. (00)
• MOSR
– Cinematrix (91)
– Cannon, McDonald, et al. (97)
– Goldberg, Chen, et al. (00, 01)
Sharecam: System Architecture
Users
ShareCam Server
Internet
Video Server
Sharecam Software
Login CGI
Console/Log
Communication
Java
Core (with shared
memory segments)
Apache
module
Apache
module
User
database
Registration
MySQL
Apache
module
ShareCam web server
TCP/IP
Calibration
Camera control
RS232C
Canon VCC3 Camera
Gnu C++
InetCam server
Panoramic image
generation
HTTP
ShareCam applet
PERL
TCP/IP
Video server
InetCam applet
Client
frame selection problem:
given n requests, find optimal frame
Problem Definition
Requested frames: i=[xi,
yi, zi], i=1,…,n
Problem Definition
• “Satisfaction” for frame i: 0  Si  1
 =   i
 = i
Si = 0
Si = 1
Similarity Metrics
• Symmetric Difference
Area (i   )  Area (i   )
SD 
Area (i   )
• Intersection-Over-Union
Area (i   )
IOU 
 1  SD
Area (i   )
Nonlinear functions of (x,y)
Intersection over Maximum:
s (i ,  )  ( pi / ai ) min(( zi / z ) ,1)
b
pi
Area (i   )


max( ai , a)
Max (i ,  )
Requested
frame i ,
Area= ai
Candidate
frame 
Area = a
pi
• global satisfaction:
n
S ( )   ( pi / ai ) min(( zi / z ) b ,1)
i 1
n
S ( x, y )    i pi ( x, y )
i 1
for fixed z
find  * = arg max S()
approximation
Compute S(x,y) at lattice of sample points:
y
d
d : lattice spacing
d z : resolution spacing
whg
O( 2 n)
d dz
x
w, h : width and height,
g: size range
error bound
c*
Optimal frame
ĉ
Smallest frame on
lattice that encloses c*
c~
Optimal at lattice
s(c )  s(c~)  s(cˆ)
s( c~) s(cˆ)
1

*
s ( c ) s ( c* )
*
run time:
O(n /  3 )
zmin
 ... 
zmin  2d z
 1 
frame selection algorithms
Processing
Zoom
Type
Complexity
Centralized Discrete
Exact
O(n2)
Centralized Discrete
Approx
O(nk log(nk)), k=(log(1/ε)/ε)2
Centralized Contin.
Exact
O(n3)
Centralized Contin.
Approx
O((n + 1/3) log2 n)
Distributed
Discrete
Exact
O(n), Client: O(n)
Distributed
Contin.
Approx
O(n), Client O(1/3)
ShareCam Application:
Game based learning : global environment
Collaborative Observatories
for Natural Environments (CONEs)
sensor
networks
humans:
amateurs
and profs.
timed checks
robotic video
cameras
motion sensors
Thank you.
ShareCam: Interface, System Architecture, and Implementation of a
Collaboratively Controlled Robotic Webcam
Frank van der Stappen (CS, Utrecht)
Vladlen Koltun (EECS, UC Berkeley)
George Bekey (CS, USC)
Karl Bohringer (CS, UW)
Anatoly Pashkevich (Informatics, Belarus)
Judith Donath (Media Lab, MIT)
Eric Paulos (Intel Research Lab, Berkeley)
Dana Plautz (Intel Research Lab, Oregon)
Sariel Har-Peled (CS, UIUC)
Satellite Imaging
MIT Press, 2002
Networked Robots
•
•
•
•
•
•
Tele-Operation
Internet Tele-Operation
Collaborative Tele-Operation
Tele-Actor
Co-Opticon
Co-Opticon Algorithms
www.ken.goldberg.net
Infiltrate
Networked
Robots
internet
tele-robot:
RoboMotes: Gaurav S. Sukhatme, USC
Smart Dust: Kris Pister, UCB (Image: Kenn Brown)
Networked
Cameras
Where to look?
Sensornet detects activity
• “Motecams”
• Other sensors:
audio, pressure switches,
light beams, IR, etc
• Generate bounding boxes
and motion vectors
• Transmit to PZT camera
Activity localization
1.
2.
3.
4.
5.
Network Standards: HTML, Browsers, Java
Infrastructure: Backbone, Routers, ADSL
Public Adoption
Bandwidth: 10 Mbps, 100Mbps, Gbps
Video/Audio Compression: MP2,3,4
Networked robots
Systems that couple communication to control
of one or more robots in a network that often
includes sensors and remote human operators.
Two subclasses :
1) Tele-operated, where human supervisors send
commands and receive feedback via the network. Such
systems support research, education, and public
awareness by making valuable resources accessible to
broad audiences.
2) Autonomous, where robots and sensors exchange data
via the network. In such systems, the network extends
the effective sensing range of individual robots.
Challenges:
1.
2.
3.
4.
5.
6.
7.
8.
Variable Time Delays, Congestion
Latency
Access Control, Security, Interfaces
Protocol Design
Noise, Error Detection and Recovery
Deployment, Dynamic Routing
Power Management
Hybrid Architectures
Conventional Security Cameras
• Immobile or Repetitive Sweep
• Low resolution
Future Work
• Continuous zoom (m=)
• Multiple outputs:
– p cameras
– p views from one camera
• “Temporal” version: fairness
– Integrate si over time: minimize accumulated
dissatisfaction for any user
• Network / Client Variability: load balancing
• Obstacle Avoidance
Outline
•
•
•
•
Collaborative Teleoperation
Cinematrix
Co-opticon
Tele-Twister
"In times of terror, when everyone is
something of a conspirator,
everyone will be in a situation where
he has to play detective."
-- Walter Benjamin (1938)
Statistics of Satellite Imaging
• 2.5 Billion Market in 2003
• Increasing 300% per year since
1999
• Major clients
– Government / Military
– Oil exploration
– Weather Prediction
– Agriculture
Ikonos, 1999
Intersection over Maximum: si( ,i)
Requested frame i
Candidate frame 
si = 0.20
0.21
0.53
• Staircase Approximation
– Exact algorithm O(n3/2 log3 n)
[data structure]
– Approximation Algorithm O(nk log(nk)), k=(log(1/ε)/ε)2
• Staircase approximation, large constant factors