Transcript ucsb08 - Columbia University

The Vision and Reality of Ubiquitous
Computing
Prof. Henning Schulzrinne
Dept. of Computer Science
Columbia University
(with Arezu Moghadam, Ron Shacham, Suman Srinivasan,
Xiaotao Wu and other IRT members as well as the IETF
GEOPRIV and ECRIT WGs)
January 2008
UCSB
1
Overview
•
•
•
•
•
•
The original vision of ubiquitous computing
User challenges
Beyond terminal mobility
Location as new core service
Beyond connectivity: 7DS
CS technology into reality
January 2008
UCSB
2
Ubiquitous computing  ubiquitous
communications
• “It is invisible, everywhere computing that does not live on a
personal device of any sort, but is in the woodwork everywhere.”
• Weiser’s original vision (“Nomadic Issues in Ubiquitous Computing”,
1996)
–
–
–
–
–
“one person, many computers”
many computers embedded in environment
dynamic ownership
PC phonebooth
“IR use will grow rapidly”
• Updated version, 2007
–
–
–
–
–
not physically invisible, but transparent
emphasis on communications, not computing
most devices are mobile (or nomadic)
cheap electronics personal devices
radio (channelized and UWB)
January 2008
UCSB
3
Overview
• The original vision of ubiquitous
computing
• User challenges
• Beyond terminal mobility
• Location as new core service
• Universality: 7DS
• CS technology into reality
January 2008
UCSB
4
User challenges vs. research challenges
• Are we addressing real user needs?
– Engineering vs. sports scoring
• My guesses
ease of use
reliability
no manual
no re-entry
no duplication
integration
phishing
data loss
cost
January 2008
limited risk
UCSB
5
Example: Email configuration
• Application configuration
for (mobile) devices painful
• SMTP port 25 vs. 587
• IMAP vs. POP
• TLS vs. SSL vs. “secure
authentication”
• Worse for SIP...
January 2008
UCSB
6
Example: SIP configuration
•
•
•
•
•
partially explains
highly technical parameters, with differing names
inconsistent conventions for user and realm
made worse by limited end systems (configure by multi-tap)
usually fails with some cryptic error message and no
indication which parameter
out-of-box experience not good
January 2008
UCSB
7
Mobile why’s
•
•
•
•
•
•
•
•
•
Not research, but examples of real annoyances
Why does each mobile device need its own power supply?
Why do I have to adjust the clock on my camera each time I travel?
Why do I have to know what my IMAP server is and whether it uses
TLS or SSL?
Why do I have to type in my address book?
Why do I have to “synchronize” my PDA?
Why do I have to manually update software?
Why is connecting a laptop to a projector a gamble?
Why do we use USB memory sticks when all laptops have 802.11b?
January 2008
UCSB
8
Consumer wireless & mobile devices
Prius key
Garage door opener
TAN display
Water leak alarm
SPOT watch
wireless door bell
MSN Direct weather
January 2008
UCSB
9
Mobile systems - reality
GPS
•
idea: special purpose (phone) --> universal
communicator
–
•
idea is easy...
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
mobile equipment: laptop + phone
–
•
•
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
location data
sufficiently different UI and capabilities
we all know the ideal (converged) cell phone
difficulty is not technology, but integration
and programmability
–
–
–
–
(almost) each phone has a different flavor of
OS
doesn’t implement all functionality in Java
APIs
no dominant vendor (see UNIX/Linux vs.
Microsoft)
external interfaces crippled or unavailable
•
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
e.g., phone book access
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompress ed) dec ompres sor
are needed to s ee this pic ture.
January 2008
UCSB
10
Example: displays and speakers
January 2008
UCSB
11
The mobile ubiquitous challenge
Mobile phone
Mobile Internet access
Interconnected devices
“Internet
of things”
January 2008
UCSB
13
What do we need?
• Standards, not new technology
• Radio connectivity
– 802.11a/b/g/n, 802.15.4 
– better discovery of networks
• Location information everywhere
• Discovery: devices & services
– network-local discovery via Bonjour (mDNS) 
– missing: location-based discovery
• Advanced mobility: session, personal, service
• Event notification
• Data formats
– location, sensor events, ...
January 2008
UCSB
14
Examples of “invisible” behavior
• MP3 player in car automatically picks up new files in home server
• A new email with vcard attachment automatically updates my cell
phone address book
• The display of my laptop appears on the local projector
– without cable or configuration
• I can call people I just met at UCSB
– without exchanging business cards
•
•
•
•
My car key opens my front door
My cell phone serves as a TAN (one-time password) generator
My cell phone automatically turns itself off during a lecture
My camera knows where the picture was taken
January 2008
UCSB
15
An interconnected system
opens doors
generates TAN
incoming call
updates location
time, location
address book
alert, events
any weather service
school closings
January 2008
acoustic alerts
UCSB
16
Thinking beyond 802.11 and UMTS
• Many interesting networks beyond those covered in conferences
– ease of access by researchers vs. importance
– 90% of papers on 802.11b and maybe GPRS, BlueTooth
• New wireless networks
–
–
–
–
–
–
–
–
broadcast instead of unicast -- useful for many ubiquitous applications
S5 for low-rate sensors (city scale)
QuickTime™ and a
Zigbee (802.15.4) for local sensors (20 - 250 kb/s)
TIFF (Uncompressed) decompresso
are needed to see this picture.
FM subcarrier (not really new) -- MSN Direct
FM Radio Data System -- TMC
Sirius / XM
HD radio
paging
January 2008
UCSB
17
Overview
•
•
•
•
•
•
The original vision of ubiquitous computing
User challenges
Beyond terminal mobility
Location as new core service
Universality: 7DS
CS technology into reality
January 2008
UCSB
18
Application-layer mobility
• terminal mobility
– one terminal, multiple network addresses
• Personal mobility
– one person, multiple terminals
– e.g., Grandcentral
• session mobility
– one user, multiple terminals in sequence or in parallel
• service mobility
– services move with user
January 2008
UCSB
19
Session mobility
•
Walk into office, switch from cell
phone to desk phone
– call transfer problem  SIP
REFER
•
related problem: split session
across end devices
– e.g., wall display + desk phone +
PC for collaborative application
– assume devices (or stand-ins) are
SIP-enabled
– third-party call control
January 2008
UCSB
20
How to find services?
•
Two complementary developments:
–
–
smaller devices carried on user instead of stationary
devices
devices that can be time-shared
•
•
•
•
•
•
Need to discover services in local environment
–
SLP (Service Location Protocol) allows querying for
services
•
•
–
–
–
•
“find all color displays with at least XGA resolution”
slp://example.com/SrvRqst?public?type=printer
SLP in multicast mode
SLP in DA mode
Apple Bonjour
Need to discover services before getting to
environment
–
–
–
January 2008
large plasma displays
projector
hi-res cameras
echo-canceling speaker systems
wide-area network access
“is there a camera in the meeting room?”
SLP extension: find remote DA via DNS SRV
LoST to find services by geographic location
UCSB
21
Session mobility
Local Devices
Transcoder
Internet
SLP DA
SLP SA
SLP UA
SIP SM
SIP UA
SIP UA
Correspondent
Node (CN)
January 2008
SIP SM
SLP UA
SIP UA
UCSB
Mobile Node (MN)
SLP
SIP
RTP
22
Overview
• The original vision of ubiquitous
computing
• User challenges
• Beyond terminal mobility
• Location as new core service
• Universality: 7DS
• CS technology into reality
January 2008
UCSB
23
Context-aware communication
•
•
•
context = “the interrelated conditions in which something
exists or occurs”
anything known about the participants in the (potential)
communication relationship
both at caller and callee
time
chronology, uniqueness
capabilities
caller preferences
location
location-based call routing
location events (emergency alerts)
security (access control)
service discovery
activity/availability
presence
sensor data (mood, bio)
privacy issues similar to location data
January 2008
UCSB
24
GEOPRIV and SIMPLE architectures
rule
maker
DHCP
XCAP
(rules)
target
presentity
caller
January 2008
publication
interface
PUBLISH
INVITE
location
server
presence
agent
notification
interface
location
recipient
GEOPRIV
watcher
SIP
presence
SUBSCRIBE
NOTIFY
INVITE
UCSB
callee
SIP
call
25
Presence data architecture
presence sources
PUBLISH
create
view
(compose)
raw
presence
document
XCAP
select best source
resolve contradictions
depends on watcher
XCAP
privacy
policy
composition
policy
(not defined yet)
January 2008
privacy
filtering
draft-ietf-simple-presence-data-model
UCSB
26
Presence data architecture
candidate
presence
document
remove data not of
interest
watcher
filter
post-processing
composition
(merging)
SUBSCRIBE
difference
to previous notification
watcher
January 2008
raw
presence
document
NOTIFY
UCSB
final
presence
document
27
Presence data model
person
“calendar”
“cell”
“manual”
(presentity)
(views)
services
[email protected]
audio, video, text
[email protected]
video
devices
January 2008
UCSB
28
RPID = rich presence
•
•
Provide watchers with better information about the what, where, how of
presentities
facilitate appropriate communications:
–
–
–
•
designed to be derivable from calendar information
–
•
“wait until end of meeting”
“use text messaging instead of phone call”
“make quick call before flight takes off”
or provided by sensors in the environment
allow filtering by “sphere” – the parts of our life
–
don’t show recreation details to colleagues
January 2008
UCSB
29
Rich presence
• More information: for (authorized) people and applications
• automatically derived from
– sensors: physical presence, movement
– electronic activity: calendars
• Rich information:
–
multiple contacts per presentity
•
•
–
–
–
–
–
–
device (cell, PDA, phone, …)
service (“audio”)
activities, current and planned
sphere (home vs. work)
current user mood
surroundings (noise, privacy, vehicle, …)
contact information
composing (typing, recording audio/video IM, …)
January 2008
UCSB
30
Presence and privacy
•
•
All presence data, particularly
location, is highly sensitive
Basic location object (PIDFLO) describes
– distribution (binary)
– retention duration
•
Policy rules for more detailed
access control
– who can subscribe to my
presence
– who can see what when
January 2008
<tuple id="sg89ae">
<status>
<gp:geopriv>
<gp:location-info>
<gml:location>
<gml:Point gml:id="point1“
srsName="epsg:4326">
<gml:coordinates>37:46:30N 122:25:10W
</gml:coordinates>
</gml:Point>
</gml:location>
</gp:location-info>
<gp:usage-rules>
<gp:retransmission-allowed>no
</gp:retransmission-allowed>
<gp:retention-expiry>2003-06-23T04:57:29Z
</gp:retention-expiry>
</gp:usage-rules>
</gp:geopriv>
</status>
<timestamp>2003-06-22T20:57:29Z</timestamp>
</tuple>
UCSB
31
Events as missing Internet capability
• aka PUB/SUB
• Used across applications, e.g.,
–
–
–
–
–
–
–
–
email and voicemail notification
presence
replace RSS (= poll!)
web service completion
emergency alerts (“reverse 9-1-1”)
network management
home control
data synchronization
• Rich research history
– but too complex, optimize the wrong thing
• XMPP and SIP as likely transport candidates
January 2008
UCSB
32
Local Switch
Automatic
Number
Identification
Automatic
Location
Identification
January 2008
Collaboration between
local phone providers and
local public safety agencies
UCSB
33
911 technology failures
•
NY Times (“An S O S for 911 Systems in Age of High-Tech”), 4/6/07:
– “40% of … counties, most of them rural or small-town …, cannot yet pinpoint the
location of the cellphone callers, though the technology to do so has been
available for at least five years.”
“In … Okmulgee, Okla., last November, 4-year-old Graciella Mathews-Tiger died
in a house fire after a 911 operator who lacked the technology to pinpoint the call
misheard the address.”
• Phase II wireless; billions of dollars spent
• In Mississippi, only 1 of out 5 counties
– “As it ages, it is cracking, with problems like system overload, understaffing,
misrouted calls and bug-ridden databases leading to unanswered calls and
dangerous errors.”
• operator (CAMA) trunks, with 8-digit number delivery
• MSAG and ALI databases
January 2008
UCSB
34
Location delivery
GPS
HELD
HTTP
wire
map
DHCP
LLDP-MED
January 2008
UCSB
35
Location, location, location, ...
Voice Service Provider (VSP)
sees emergency call
but does not know caller location
ISP/IAP knows user location
but does not handle call
January 2008
UCSB
36
Options for location delivery
•
Wireless
–
–
–
–
•
GPS
S5 wireless (active sensors + triangulation)
Skyhook (802.11 in urban areas)
HDTV
L2: LLDP-MED (standardized version of CDP + location data)
– periodic per-port broadcast of configuration information
•
L3: DHCP for
– geospatial (RFC 3825)
– civic (RFC 4676)
•
L7: proposals for retrievals: HELD, SIP, …
–
–
–
–
–
for own IP address or by third party (e.g., ISP to infrastructure provider)
by IP address
by MAC address
by identifier (conveyed by DHCP or PPP)
HTTP-based
January 2008
UCSB
37
Locating Caller using LLDP-MED
LLDP-MED stands for: *
* From Wikipedia
Link Layer Discovery Protocol
“a vendor-neutral Layer 2 protocol that allows a network device
to advertise its identity and capabilities on the local network.”
Media Endpoint Discovery
“an enhancement to the LLDP that allows discovery of other
things including location “
“I am LLDP-MED Capable.
I can process location information.”
LLDP-MED
SWITCH
CALLER
EQUIPMENT
January 2008
UCSB
“Your location is:
38
500 W 120TH st. New York NY 10027”
Location determination options
Method
CDP or LLDPMED
DHCP
HELD
GPS
manual entry
Layer
L2
L3
L7 (HTTP)
-
user
advantages
• simple to
implement
• built into switch
• direct port/room
mapping
• simple to
implement
• network
locality
• traverses
NATs
• can be
operated by
L2 provider
• accurate
• mobile
devices
• no carrier
cooperation
• no
infrastructure
changes
• no carrier
cooperation
problems
may be hard to
automate for large
enterprises
mapping MAC
address to
location?
mapping IP
address to
switch port?
• indoor
coverage
• acquisition
time
• fails for
mobile
devices
• unreliable for
nomadic
Use
Ethernet LANs
Enterprise
LANs
Some ISPs
DSL, cable
mobile devices
fall back
January 2008
UCSB
39
SIP message for Location Info.
INVITE urn:service:sos SIP/2.0
To: urn:service:sos
Call-ID: [email protected]
Via: SIP/2.0/TCP 192.168.1.106:4064;rport
Content-Type: multipart/mixed; boundary
From: sip:[email protected]
Contact: <sip:[email protected]:5060>
CSeq: 1 INVITE
Content-Length: 1379
request line
------- =_ZGY1NTFlZDJkMDkxY2FkMTIxMWI2MzIzNjE1M2U0OTY=
MIME-Version: 1.0
Content-Type: application/pidf+xml
Content-Transfer-Encoding: 8bit
<?xml version="1.0" encoding="ISO-8859-1"?>
<presence xmlns="urn:ietf:params:xml:ns:pidf"
xmlns:gp="urn:ietf:params:xml:ns:pidf:geopriv10"
xmlns:cl=" urn:ietf:params:xml:ns:pidf:geopriv10:civilLoc"
xmlns:gml="urn:opengis:specification:gml:schema-xsd:feature:v3.0"
entity="sip:[email protected]">
<tuple id="28185">
<status>
<gp:geopriv>
<gp:location-info>
<cl:civilAddress>
<cl:country>us</cl:country>
<cl:A1>ny</cl:A1>
<cl:A2>new york</cl:A2>
<cl:A3>new york</cl:A3>
<cl:A6>amsterdam</cl:A6>
<cl:HNO>1214</cl:HNO>
</cl:civilAddress>
</gp:location-info>
<gp:method>Manual</gp:method>
</gp:geopriv>
</status>
<contact priority="0.8">sip:[email protected]:5060</contact>
<timestamp>2005-09-26T15:57:34-04:00</timestamp>
</tuple>
</presence>
------- =_ZGY1NTFlZDJkMDkxY2FkMTIxMWI2MzIzNjE1M2U0OTY=--
header fields
------ =_ZGY1NTFlZDJkMDkxY2FkMTIxMWI2MzIzNjE1M2U0OTY=
MIME-Version: 1.0
content-Type: application/sdp
Content-Transfer-Encoding: 8bit
v=0
o=eddie 1127764654 1127764654 IN IP4 192.168.1.106
s=SIPC Call
c=IN IP4 160.39.54.70
t=0 0
m=audio 10000 RTP/AVP 0 3
m=video 20000 RTP 31
SDP
January 2008
UCSB
PIDF-LO40
January 2008
UCSB
41
Tracking
January 2008
UCSB
42
Data formats: location
• Civic (street)
– jurisdictional & postal
• Geo (longitude & latitude)
– point, polygon, circle, …
• see GeoRSS for simple example
January 2008
UCSB
43
ECRIT: LoST Functionality
•
Civic as well as geospatial queries
–
•
•
civic address validation
Recursive and iterative resolution
Fully distributed and hierarchical
deployment
–
–
•
Indicates errors in civic location data 
debugging
–
•
but provides best-effort resolution
Can be used for non-emergency services:
–
–
directory and information services
pizza delivery services, towing companies, …
can be split by any geographic or civic
boundary
same civic region can span multiple
LoST servers
January 2008
<findService xmlns="urn:…:lost1">
<location profile="basic-civic">
<civicAddress>
<country>Germany</country>
<A1>Bavaria</A1>
<A3>Munich</A3>
<A6>Neu Perlach</A6>
<HNO>96</HNO>
</civicAddress>
</location>
<service>urn:service:sos.police</service>
</findService>
UCSB
44
LoST: Location-to-URL Mapping
VSP1
cluster serving VSP1
replicate
root information
cluster
serves VSP2
123 Broad Ave
Leonia
Bergen County
NJ US
LoST
NJ
US
sip:[email protected]
root
NY
US
nodes
search
referral
Bergen County
NJ US
Leonia
NJ US
January 2008
UCSB
45
LoST Architecture
G
tree guide
G
G
G
T1: .us
broadcast (gossip)
T2: .de
G
resolver
seeker
313 Westview
Leonia, NJ US
T2
T1
(.us)
January 2008
(.de)
T3
(.dk)
Leonia, NJ  sip:[email protected]
UCSB
46
Left to do: event notification
•
notify (small) group of users when something of interest happens
–
–
–
–
–
•
presence = change of communications state
email, voicemail alerts
environmental conditions
vehicle status
emergency alerts
kludges
– HTTP with pending response
– inverse HTTP --> doesn’t work with NATs
•
•
Lots of research (e.g., SIENA)
IETF efforts starting
– SIP-based
– XMPP
January 2008
UCSB
47
Overview
• The original vision of ubiquitous
computing
• User challenges
• Beyond terminal mobility
• Location as new core service
• Universality: 7DS
• CS technology into reality
January 2008
UCSB
48
Problems with Wide Area Wireless
• 802.11 currently hard to deploy across city or large area
• Problem: How can mobile devices / gadgets get information
while on the move?
• Use local peer-to-peer wireless networks to exchange
information
– Peers can get information they do not have from another peer
Solution: 7DS!
January 2008
UCSB
49
How 7DS Works
1. When devices are in the same BSS (Basic service set) of
802.11 ad-hoc network, they discover each other using
service discovery of Zeroconf
zeroconf
January 2008
UCSB
50
How 7DS Works
2. If there is no Internet connection, the devices can communicate
with each other to exchange information
January 2008
Internet
UCSB
51
Web Delivery Model
• 7DS core functionality: Emulation of web content access
and e-mail delivery
January 2008
UCSB
52
Design
• Peer-to-peer network set up using zeroconf
– Protocol enables devices to communicate with each other
without a DHCP server, a DNS server and a Directory server
•
•
•
•
January 2008
Proxy server serves content
Search engine searches for local data
MTA store and forward
In progress: File synchronization, BBS
UCSB
53
Store and Forward
• Forwarding e-mail in the ad-hoc network
• Acts as an MTA
January 2008
UCSB
54
Search Engine
• Provides ability to query self
for results
• Searches the cache index
using Swish-e library
• Presents results in any of three
formats: HTML, XML and plain
text
• Similar in concept to Google
Desktop
January 2008
UCSB
55
Query Multicast Engine
•
•
•
•
Used to actually exchange
information among peers
Requesting peer broadcasts a
query to the network
Responding peers reply if they
have information
– Send encoded string with list
of matching items
Requesting peer retrieves suitable
information
January 2008
UCSB
56
File Synchronization
SERVICE
ANNOUNCEMENT
FILE SYNCHRONIZATION
RESOLUTION USING RSYNC PROTOCOL
SRV : 7ds-fs1.filesync._7ds._udp.local.
“I want
7ds-device1.local:2525
Word.doc and presentation.ppt”
TXT : file1.xml
TXT : file2.xml
SRV : 7ds-fs2.filesync._7ds._udp.local.
“I want
7ds-device2.local:2525
File1.xml
and file2.xml”
TXT : word.doc
File1.xml
File2.xml
TXT : presentation.ppt
Word.doc
Presentation.ppt
File1.xml
File2.xml
Word.doc
Presentation.ppt
January 2008
UCSB
57
Overview
• The original vision of ubiquitous
computing
• User challenges
• Beyond terminal mobility
• Location as new core service
• Universality: 7DS
• CS technology into reality
January 2008
UCSB
58
CS research to reality
CS as science
CS as engineering
January 2008
UCSB
59
CS tech transfer, mode 1
January 2008
UCSB
60
CS tech transfer, mode 2
January 2008
UCSB
61
The standards route
January 2008
UCSB
62
Role of standards
•
Widespread use of technology requires standards
– railroad gauges, nuts & bolts, Morse code, phone system
• industrial age = interchangeable parts
–
–
–
–
•
CD ROM, DVD, HD-DVD/BlueRay
JPEG, MPEG, ...
SQL
C, Java
Particularly for network technology
– many mid-size actors
– “network effect” = utility increases with number of users
• e.g., cars have inverse network effects
• chess game vs. email and word processor
– DECnet, SNA, ARCnet  Ethernet, IP
– “hypermedia”  HTML, HTTP
January 2008
UCSB
63
Standards
• Tanenbaum: “The nice thing about standards is that you
have so many to choose from.”
component technologies
VHS + Beta
Ethernet + Tokenring
ATM + IP
!
January 2008
UCSB
64
Standards: technology translator
• Similar in some ways to text books
• “accepted technology”
– lower/known risks (“vetted”)
– infrastructure (“eco system”)
• requires expertise and broader training
– many CS standards don’t have either
• example: HTTP/1.0, HTML 1.0
• thus, way to provide technology leadership
January 2008
UCSB
65
Examples of standards needed
• each with multi-B$ impact:
– medical records
– energy control
– transportation
• inter-vehicle safety systems, traffic alerts, ...
– academic records
– business transactions
• e.g., your travel expense report
• requires both CS and domain expertise
January 2008
UCSB
66
Conclusion
• Motivate mobile ubiquitous research by user problems
• From stovepipe mobile & wireless systems to personal
and shareable wireless networks
• Thinking beyond single applications
– presence event notification
– 9-1-1 location  time & location as infrastructure
• Need new models of creating services
– domain-specific languages, not Java APIs
January 2008
UCSB
67