Transcript UbiCom Book Figures - Queen Mary University of London

UbiCom Book Slides
Chapter 6
Tagging, Sensing & Controlling
Stefan Poslad
http://www.eecs.qmul.ac.uk/people/stefan/ubicom
Ubiquitous computing: smart devices,
environments and interaction
1
Overview
•
•
•
•
•
•
•
•
Introduction 
Tagging the Physical World
Sensors and Sensor Networks
Micro Sensing & MEMS
Micro Actuation & MEMS
Embedded Systems and Real-time Systems
Control Systems (For Physical World Tasks)
Robots
Ubiquitous computing: smart devices, environments and interaction
2
Chapter 6: Overview
The slides for this chapter are also expanded and split into
several parts in the full pack
• Part A: Tagging physical world & augmented reality
• Part B: Sensors, Sensor Nets
• Part C: MEMS
• Part D: Embedded Systems
• Part E: Control Systems & Robots
Ubiquitous computing: smart devices, environments and interaction
3
Overview
Chapter 6 focuses on:
• internal system properties: context-awareness & autonomy
• external interaction with the physical environment.
Ubiquitous computing: smart devices, environments and interaction
4
Introduction
To enable Smart (Physical) Environments, devices should:
• Spread more into the physical environment, becoming part
of more user activities in physical environment
• Be cheap to operate: autonomous energy etc
• Be low maintenance: automatic
• Be able to interact with physical environment context
• Be sometimes small enough so as to …
• Be able to be encapsulated and embedded
• Be cheap to manufacture
Ubiquitous computing: smart devices, environments and interaction
5
UbiCom Internal System Properties
Physical Environments
Physical
Phenomena
CPI (Sense,
CPI
Adapt)
implicit HCI
Autonomous
ContextAware
Distributed
Intelligent
UbiComp
System
ICT
Ubiquitous computing: smart devices, environments and interaction
6
Smart Physical Environments
Nanotechnology
Integrated
Circuits
Robots
Embedded RT
Nanobots
MEMS
Micro sensors
actuators &
Operating
systems
Process Control
Macro
Control
Systems
Sensor
Nets
Virtual
Tags
RFID
Augmented
Reality
Locators
Annotation
Smart Physical Environments
Ubiquitous computing: smart devices, environments and interaction
7
Smart (Physical) Environments
Smart Devices
CPI
Context-aware systems
Physical Environment Devices
Natural Physical
Environment
OS
Contexts
Types
Dimensions
ASOS
Controllers
Tags
MTOS
Sensors
macro
micro
Actuators
Augmented
Reality
RTOS
Sensor Nets
Nano
Types
Physical
Virtual
Data Mgt
MEMS
Skins
Programmable
Paint
Matter
RFID
Link
PID
Dust
Robot
Active
Site, Anchor Etc.
Passive
Adaptive Arms
Wheels
Mobile
Nanobot
Legs
Ubiquitous computing: smart devices, environments and interaction
8
Overview
•
•
•
•
•
•
•
Introduction
Tagging the Physical World 
Sensors and Sensor Networks
Micro Actuation and Sensing: MEMS
Embedded Systems and Real-time Systems
Control Systems (For Physical World Tasks)
Robots
Ubiquitous computing: smart devices, environments and interaction
9
Tagging (or Annotating) the Physical
World
Outline of this section
• Applications
• Life-cycle for Tagging Physical Objects
• Tags: Types and Characteristics
• Physical and Virtual Tag Management
• RFID Tags
• Personalised and Social Tags
Ubiquitous computing: smart devices, environments and interaction
10
Tagging: Applications
• Locate items, e.g.?
• Retrieve annotations associated with physical objects
(augmented reality) e.g. ?
• Security, e/g/. .
• Tracking, e.g.,
• Automated Routing: of physical objects, e.g., ?
• Automated Physical Access: e.g., ?
Ubiquitous computing: smart devices, environments and interaction
11
Tagging Applications: Automated
Physical Access
Ubiquitous computing: smart devices, environments and interaction
12
Tagging Applications: Asset
Tracking
Ubiquitous computing: smart devices, environments and interaction
13
Tagging Applications: Security
Ubiquitous computing: smart devices, environments and interaction
14
Physical versus Virtual Tags
Virtual View of
physical objects,
e.g., digital Photo
Physical Tag,
e.g., RFID
Stefan’s car
Virtual Tag
Physical Object
Ubiquitous computing: smart devices, environments and interaction
15
Life-cycle for Tagging Physical
Objects
Managing :
Accessing :
Presenting
Capturing:
Anchoring :
.
Organising:
Ubiquitous computing: smart devices, environments and interaction
16
Design issues for Anchoring
Tags on Physical Objects
Different ways to characterise and classify tagging
• By how to augment physical world objects for use in virtual
(computer) environments
• By use of Onsite versus Offsite and attached versus
detached classification of tags
Ubiquitous computing: smart devices, environments and interaction
17
Augment physical environments for
use in virtual environments
• Augment the user:
• Augment the physical object:
• Augment the surrounding environment:
Ubiquitous computing: smart devices, environments and interaction
18
Onsite versus Offsite & Attached
versus Detached Annotation
2 dimensions:
• User of the annotation is
– onsite (co-located or local) with physical object versus
– offsite (not co-located or remote).
• Annotation is
– attached (or augments) physical object it refers to
versus
– being detached (not augmented or not collocated) with
the physical object.
Ubiquitous computing: smart devices, environments and interaction
19
Onsite versus Offsite & Attached
versus Detached Annotation
Attached
Detached
Offsite
Onsite
Ubiquitous computing: smart devices, environments and interaction
20
Design issues
for Anchoring
Tags on Physical
Objects
Physical Environment Objects
Static States
Dynamic States
Tags
Sensors
Digital
Analogue
Physical
Virtual (annotation)
AR
Physical-Virtual Tag Link
Augment
User
RFID
Onsite versus
off-site
Augment Physical
Object
Cardinality
Augment Physical
Environment
Static vs.
Dynamic
Attached versus
detached
Ubiquitous computing: smart devices, environments and interaction
Design issues for Tagging Physical
environment
• Tags read outdoors in noisy, wet, dark or bright
environments.
• Annotation data storage, distribution & integration with data
• Data management must start as soon as the data is
captured (readers).
• Multiple tags & readers per unit Vol..
– Challenges?
• Redundant annotations: similar items are captured, many
times over.
– Solutions?
• Applications and businesses need to define the level of
aggregation, reporting, analysis
Ubiquitous computing: smart devices, environments and interaction
22
RFID Tags
• A type of on-site tag, attached to physical object
• RFID (Radio Frequency Identifier) Tags, attached to objects
to enable identification of objects in the world over a
wireless link.
RFID Tags versus Bar codes?
Ubiquitous computing: smart devices, environments and interaction
23
RFID Tags: Applications
• ???
Ubiquitous computing: smart devices, environments and interaction
24
Types of RFID Tag
• RFID tags may be classified into whether or not they:
– Active:
– Passive:.
• Active tags are more expensive and require more
maintenance but have a longer range compared to passive
tags.
• Typical RFID system main components:
• tag itself, reader, data storage, post-processing
• RFID tag versus RF Smart Card?
Ubiquitous computing: smart devices, environments and interaction
25
Active RFID Tags
•
•
•
•
•
•
Active RFID tags used on large, more expensive assets
.
Typically operate at 0.455, 2.45 or 5.8 GHz frequencies
Have a read range of 20 M to 100 M,
Cost?
Complex active tags could also incorporate sensors.
How? Why?
• 2 types of active tags: transponders and beacons
Ubiquitous computing: smart devices, environments and interaction
26
Active RFID Transponders
• Active transponders are woken up when they receive a
signal from a reader.
• Transponders conserve battery life. How?
• Important application of active transponders is in toll
payment collection, checkpoint control and other systems.
Ubiquitous computing: smart devices, environments and interaction
27
Active RFID Beacons
• Main difference c.f. Transponder is long range, global?
beacon reader
• Beacons are used in Real-Time Location Systems (RTLS)
• Longer range RTLS could utilise GPS or mobile phone
GSM trilateration
– See Chapter 7
•
In RTLS, a beacon emits a signal with its unique identifier
at pre-set intervals
–
Ubiquitous computing: smart devices, environments and interaction
28
Active RFID Transponder Application:
toll booths
Ubiquitous computing: smart devices, environments and interaction
29
Passive RFID Tags
•
•
•
•
•
Contain no power source and no active transmitter
Power to transmit comes from where?
Cheaper than active tags, cost?
Shorter (read access) range than active tags, typically ??
Passive RFID transponder consists of a microchip attached
to an antenna, e.g., same as smart card
• Lower maintenance
• Passive Transponders can be packaged in many different
ways,
– ????
Ubiquitous computing: smart devices, environments and interaction
30
Passive RFID Tags
• Passive tags typically operate at lower frequencies than
active tags
–
• Low-frequency tags are ideal for applications where the tag
needs to be read through certain soft materials and water
at a close range. Why?
Ubiquitous computing: smart devices, environments and interaction
31
Passive Tags: Near Field
• 2.different approaches to transfer power from the reader
to passive tags: near field and far field
Near field
• Passive RFID interaction based upon electromagnetic
induction.
• Explain how this works here
Ubiquitous computing: smart devices, environments and interaction
32
Passive Tags: Far Field
• Why can’t electromagnetic induction be used?
• So how does far field RFID interaction work?
Ubiquitous computing: smart devices, environments and interaction
33
Business Use of Annotation
• Physical artefact annotation is often driven by business
goals.
• Uniquely identify objects from manufacture during
business processes
Ubiquitous computing: smart devices, environments and interaction
34
Personal use of Annotation
• Tags are less specific, deterministic, multi-modal (using
multiple sensory channels) using multimedia.
• Subjective annotations are used in multiple contexts,
multiple applications and multiple activities by users.
• Semantic gap challenge: between the low-level object
features extracted and their high-level meaning with
respect to a context of use
• Several projects to tag personal views of physical world
–
–
–
–
MyLifeBits
Semacode
Google Earth? But Is it personalised?
etc
Ubiquitous computing: smart devices, environments and interaction
35
Personal use of Annotation:
Semacode
• Semacode (2005) propose a scheme to define labels that
can be automatically processed from captured images and
linked to a Web-based spatial information encyclopaedia.
• How does a semacode encodes URLs??
• How to create a semacodes?
• How do read a Semacaode
• Some management may be needed to control malicious
removal, movement and attachment.
Ubiquitous computing: smart devices, environments and interaction
36
Semacode Use
Attach to
physical
world
Convert URL
to visual
code
Photograph
(Read Code)
get
Web Page
post
Phone
Ubiquitous computing: smart devices, environments and interaction
37
Overview
•
•
•
•
•
•
•
Introduction
Tagging the Physical World
Sensors and Sensor Networks 
Micro Actuation and Sensing: MEMS
Embedded Systems and Real-time Systems
Control Systems (For Physical World Tasks)
Robots
Ubiquitous computing: smart devices, environments and interaction
38
Sensors: introduction
• Sensors are transducers that convert some physical
phenomenon into an electrical signal
• Wireless sensors:
• Sensors can be networked – sensor nets
Ubiquitous computing: smart devices, environments and interaction
39
Sensor Applications
Give some examples of sensor use
• Cars
• Computers
• Retail, logistics:
• Household tasks
• Buildings
• Environment monitoring
• Industrial sensing & diagnostics
Ubiquitous computing: smart devices, environments and interaction
40
Sensors Types
Sensors can be characterised according to:
• Passive (tags) vs. active
• Single sensors vs sensor arrays vs sensor nets
• Read-only program vs. re-programmable
Ubiquitous computing: smart devices, environments and interaction
41
Sensors versus Tags
• ???
Ubiquitous computing: smart devices, environments and interaction
42
Distribution field of
phenomena that
can be detected
measured
Physical
Phenomena
S
S
Sensor
net
S
S
S
Storage
S
S
S
Sensor net
Sensor
net
S
S
S
S
S
Sensors that
detect event
User
S
Access
Node
Sensors that
notify access
node
Internet
Ubiquitous computing: smart devices, environments and interaction
43
Sensor Nets
• Main components of a typical sensor network system are
networked sensors nodes serviced by sensor access node.
• Slightly different but compatible view of a sensor network is
to view sensors as being of three types of node):
– common nodes
– sink nodes
– gateway (access)
• In scenario given earlier, some sensors in the network can
act as sink nodes within the network in addition to the
access node.
• Concepts of sensor node & sensor net can be ambiguous:
– A sensor can act as a node in a network of sensors versus there is
a special sensor network server often called a sensor (access) node
Ubiquitous computing: smart devices, environments and interaction
44
Sensor Net: Functions
• The main functions of sensor networks can be layered in a
protocol stack according to:
– physical network characteristics,
– data network characteristics
– data processing and sensor choreography
• Use small network protocol stack for sensor nets. Why?
• Other conceptual protocol layered stacks could also be
used instead to model sensor operation,
Ubiquitous computing: smart devices, environments and interaction
45
Sensor Net: Functions
Data processing
Collaborative
processing
Data
storage
Internetwork
Sensor to Network
Sensor Electronics
Event definition
& processing
In-situ
processing
Routing Intra vs.
inter node
Sensor
distribution
& density
DSP
Data
discovery
Data
uncertainty
Addressing
RF , Optical
transmission
characteristics
Physical
environment
characteristics
Power
management
Ubiquitous computing: smart devices, environments and interaction
46
Sensors: Electronics
Processing
Storage
Sensor
Transducer
Analogue
Filter
Amplifier
ADC
DSP
Transceiver
Modulator
Power
Power
management
Battery
Antenna
Transmitter
Switch
Demodulator
Receiver
Sensor Net Design: Signal Detection &
Processing
Positioning & coverage of networks is important. Why?
Ubiquitous computing: smart devices, environments and interaction
48
Sensor Net Design: Positioning &
Coverage
• Given: sensor field (either known sensor locations, or
spatial density)
– Where to add new nodes for max coverage?
– How to move existing nodes for max coverage?
• Can Control
– Area coverage:
– Detectability:
– Node coverage:
Ubiquitous computing: smart devices, environments and interaction
49
Sensor Net Design: Improved SNR
Through Using Denser Sensor Nets
• Sensor has finite range determined by base-line (floor)
noise level
• Denser sensor field improves detection of signal source
within range. How?
Ubiquitous computing: smart devices, environments and interaction
50
Overview
• Overview: Sensor Net Components & Processes
• Physical Network: Environment, Density &
Transmission
• Data Network: Addressing and Routing 
• Data Processing: Distributed Data Storage & Data
Queries
Ubiquitous computing: smart devices, environments and interaction
51
Senor Net Design: Sensor Data Routing
•
•
•
•
•
Networking sensors versus networking computers?
Sensors form P2P network with a mesh topology network
Sensors are massively distributed and work in real-time
No universal routing protocols or central registry.
Each node acts a router and application host.
Ubiquitous computing: smart devices, environments and interaction
52
Sensor Routing
• Make sensor address resolution efficient
• Data centric routing,
– Directed Diffusion
– Flooding
– Gossiping
• Routing classification
– Network structure: flat, hierarchical, hybrid
– By interaction protocol
Ubiquitous computing: smart devices, environments and interaction
53
Sensor Networks vs. Ad Hoc
Networks
???
Ubiquitous computing: smart devices, environments and interaction
54
Sensor Net Topologies
• ??
Ubiquitous computing: smart devices, environments and interaction
55
Senor Net Design: In-Network
Processing
• Why perform In-Network Processing?
Sensor
Node
Sensors
Ubiquitous computing: smart devices, environments and interaction
56
Sensor Net: Data Storage & Retrieval
• What designs/ architectures can we use for sensor net data
storage an retrieval?
Ubiquitous computing: smart devices, environments and interaction
57
Sensor Database System
•
Characteristics of a
Sensor Network:
• Can existing database
techniques be reused?
Ubiquitous computing: smart devices,
environments and interaction
58
Sensor Net: Technologies, Kits &
Standards
•
•
•
•
Sun Spot: Java
Berkeley Motes: TinyOS, C
SPINE (Signal Processing in Node Environment)
OGC Standards: SensorML etc
Ubiquitous computing: smart devices, environments and interaction
59
Overview
•
•
•
•
•
•
•
Introduction
Tagging the Physical World
Sensors and Sensor Networks
Micro Actuation and Sensing: MEMS 
Embedded Systems and Real-time Systems
Control Systems (For Physical World Tasks)
Robots
Ubiquitous computing: smart devices, environments and interaction
60
Micro Actuation and Sensing: MEMS
•
•
•
•
•
Fabrication
Micro-Actuators
Micro-Sensors
Smart Surfaces, Skin, Paint, Matter and Dust
Downsizing to Nanotechnology and Quantum Devices
Ubiquitous computing: smart devices, environments and interaction
61
Trend: Miniaturisation
• Electronic components become smaller, faster, cheaper to
fabricate, lower power & lower maintenance, they can be
more easily deployed on a massive and pervasive scale.
• MicroElectro Mechanical Systems (MEMS) are based upon
IC Chip design
• Possibilities for miniaturization extend into all aspects of
life, & potential for embedding computing & comms
technology quite literally everywhere is becoming a reality.
• IT as an invisible component in everyone's surroundings
• Extending the Internet deep into the physical environment
Ubiquitous computing: smart devices, environments and interaction
62
Trend:  IC Transistor Density
• Gordon Moore (1965), Intel co-founder made a prediction,
now popularly known as Moore's Law, which states that the
number of transistors on an IC chip doubles ~ every 2 y
• Does it mean that software processing capability will also
increases in this way?
•  IC Chip density =  Software Performance?
Ubiquitous computing: smart devices, environments and interaction
63
MEMS: Introduction
• MEMS (Micro-electromechanical systems): micron- to
millimetre-scale electronic devices fabricated as discrete
devices or in large arrays
• MEMS perform 2 basic types of functions: sensors or
actuators.
• Both act as transducers converting one signal into
another.
• MEMS actuators: electrical signal -> physical phenomena
to move or control mechanisms.
• MEMS Sensors work in reverse to actuators
Ubiquitous computing: smart devices, environments and interaction
64
MEMS Examples
Actuator
Gyroscope
Hinge
Electrostatic motor
Ubiquitous computing: smart devices, environments and interaction
65
MEMS: Fabrication
• MEMS comprising mechanical and discrete electronic
components
• MEMS design is different from macro devices
• MEMS design are based upon IC chips design
• Silicon based materials have:
– Well understood electrical properties
– Good mechanical properties
Ubiquitous computing: smart devices, environments and interaction
66
MEMS: Fabrication
• Design a new circuit = design of interconnections among
millions of relatively simple and identical components.
• Diversity and complexity of the interconnections -> diversity
of electronic components including memory chips and
CPUs.
• Multiplicity, batch fabrication, is inherent.
• Miniaturisation of IC based MEMS processing has
important advantages over macro electromechanical
devices and systems?
Ubiquitous computing: smart devices, environments and interaction
67
MEMS : Fabrication
• Micromachines are fabricated just like ICs.
• MEMS type ICs can be fabricated in different ways using:
– Bulk micro-machining
– Surface micro-machining
– LIGA deep structures.
Ubiquitous computing: smart devices, environments and interaction
68
Micro-Actuator
• Mechanisms involved in micro-actuation whilst conceptually
similar to equivalent macro mechanisms may function
fundamentally differently,
• Are engineered in a fundamentally different way using IC
Ubiquitous computing: smart devices, environments and interaction
69
Micro-actuator: Applications
•
•
•
•
•
Micro-mirrors, e.g., ??
Micro-fluid pumps, e.g., ??
Miniature RF transceivers, e.g., ??
Miniature Storage devices, e.g., ??
Etc
Ubiquitous computing: smart devices, environments and interaction
70
Micro-sensors
• Sensors are a type of transducer
• Microsensors can work quite differently from equivalent
macro sensor,
• Sensors enable adaptation
• Often embedded into system as part of a control loop
Ubiquitous computing: smart devices, environments and interaction
71
MEMS: Applications
• Micro-accelerometers,
– E.g., ??
• Micro-gyroscopes
– E.g.,
• Detecting Structural Changes
– E.g.,
Ubiquitous computing: smart devices, environments and interaction
72
Smart Device Form Factors: Smart
Dust, Skins & Clay
• 3 forms proposed by Weiser (1 tabs, 2 pads & 3 boards)
can be extended to include 3 more forms:
4. Smart Dust:
5. Smart Skins:
6. Smart Clay:
Ubiquitous computing: smart devices, environments and interaction
73
Smart Dust: MEMS
• MEMS can be sprayed into physical environment
• E.g., Smart Dust project (Pister, UC,Berkely)
• (see Chapter 2)
Ubiquitous computing: smart devices, environments and interaction
74
Smart Skins: MEMS
• MEMS can be permanently attached to some fixed
substrate forming
– smart surfaces
– smart skin
• E.g. Paint that is able to sense vibrations
• See also Organic Displays (Chapter 5)
Ubiquitous computing: smart devices, environments and interaction
75
Smart Clay: MEMS
•
•
•
•
•
Claytronics project
Can behave as malleable programmable matter
Are MEMS ensembles
Self-assembled into any arbitrary 3D shape
Goal to achieve a synthetic reality.
Ubiquitous computing: smart devices, environments and interaction
76
MEMS: Challenges
• Establishing ownership of all of these micro items.
• Coping with data overload
• Different Low-level patterns of signals may be ambiguous
and variable.
• Handling context switches between these augmented
environment events via assisted senses and the unassisted
ones.
• Are micro-devices either easy to dispose of or hard to
dispose of?
– What is we swallow / breath them in?
• How to manage MEMS?
– See Chapter 12
Ubiquitous computing: smart devices, environments and interaction
77
Nanocomputing
• Nanocomputing can be defined as the manipulation,
precision placement, measurement, modelling, and
manufacture to create systems with less than 100 nm
• Also referred to as nanotechnology
• Is based upon a broader range of materials, mechanisms &
sizes down to molecular level
• MEMS Vs. Nanocomputing?
Ubiquitous computing: smart devices, environments and interaction
78
Nanocomputing
• The drive to switch transistors faster and to be lowpowered has been to make them smaller.
• When electronic components approach nanometer sizes,
odd things begin to happen. What?
• This raised an early concern about the feasibility of
nanotechnology.
Other challenges are:
• thermal noise
• positioning and the control of structures at this level
Ubiquitous computing: smart devices, environments and interaction
79
Nanocomputing
• Nanotechnology at first proposed to use a bottom-up
approach to design, to be able to assemble custom-made
molecular structures for specific applications,
• A major challenge to this design process is the complexity
and novelty in understanding and being able to model
materials at this level.
• More research is needed to understand how combinations
of materials, in particular compounds, gives materials at the
molecular level certain physical and functional properties..
Ubiquitous computing: smart devices, environments and interaction
80
Overview
•
•
•
•
•
•
•
Introduction
Tagging the Physical World
Sensors and Sensor Networks
Micro Actuation and Sensing: MEMS
Embedded Systems and Real-time Systems 
Control Systems (For Physical World Tasks)
Robots
Ubiquitous computing: smart devices, environments and interaction
81
Embedded Systems: Introduction
•
•
•
•
Is a component in a larger system
Is programmable
Performs a single, dedicated task.
May or may not be visible as a computer to a user of that
system
• May or may not have a visible control interface
• E.g., ???
• May be local or remote,
– e.g., ??
• fixed or mobile
– e.g??
Ubiquitous computing: smart devices, environments and interaction
82
Embedded System Characteristics
(Embedded vs. MTOS Systems)
Traditionally, embedded systems differ from MTOS systems
OS of Embedded systems differ vs. MTOS system
1. Specialised to single task enactment (ASOS)
2. Actions on physical world tasks are often scheduled with
respect to real-time constraints (RTOS)
3. Safety-criticality is considered more important
Ubiquitous computing: smart devices, environments and interaction
83
Embedded vs. MTOS Systems
• Often have constraints concerning power consumption
• Often are designed to operate over a wide-range of
physical environmental conditions compared to PC
– e.g.,
• Often operate under moderate to severe real-time
constraints.
• System failures can have life-threatening consequences.
– E.g.,
Ubiquitous computing: smart devices, environments and interaction
84
Embedded vs. MTOS Systems
• Each embedded computing devices may be designed for
its own rigidly defined operational bounds
– e.g.,
•
•
•
•
•
•
Linking embedded systems to external systems
Designs often engineered for a trade-off
Fewer system resources then PC. How?
Embedded systems not always easy to programme. Why?
Most embedded designs (hardware & software) are unique
Use a far simpler & cheaper OS & hardware. Why?
Ubiquitous computing: smart devices, environments and interaction
85
Embedded Systems: Hardware
• Microprocessors
• Microcontroller
• FPGA (Field Programmable Gate Arrays):
Ubiquitous computing: smart devices, environments and interaction
86
Real-Time System (RTS)
• Real-time systems (RTS) can be considered to be
resource-constrained
• Often RTS perform safety-critical tasks
• RTS reacts to external events that interrupt it:
• RTS uses mechanisms for priority scheduling of interrupts
• RTOS may also use additional process control:
– .
Ubiquitous computing: smart devices, environments and interaction
87
RTS Design Concerns
• There are a range of real-time design concerns to support
critical response time of a task:
–
• Need to optimise
– both response time and data transfer rate
– optimising these when there are simultaneous tasks.
• Key factors that affect the response time are?
– process context-switching
– interrupt latency
Ubiquitous computing: smart devices, environments and interaction
88
RTS: Hard vs. Soft
• Timeliness is single most important aspect of RT system.
• RTS system is one where timing of result is just as
important as the result itself.
• A correct answer produced too late is just as bad as an
incorrect answer or no answer at all.
• RTS correctness of computations not only depends upon
the logical correctness of the computation but also upon
time to produce results.
• If the timing constraints are not met, system failure occurs
• Timing constraints can vary between different real-time
systems.
• Therefore, RTS can fall into one of three categories: soft,
hard or firm..
Ubiquitous computing: smart devices, environments and interaction
89
RTS: Soft
• Single computation arriving late may not be significant to
the operation of the system,
–
• Although many late arrivals might be significant
• Timing requirements can be defined by using an average
response time.
Ubiquitous computing: smart devices, environments and interaction
90
RTOS: Hard
•
•
•
•
Timing requirements are vital.
Response that’s late is incorrect and system failure results.
Activities must complete by specified deadline, always.
Different types of deadlines. What?
• If a deadline is missed the task fails
– E.g., ??
• This demands that the system has the ability to predict how
long computations will take in advance.
Ubiquitous computing: smart devices, environments and interaction
91
Safety-Critical Systems
• Instructors could add some text here or delete this slide.
Ubiquitous computing: smart devices, environments and interaction
92
Overview
•
•
•
•
•
•
•
Introduction
Tagging the Physical World
Sensors and Sensor Networks
Micro Actuation and Sensing: MEMS
Embedded Systems and Real-time Systems
Control Systems (For Physical World Tasks) 
Robots
Ubiquitous computing: smart devices, environments and interaction
93
Links to other Topics
• Control systems / robots can be simple, operate in static
deterministic environments.
• To operate in more dynamic non- deterministic
environments, they can make use of AI techniques
(Chapters 8-10).
• HCI aspects of (biologically inspired) robots such as
affective computing etc (Chapter 5)
Ubiquitous computing: smart devices, environments and interaction
94
Control Systems (For Physical World
Tasks)
• Simply type of control
– Activated only when defined thresholds are crossed,
– e.g., .
• Disadvantages?
–
• Solutions?
–
Ubiquitous computing: smart devices, environments and interaction
95
Control Systems: Feedback
Control
• 2 basic kinds of feedback:
– negative
– positive
Negative feedback
• Seeks to reduce some change in a system output or state
• Based upon derivative of output
• Which is then used to modify input to regulate output.
• Several types of feedback control: D, P, I, PID
Positive feedback
• Acts to amplify a system state or output
Ubiquitous computing: smart devices, environments and interaction
96
Control Systems: Derivative (D)
Feedback Control
Reference
Value r(t)
∑
-
Control System
+
Controller
Error
e(t)=r(t)–f(t)
f(t)
DAC &
Drive
Input
i(t)
ADC
Plant
Output
o(t)
Transducer
Feedback
Ubiquitous computing: smart devices, environments and interaction
97
Control Systems: Proportional (P)
Feedback Control
• In simple proportional (-ve feedback) control system
• Action taken to negatively feedback a signal to the plant,
• Is in proportion to the degree the system diverges from the
reference value
• This leads to a much smoother regulation
– e.g.,.
P Controller
e(t)
Proportional
g.e(t)
Ubiquitous computing: smart devices, environments and interaction
98
Control Systems: PID Controllers
• Sometimes P type controller output is not regulated
correctly
– e.g., ??
• To solve this problem either integral or differential control
or both can be added to the control.
• PID controller is so named because it combines
Proportional, Integral and Derivative type control
• Proportional (P) controller is just the error signal
multiplied by a constant and fed out to a hardware drive.
Ubiquitous computing: smart devices, environments and interaction
99
Control Systems: PID Controllers
• Integral (I) controller deals with past behaviour of
control.
–
• Derivative (D) type controller is used to predict the plant
behaviour
• P, PI, PD or PID control are often simple enough, to be
hard-coded into controllers
• Usually support some adjustment controls,
– e.g.,
• PID controllers can be designed to be programmable
Ubiquitous computing: smart devices, environments and interaction
100
PID Controllers
PID Controller
Proportional
+
Integral
e(t)
f(t)
+
∑
Derivative
Ubiquitous computing: smart devices, environments and interaction
101
Programmable Controllers:
Microcontrollers
• Hardware architecture of microcontrollers is much simpler
than general purpose processor mother-boards in PCs?
• I/O control support can be simpler as there may not be any
video screen output or keyboard input.
• Micro-controllers can range in complexity
• Originally, programmed in assembly language, later in C
• Control programs often developed in an emulator on a PC
• More recent microcontrollers can be integrated with on-chip
debug circuitry accessed by an in-circuit emulator
Ubiquitous computing: smart devices, environments and interaction
102
Complex Control Systems
• PID control Useful for coarse-gained, static control
– E.g., palletising, coarse-controlled locomotion, etc
• PID control not suitable for ?
– fine-grained
– dynamic control
– uncertainties in control
Ubiquitous computing: smart devices, environments and interaction
103
Complex Control
• Several sources of uncertainty?
• Techniques for controlling uncertain systems?
Ubiquitous computing: smart devices, environments and interaction
104
Overview
•
•
•
•
•
•
•
Introduction
Tagging the Physical World
Sensors and Sensor Networks
Micro Actuation and Sensing: MEMS
Embedded Systems and Real-time Systems
Control Systems (For Physical World Tasks)
Robots 
Ubiquitous computing: smart devices, environments and interaction
105
Robots
• Early 1960s, robots started to be used to automate
industrial tasks particularly in manufacturing
Why Automate?
Ubiquitous computing: smart devices, environments and interaction
106
Main Robot Components
Robots consist of:
• End effectors or actuators:
• Locomotion:
• Drive:
• Controller
• Sensor
Ubiquitous computing: smart devices, environments and interaction
107
Robots: Localisation
• Localisation is used to determine a robot’s position in
relation to its physical environment.
• Localisation can be local or global.
• Local localisation is often simpler in which a robot corrects
its position in relation to its initial or other current reference
location.
• Global localisation is discussed more in context-aware
systems part.
Ubiquitous computing: smart devices, environments and interaction
108
Robots: Types
3 Main Types
• Robot manipulator or robot arm
• Mobile robots
• Biologically inspired robots
Ubiquitous computing: smart devices, environments and interaction
109
Robot Manipulators
• A manipulator consists of a linked chain of rigid bodies that
are linked in an open kinematic chain at joints.
• rigid body can have up to 6 Degrees Of Freedom (DOF) of
movement.
• This comprises 3 translational DOF
– ???
Ubiquitous computing: smart devices, environments and interaction
110
Robot Manipulators
• Also comprises 3 rotational DOF
– ???
• Joints are designed to restrict some DOF.
• Human operators may be in the control loop of robot
manipulators. Why?
Ubiquitous computing: smart devices, environments and interaction
111
Robot Manipulators: Design
•
•
•
•
Motion planning needed
Control algorithms?
Regulation of contact force
Manipulators need to cope with variations in components
and objects being manipulated. Solutions?
– Use adaptive AI techniques (Chapter 8)
– Put human in the control loop
Ubiquitous computing: smart devices, environments and interaction
112
Mobile Robots
• Mobile robots use various kinds of locomotion systems
– ?
• Simplest types of mobile robots to control
– ??
• In dynamic non-deterministic environments, control is more
complicated
–
• A more complex, well-known & highly successful use of
mobile robots was Mars Explorer Robots
–
Ubiquitous computing: smart devices, environments and interaction
113
Mobile Robots
• No. of DOF is often less compared to a robot manipulator.
–
Need ways to navigate obstacles?
• Simple approach: use collision detection
•
More complex approach: anticipate & avoid collisions
– Need environment models (AI, Chapter 8)
– Need to replan paths to reach goal destinations (AI, Chapter 8)
Ubiquitous computing: smart devices, environments and interaction
114
Biologically Inspired Robots: Legged
locomotion
• Biologically inspired robots are more complex type of robot
– Combines legged locomotion capabilities & manipulator
• 2 main focuses to these robots:
– Legged locomotion (in combination with manipulator)
– Human-Robot Interaction
Ubiquitous computing: smart devices, environments and interaction
115
Biologically Inspired Robots: Legged
locomotion
• The use of legs enables legged robots to travel over
irregular terrain
• Biped robots often have more DOF than either the mobile
robot or robot manipulator
• Particular design challenge for biped robots is stability
–
Ubiquitous computing: smart devices, environments and interaction
116
Biologically Inspired Robots: Human
Robot Interaction
• Human robot Interaction:
– a specialisation of HCI, see Chapter 5
• Robots can assist humans and extend sensing capabilities
of (less able?) humans – Posthuman model.
• Robots can fulfil social roles
– i.e., affective computing (Chapter 5)
– e.g., artificial pets
• Social guided learning
– Learning by imitation or by tutelage
• Use of more human oriented interface & interaction
– E.g., speech recognition
Ubiquitous computing: smart devices, environments and interaction
117
Nanobots
• Nanobots can be manufactured as MEMS or at molecular
level.
• Microscopic world is governed by the same physical laws
as the macroscopic world
• But relative importance of the physical laws change in how
it affects the mechanics and the electronics at this scale
Ubiquitous computing: smart devices, environments and interaction
118
Nanobots
• Nature in terms of micro-organisms can be harnessed in
order to provide a host body for nanobots to move about
– e.g.,
• Shrinking device size to these nano dimensions leads to
many interesting challenges:
Ubiquitous computing: smart devices, environments and interaction
119
Developing UbiCom Robot
Applications
• Industrial types of robots
• Low cost consumer type robots
• Robots toolkits that are programmable.
Ubiquitous computing: smart devices, environments and interaction
120
Light Sensor
Motor A
Ultrasonic Sensor
Motor B
Motor C
Ubiquitous computing: smart devices,
environments and interaction
121
Developing UbiCom Robot
Applications
• Task: robot manipulates a Rubik’s Cube to its solved state
• Goal: robot performs whole task or guides humans to do it
Design involves
• Design: of the robot mechanics
–
• Design: how and when the robot senses state of the world
– e.g. ,
• Planning algorithm: to link individual actions
• Overall architecture: to integrate different sub-tasks
– e.g.,
Ubiquitous computing: smart devices, environments and interaction
122
Developing UbiCom Robot
Applications
Several practical issues for physical robots tasks execution
• Sensor accuracy
• Position accuracy
• Variable amounts of friction during movement
• Some elasticity in the robot arm
•
•
•
•
Low-level design to tell robots to carry out specific tasks
Tasks need to be designed to fit the robots capabilities
In open physical world, much non-determinism to handle
-> There does not yet exist, flexible general purpose
UbiCom robots, which can act as autonomous assistants or
servants for mass human use.
Ubiquitous computing: smart devices, environments and interaction
123
Overview
•
•
•
•
•
•
•
Introduction 
Tagging the Physical World 
Sensors and Sensor Networks 
Micro Actuation and Sensing: MEMS 
Embedded Systems and Real-time Systems 
Control Systems (For Physical World Tasks) 
Robots 
Ubiquitous computing: smart devices, environments and interaction
124
Summary & Revision
For each chapter
• See book web-site for chapter summaries, references,
resources etc.
• Identify new terms & concepts
• Apply new terms and concepts: define, use in old and
new situations & problems
• Debate problems, challenges and solutions
• See Chapter exercises on web-site
Ubiquitous computing: smart devices, environments and interaction
125
Exercises: Define New Concepts
• Annotation
Ubiquitous computing: smart devices, environments and interaction
126
Exercise: Applying New Concepts
Ubiquitous computing: smart devices, environments and interaction
127
Supplementary Slides
• Exercises & Solutions
Ubiquitous computing: smart devices, environments and interaction
128
Sensor Applications
Ex: Give some examples of sensor use
• Cars: air pressure, brake-wear, car-doors, engine etc
• Lap-top: accelerometers – switch off computer disks
when dropped
• Retail, logistics: RFIDs
• Heaters: thermostats
• Infrastructure protection / Intrusion detection (active
sensors)
• Environment monitoring
• Industrial sensing & diagnostics
• Battlefield awareness
• Sensors can be characterised according to:
– passive (tags) vs. active
– Single sensors vs sensor arrays vs sensor nets
– Read-only program vs. re-programmable
Ubiquitous computing: smart devices, environments and interaction
129