Transcript Components for Cosmically Distributed Systems

Components for
Cosmically Distributed
Systems
Flight Software Workshop
November 2009
David E. Smyth
Principal Computer Scientist
David E. Smyth, November 2009
Slide 1
My Background
FSW for missiles, SR-71, Space Shuttle, …
 Research in software fault tolerance, AI, robotics,
operating systems, languages, distributed systems,
integration labs, …
 Business systems for DirecTV, UBS, Alliance, Siemens,
many dot.coms, …
 At JPL, architect/cog-e/developer:

 Ground Data System (SFOC SEQ, DMD, TDS, …)
 Mars Pathfinder (MPF)
 Software Fault Tolerance (DRC)
 Mission Data System (MDS)
 Mars Exploration Rovers (MER)
 Mars Science Laboratory (MSL)
David E. Smyth, November 2009
Slide 2
Mars Pathfinder:
Better, Faster, Cheaper
 Mars mission initially proposed at $2 Billion
 Viking missions over $3 Billion in current dollars
 Dan Goldin’s response:
 Do it for $150 Million or you won’t do it!
 And if it even looks like you’ll exceed the budget, you’ll be
cancelled that day!
 A few crazy engineers
 OK. We can do that
David E. Smyth, November 2009
at JPL:
Slide 3
We Did It!
 Always
under budget
 Beat every deadline
 Provided additional capability
 State-of-the-art AI
 Extremely high quality
 4 bugs in entire mission
 Not a sweatshop
 Time to learn together
 Time to think
 Flexibility to try new things
David E. Smyth, November 2009
Slide 4
Interesting Aspects of the
Flight Software
Layers
with Different Technologies
Objects
Responsible Objects
Components
Orchestration
Artificial Intelligence that Works
Distributed Computing the Easy Way
David E. Smyth, November 2009
Slide 5
Layers with Different
Technologies
Pathfinder
Domain Concepts
Policies
Strategies
Mission Safety and Success
Primary or Alternatives
Tactics
Deploy, Find Sun, Panorama
Entities
Xmitr, Camera, Battery, Heaters
Archives
Telemetry, Science, Diagnostics
David E. Smyth, November 2009
Slide 6
Layers with Different
Technologies
Human Organizational
Domain Concepts
Policies
Strategies
What an Executive May Change
What a Team May Do
Tactics
What a Specialist May Do
Entities
What a Specialist Manipulates
Archives
What is Remembered
David E. Smyth, November 2009
Slide 7
Layers with Different
Technologies
What Happens
at Each Level
Policies
Orchestrate
Strategies Collaborate
Tactics
Encapsulate
Entities
Interrogate
Archives
Administrate
David E. Smyth, November 2009
Slide 8
Layers with Different
Technologies
Pathfinder
Implementation Concepts
Policies
Orchestrate
Parameters
Strategies Collaborate
Modes
Tactics
Encapsulate
Responsible Objects
Entities
Interrogate
Hardware Objects
Archives
Administrate Infrastructure
David E. Smyth, November 2009
Slide 9
Layers with Different
Technologies
Pathfinder
Implementation Technologies
Policies
Orchestrate
Parameters
Data and Logic
Strategies Collaborate
Modes
Switch & Methods
Tactics
Encapsulate
Resp Objs
OOP in C
Entities
Interrogate
HW Objs
C (Real-time)
Archives
Administrate Infrastructure IpcQ, VxWorks
David E. Smyth, November 2009
Slide 10
Components
A
Better Definition of Good Software
Better
Good
Objects
Tool for Re-Use
Better
Quality Software?
David E. Smyth, November 2009
Slide 11
Pathfinder Objects
(Components)
Infrastructure
 Hardware Objects
 Responsible Objects
 Spacecraft Mode Commander

David E. Smyth, November 2009
Slide 12
Infrastructure
 Initialization,
Interrupts, Time, Error Logging
 IpcQ
 190-350 LOC (flight-distributed)
 Distributed, multi-threaded system of single threaded objects
 Minimize semaphore grabs
 External interfaces looked like functions, but built message
structs, copied into queue, read and dispatched within
receiver’s event loop
 Little language for specifying interface, generating include files,
msg structs, send/recv, event loop, and dispatch
 Contributed one of the 4 bugs...
David E. Smyth, November 2009
Slide 13
Hardware Objects
 One
per hardware device
 Real-Time Code
 Minimized response requirements via buffering and timers
 Interrupt
handlers
 Register read/write
 Information abstraction
 Type conversion (int --> 3-bits --> int)
 Device level transactions
 disable interrupts, read, mask, write, set “command” bit in CSR,
re-enable interrupts
David E. Smyth, November 2009
Slide 14
Responsible Objects
 Initially, one per
 RFS, ACS, 1553, …
“subsystem”
 Eventually, one per tactical behavior
 Downlink, Surface Pointing, Entry-Descent-Landing, …
 Responsible:
Nominal and
Foreseeable Off-Nominal Behavior
 No status nor failure return values
 GREATLY reduced size of ACTUAL interfaces
 Eliminated a page of code per call within camera system
 GREATLY reduced state dependencies
 Enabled exhaustive testing
 Trivialized integration
David E. Smyth, November 2009
Slide 15
Examples of Responsibility

DWN_DownlinkSession( startTime, endTime, rate )

SPO_FindSun( onSuccessCB )
 vector = Dali_LikeWhereIsTheSunMan()

ACS_EarthPoint()

EDL_DoEdl()

RVR_DeployRover()

IMP_Panorama( time, filter )
David E. Smyth, November 2009
Slide 16
Responsible Objects were our
Components
 Strongly
typed interfaces
 Specific patterns
 Objects
 Setters
 Subscribe for change or other events (no external Getters)
 Boot, Init, Start, Mode Change
 Act-at-time, Seek and Maintain Goal
 IpcQ
 Demeter
 Independent
 stubs, simulators, real thing
David E. Smyth, November 2009
Slide 17
Orchestration
 Spacecraft Mode Commander
 A dozen or so modes
 Each mode would set every property of every object
 Generally let responsible objects handle details
 Lots of parameters
 No logic except mode switch
David E. Smyth, November 2009
Slide 18
Artificial Intelligence that Works
 SCM
 Hormonal
 Responsible
 Hardware
 AI?
Objects
Objects
On a Spacecraft?
No, Not Us!
David E. Smyth, November 2009
Activation
 Behavior
 Reflex
 Rod
 Maja
Brooks
Materic
Slide 19
Components for
Productivity and Quality

Components = Objects + Interfaces + Patterns
 Key Pattern for Distributed Systems:
The Law of Demeter
Simplified Distributed Computing
Faster Development, Increased Productivity
 Trivial Integration
 Increased Quality and Stability

David E. Smyth, November 2009
Slide 20
Simplified
Distributed Computing
 The Law of Demeter
 Any method must operate only on the object’s existing state
and the arguments to the methods: no data shall be required
from any other source during the execution of any method.
 First Corollary
 No method can return any value synchronously (otherwise, the
user of the method would be violating the law)
 Observations
 Dataflow with Objects
 Each object can have its own thread
 Temporal coupling and synchronization is eliminated
 Each object can be distributed, in any location
David E. Smyth, November 2009
Slide 21
Eliminate State Coupling via the
Law of Demeter
 Second Corollary
 Objects generally have state, but their interfaces must not
– or –
 Objects must be able to accept any message at any time
 Observations
 Message order is irrelevant
 Objects can have multiple message queues
 Objects can choose which messages to dequeue, which to
leave buffered
 Objects can re-order their own messages
David E. Smyth, November 2009
Slide 22
The Web and Demeter
 Web based enterprise
 Extremely quick to deploy
 Very flexible
 Very reliable
information systems
 The Web obeys the law!
 URL hit: cgi/servlet/asp acts on form object only
 post data + page objects + DB objects, but all generally
closely related
 No interface state
 Object state in cookies, hidden, JS, DB, …
 Messages (URL hits) in any order
 Watch those bookmarks!
 Multiple browser windows
David E. Smyth, November 2009
Slide 23