Transcript Computer Control of Theater Performance Electronics

IOWA STATE UNIVERSITY
Department of Electrical and Computer Engineering
April 26, 2006
SD May06-18
Computer Control of Theater
Performance Electronics
Clients:
Co-Motion Dance Company
Iowa State Dance
Team Members:
Faculty Advisors:
Amanda Farniok
Dr. Julie Dickerson
Sheng Ly
Prof. Gerald Sheble*
Alex Sills
Dr. David Stephenson
Tarun Bhatia*
*Only on project Fall 2005
Presentation Outline





Project Overview
Research Activities
Hardware
Configuration
Software
Development
Implementation






Resources and
Scheduling
Commercialization
Future Work
Lessons Learned
Closing Remarks
Questions
Definitions







Isadora - Isadora is a graphic programming environment that
provides interactive control over digital media, with special
emphasis on the real-time manipulation of digital video.
SDK - Software development kit
Actor – Module for Isadora software developed as an end
product.
Scene – A collection of user grouped actors in Isadora.
TinyOS - TinyOS operating system designed for the Crossbow
transceivers
nesC - an extension to the C programming language designed
to embody the structuring concepts and execution model of
TinyOS.
OSC – Open Sound Control
Acknowledgement








Janice Baker – Iowa State Dance
Valerie Williams – Co-Motion Dance
Company
Dr. Julie Dickerson – faculty advisor
Dr. Gerald Sheble – faculty advisor
Dr. David Stephenson – emeritus faculty
Jason Boyd – ECpE departmental technician
Daji Qiao – associate professor
Wen-Chieh – HCI 572 student
Problem Statement




Control dancer environment without the
help of stagehands or technicians
Need dancer-operated sensor pack which
sends input to be received by a computer
program called Isadora
Isadora can be programmed to control
lighting, video projection, audio, and other
characteristics of the dancer’s stage
environment
Similar commercial technologies are not
able to differentiate between the inputs
Solution-approach Statement




Crossbow Technology Components: Sensor
Board, Transceivers, Serial Interface Board
Flex sensors (variable resistors) used to
monitor movement
Used nesC code in TinyOS (Crossbow
program) to program components and
forward data
Used C code to write serial-to-OSC relay
Operating Environment



Theatrical dance settings: Betty Toman
Dance Studio in Forker Hall, Fischer Theater,
Stephens Auditorium, Ames City Auditorium
Possibility of ambient electromagnetic noise
caused by the multitude of electrical
equipment in the immediate vicinity
Substantial abuse caused by the motion of
the dancers, their impact with the floor or
wall, and the sweat and heat that their
bodies produce during performance
Intended Users and Uses
Users
 Members of Iowa State
Dance
 Members of Co-Motion
Dance Company
 Directors and colleagues of
the companies
Uses
 Practices, performances,
and productions of the
users
 Usable in any situation
analogous to the conditions
of these groups’ activities
 Transmitter is within the
requested range of 60 feet
from the receiver
 Impacts inflicted on the
transmitter and sensor by
the dancers are minor
Assumptions and Limitations
Assumptions
 Flex sensor data
transmission only
 No clear line of sight from
the transmitter to the
receiver
 Receiver and computer, as
well as the visual aids, will
run on power supplied from
an external source
 Sufficient power to operate
for at least the duration of
an ISU/Co-Motion dance
production
Limitations
 End-product will be only
one transmitter/receiver
pair but will allow for
expansion for up to four
transmitters
 Small and compactly
packaged to allow for full
range of motion by the user
 Distance from the
transmitter to the receiver
will be at most 60 feet
End-Product and Deliverables




Wireless transmitter with four sensors
Receiver connected through
Serial/USB adapter to computer
Sensor watcher written to
communicate between TinyOS and
Isadora
User Manual for client to recreate and
use full functionality of system
Data Flow - Functionality
Hardware Flow Diagram
Flex Sensor, Analog Voltage Signal
MICAz Receiver Mote
MDA300 Analog Input Board
MIB510 Serial Interface
MICAz Transmitter Mote
HARDWARE CONNECTED TO COMPUTER
HARDWARE WORN BY DANCER
2.4 GHz
Wireless
Transmission
Previous Research Project


VrJuggler was previously used on a research
project to manipulate input from sensors and
display 3D video on projectors.
Certain aspects of previous research were taken
into consideration
– Flex sensors
– Crossbow Technology Inc.
Present Accomplishments




Transmitter/receiver tested for range and
use with sensors, being used in theater
Hardware programmed through nesC for
correspondence between transmitter and
computer
Packaging purchased, installed
Able to get sensor readings into Isadora
Future Tasks


Complete User Manual
Purchase Crossbow system for client’s
possession
Hardware Design Research

Analog audio wireless transmission
method
– Problems: Modulation/compression
issues, capacity/channel issues, possible
interference problems
Hardware Design Research

Industrial SCADA-type wireless transmission
method
– Problems: Bulky packaging, insufficient
channels/capacity, possible software (Isadora)
compatibility issues
Hardware Design - XBow

Crossbow “Mote” Transmission Method
– Reasons for choice: Compact sizing,
2.4GHz band avoids interference and
allows for multiple transmitters, TinyOS
accompanying software suited for project
goals
– Possible problems: TinyOS/Isadora
communication, line of sight issues
Hardware Design - XBow
Analog Output Circuit


Crossbow motes accept analog inputs
Flex sensor variable resistors used in a
simple analog circuit to give 1.31 - 1.95
Volt output to mote board
Circuit Diagram - Four
Sensor Output
MDA300 Sensor Board



Accepts analog inputs via screw terminals
Provides excitation voltages
Converts analog to digital signal for wireless
transmission
MICAz Transceivers



Connects to sensor board and serial interface
with 51-pin connector
Transmits on 2.4 GHz Zigbee compliant band
(good range, multiple channels, minimal
interference problems)
Powers sensor board, simple AA battery power
supply
MIB510 Serial Interface



Receives signal from transceiver and transmits
to computer through TinyOS software
Powered by external AC adapter
Through TinyOS, programs/configures
transceivers and sensor board
Hardware Software
Integration

TinyOS is a platform bridging the
hardware serial input using nesC
– Provides Libraries for parsing sensor data
– Environment for the Programming the
Wireless Motes.

Program for receiving wireless data
and putting on the serial port provide
by Xbow
Wireless Transmitter
Program

Developed in nesC
Isadora Module
Development


Provides overall design needed for
working Isadora Plugin/Actor.
Provides data structures and functions
that are needed to implement.
General Flow of the
Functions
Open Sound Control
Solution
– C backend solution
Isadora Actor Mockup
Our Actor Mockup
Actor Link Connection
Putting the Pieces Together
Implementation


Integration between hardware and software
using the research and documentation
Implementing in theater with components
on loan from the Department of Electrical
Engineering
Testing Procedures and Results

Measuring flex sensor voltage
– 1.31 to 1.95 volts

Distance between transmitter/receiver and
line of sight issues
– Work up to 60 feet and without line of sight
– Reliability issues when in enclosed corners


Packaging integrity
Currently beta testing by client for use in a
performance
Resources
F i n a n c i a l R e so u r c e s
$ 50. 00
$ 20. 00
$ 45. 00
Cr os s bow Component s
Fl ex s ens or s
P ac k agi ng
Wi r i ng/ M i s c . P ar t s
Perso nnel Ef f o rt s
$ 590. 00
Sheng Ly
30%
Alex Sills
26%
Alex Sills (191.5 hrs)
Tarun Bhat ia (121.5 hrs)
Amanda Farniok (208.5 hrs)
Amanda Farniok
28%
Tarun Bhatia
16%
Sheng Ly (221hrs)
Schedule
Project Evaluation

Research of Hardware and Software
– 100% Complete
– Using findings to implement system

Testing
– 95% Complete
– Hardware works with software

Implementation
– 98% Complete
– Software designed and loaded onto hardware, currently in use

Documentation
– 95% Complete
– All formal reports complete
– User Manual under revision
Commercialization





Theatrical companies readily share
technological findings
Share findings with those that shared
information with us
Consult Crossbow Technology before any
package sales could be done
Selling Price: $1,000
Market: High demand in performing arts in
near future
Future Recommendations






Switch batteries from AA to 3 V coin
Custom design casing for compact yet
safe design
Connectors for between sensors and
wires and stability of sensor joint
Expand to six sensors per transmitter
Expand to four transmitters
Use different sensor input
Lessons Learned
Successes



Group teamwork and communication
Group supervision by faculty
Attainable trial hardware
Lessons Learned
Setbacks



Loss of team member
Original circuit design not applicable
Developing software before hardware
was decided
Lessons Learned
Experience Gained






nesC Programming
RF communication
Multitasking
Client relations
Communicating different expertise
Delegating responsibilities
Lessons Learned
If we did it all again…


Hardware team more involved with
software development
More research on software crossplatform development
Risk and Risk Management

Procurement of Materials
– Hardware borrowed from department for testing and
implementation before purchase

Loss of team member
– Encountered and overcome by members gathering
completed progress and taking on more tasks
– Involved other resources – HCI 572 Design and Evaluation
of HCI

Data Loss
– All documentation easily transferable between group
members

Physical Damage
– Kept components in tackle box for safe keeping
Closing Remarks




Successfully provided means for
dancers to control own environment
Human computer interaction
Leading other companies in theatrical
technology
Expansion is possible
Questions