Transcript Embodied Speech and Facial Expression Avatar Design Proposal

Embodied Speech and Facial Expression
Avatar Design Proposal
sss Dan Harbin - Evan Zoss - Jaclyn Tech - Brent Sicking sss
February 9, 2004
Problem Background
• Yano is capable of conveying many emotions through his
facial expressions
• We propose to reverse engineer this toy and create a
puppet capable of showing happiness, sadness, anger, etc.
• By adding an additional microphone input we can measure
the intensity of the speech and adjust the facial expressions
accordingly
Goals and Objectives
• Gain knowledge of robotics systems, reverse engineering,
and sensor/actuator interfacing
• Experience with human computer interaction and
multimodal interfaces
• Develop system integration skills with low- level hardware
control and graphical user interfaces
• Practice and improve our design methodology,
management process, and teamwork skills in order to
insure a quality product
Technical Survey
The Furby
• In 1999, Peter van der Linden issued a challenge with a
$250 reward for anyone who could hack Furby and make it
reprogrammable
• The first hacked Furby went to a mother of an autistic child
who had written van der Linden early in the competition
• Playing with the Furby was actually improving her son’s
speech, however she was concerned about the gibberish
the Furby speaks
• These efforts at hacking Furby brought
about the “Hack Furby Kit”
Technical Survey, Continued
Yano, Robotic F.A.C.E.
• The Media Lab at MIT created an output device for the
computer, similar to a monitor or speakers, that outputs
facial emotions
• They took out all the internal electronics of Yano, and
removed his body, so they were left with a head that had 3
motors and some gearing
• The motors are controlled by three MIT developed
microcontrollers, called iRX Boards
• Used their hacked Yano with some MIT software called
Listenin, a remote monitoring program that detects speech
and other sounds remotely and sends them back to the
local computer where Yano makes speaking movements
accordingly
Technical Survey, Continued
Yano, Robotic F.A.C.E.
Selecting a Toy: Furby vs. Yano
Furby
• Movement in the eyes,
eyelids, mouth, ears, and feet
– all controlled by one motor
• Sensors for interaction in the
head, stomach and tongue
• Speaker, microphone, and
infrared i/o
• Internal electronics are packed
in tight and sealed in a tough
plastic case with resin
Yano
• Movement in the eyelids,
eyebrows, ears, mouth,
cheeks, head, and hands
• The mouth is opened and
closed with one motor, the
cheeks are moved up and
down with a second motor,
and the ears, eyes, and
eyebrows all run off of the
third motor
• Speaker and infrared receiver
Selecting a Toy: Yano
• Yano gives us more control over the face muscles (Furby
only has a beak that can open and close) and has 3 motors
that we will be able to independently configure
Selecting an Interface
• USB - dependent on a master-slave type relationship that
requires control from the slave side
• Parallel Port - offers a simple way to control the motors
and read the sensors all in parallel; however, it would
require us to do the timing of the motor controls in
software which would be unreliable
• Serial Port - requires a microcontroller to go from the
serial communication with the computer to a parallel i/o
control for Yano; ability to time all communications to a
certain baud rate
Selecting an Interface
• USB - dependent on a master-slave type relationship that
requires control from the slave side
• Parallel Port - offers a simple way to control the motors
and read the sensors all in parallel; however, it would
require us to do the timing of the motor controls in
software which would be unreliable
• Serial Port - requires a microcontroller to go from the
serial communication with the computer to a parallel i/o
control for Yano; ability to time all communications to a
certain baud rate
Selecting a Microcontroller
OOPic
• ability to program the PIC to do the serial to parallel
conversion
• object-oriented programming
• we will need to decode serial protocols for serial to parallel
conversion
• unsure if the OOPic will be able to use the serial port as
input in the running program
Selecting a Microcontroller, Continued
SV203 “microcontroller based servo motor control”
• It has a serial port connection, the SV203 processor, 8
servo control ports, and 5 input pins
• You give it commands in ASCII text through the serial
port, and it controls the servo ports, or reads the input pins
based on the commands
• Pulse Width Modulation
• It may be more difficult to program than the OOPic, but
the manufacturer has APIs for C and Visual Basic to make
it much easier
Selecting a Microcontroller, Continued
• Overall, the OOPic is more geared toward programming
then letting it stand alone and control a device, and the
SV203 is more geared toward direct control of the device
through the serial port.
• We chose to try the SV203 for our design.
Proposed Design: Hardware
• Disassemble and hack Yano
– locate all the motors, and test them using 6 volts of DC
• Control motors with SV203
– use a transistor to enable or disable each motor, and to
use an inverter to reverse direction
– the transistors and the inverters will be connected to the
logic and ground pins of the SV203
– digital input to the control circuit will be generated by
sending commands to the SV203 through its serial
interface
– connect the end switches to the input pins of the SV203
– read the pins values using HyperTerminal
Proposed Design: Low-Level Software
• Create rudimentary serial interface for SV203
– write function to generate the command to move a motor to a
certain position in a clockwise or counterclockwise direction
– read function to read the end signal of the specified motor
• Calibration and per-motor control
– all motors will be driven in one direction until they reach the end
and a switch is triggered
– then all motors will be driven in the opposite direction until they
trigger the end switch, this time the number of pulses will be
counted
Proposed Design: High-Level Software
• Psychology of facial expressions and emotions
– combine the move functions of various motors
– create functions for many different moods that will move the
motors to certain positions to express emotions
• Graphical User Interface
– design with the RAD tools included in Microsoft Visual Studio
using C++
– matrix filled with pictures of different moods; when clicked, Yano
will mimic the picture
– sliders for each motor allowing for tweaking of the position of
each motor to create custom expressions
– voice interface to translate the intensity of the user's voice into
mouth motor movements
– calibration button
Approach for Design Validation
• Use a power supply and a voltmeter to check the motors and end
triggers of the disassembled Yano
• Use HyperTerminal to drive the motors in both directions with the
SV203, and to read the end conditions from the SV203 reliably
• Redo the last test with the C++ interface
• Execute the calibration test mentioned in the previous calibration
section. Also, move the motors to many different positions, and see if
the results can be reproduced reliably
• Check that moving between emotions is reliable, and that the
expressions are correct
• Have a user test the GUI for intuitiveness, and see if the voice interface
properly drives the Yano's mouth
Economic Analysis and Budget
Part
Cost
Yano (we will want 2 in case one is broken)
$60/ea.
SV203 (we may also want 2 of these)
$59/ea.
Transistors (for motor control)
$20 max
Serial Cable
$15 max
Total
$273
Team Management
• Brent – GUI layer manager; knows Visual Studio, C++,
and .NET
• Jaclyn – Software and API manager; psychology expert;
develop algorithms for controlling the motors, and
generating facial expressions
• Evan – Motor control circuits; enable and disable the
motors, and change their directions
• Dan – Serial port interface and communication; provide a
hardware/software interface for controlling the hardware
Schedule of Tasks
Societal, Safety, Environmental Analysis
• Practical benefits to society, especially in the area of child
therapy
– Example: help autistic children communicate better and
open up to adults
• To ensure a safe project, all of circuits must be safely
enclosed in the chest cavity of the Yano
• Yano is an environmentally- friendly toy with the
exception of the batteries it uses
– Alternative 1: DC wall adapter
– Alternative 2: rechargeable batteries
Thank You
•
Information on MIT’s Project
http://web.media.mit.edu/~stefanm/yano/
•
SV203 Motor Controller
http://www.pontech.com/products/sv200/index.htm