Transcript P2.10_Sound
Sound you tube Some from Heim Chap 13 The sound files are not included– most should work on windows 7 machines Learning outcomes • Describe the basics of human hearing • Explain the difference between visual and auditory interaction • Describe the classes and subclasses of sound output and the attributes of each • Describe the classes and subclass of sound input and recognition and attributes of each 2 Hearing • Provides information about environment: distances, directions, objects etc. – protects inner and amplifies sound – transmits sound waves as • inner ear vibrations to inner ear – chemical transmitters are released and cause impulses in auditory nerve • Sound • pitch • loudness • timbre – sound frequency – amplitude – type or quality the human 1 • outer ear • middle ear 3 • Physical apparatus: Hearing (cont) • Humans can hear frequencies from 20Hz to 15kHz • Auditory system filters sounds • can attend to sounds over background noise. • for example, the cocktail party phenomenon. • Hearing aids disrupt this filtering the human 1 4 • less accurate distinguishing high frequencies than low. • Higher frequencies disappear as you get older What if…. • You are in a noisy environment • Night clubbing the human 1 • Your hearing is below average • You are deaf 5 • Phone call/ text message? Sound versus Visual Sound exists in time and over space, vision exists in space and over time. (Gaver, 1989) - Sound is only there when it is playing/made - Vision is there until it is replaced 6 Sound Interaction • Computer Output/Generation (Human Input) • Non speech • Music • Audio Icons and Earcons • Speech • Computer Input/Recognition • Speech • Non speech • Environmental • Music 7 Computer Output: Music • Can be pre-recorded or generated • Movies • Games • Immersive experiences 8 Generating music • Exciting area for artists • Everything from pseudo real to completely abstract • There are Jazz music generators that only skilled people can differentiate from actual musicians. • Serato – dj software (www.serato.com) • Auckland company doing fantastic things • Several UOA grads there 9 Auditory Icons and Earcons • The difference between these two is subtle • ‘Natural’ sounds • ‘Artificial’ sounds (generated) Windows hardware fail insert remove 10 Auditory Icons and Earcons • Redundant Encoding • It aids memory by adding additional associations. • Can alert without interrupting • An alterative communications channel. • Positive/Negative Feedback • Auditory alarms might be crucial to the safe operation of computer-operated machinery or mission-critical environments • Too many alarms • Annoying • Ignored 11 Using Sound in Interaction Design • Learnability of the mapping between the icon and the object represented • “Oink” and “bow wow” have high articulatory directness • A swishing sound accompanying a paintbrush tool also has high articulatory directness • A system beep carries is an abstract representation 12 Innate human understanding • Volume <quiet -------------------------------------------------------------------loud> • Whisper • Shout • Pitch • Baby crying • Reassurance • Authoritative • Rhythm • Urgency • Mood • Length • Start, stop, duration 13 Can you remember earcons? • How many? • How often do you hear them? • Can you intuitively tell what these mean? On Off Sleep 14 Misrecognized Disambiguate Speech Output • Eyes free operation • Alternative output channel • Good for checking your essays • Navigation is hard • Back tracking, • Finding location of particular thing 15 Speech Output • Recorded • Menu choices for telephone systems • Books or other multimedia experiences • Generated • Good voice synthesisers built in to op systems • Windows navigator • Can give pronunciation rules • Still sound a little artificial 16 Sound Input •Speech •Environmental •Music 17 Speech Recognition • Two distinct applications: • Transaction • Transcription • Transaction • Telephone menu systems • Choose from a limited number of options, works ok • Automatic speech recognition (ASR) • Built into operating systems • Siri (iPhone) and Android are ~~ usable • This is a triumph of Artificial Intelligence • Very difficult, ongoing research problem 18 Searching Speech and Audio • Sound files do not afford easy opportunities for indexing and searching • Speech translation can be used to transcribe speech files and create transcripts that can be searched like any other text file • So long as recognition accuracy is ok, which it isn't at the moment • Tune identification apps • hum a bit of the tune and it tells you what it is! • Easier than translation 19 Summary • Describe the basics of human hearing • Explain the difference between visual and auditory interaction • Sound is transitory • Describe the classes and subclasses of sound output and the attributes of each • Non speech • • Music Earcons • Speech • Describe the classes and subclass of sound input and recognition and attributes of each • Speech • • Transaction Transcription • Sound is more complex than colour/light 20