Transcript Document

Alternative communication
& access to information
Grigori Evreinov
Department of Computer and
Information Sciences
University of Tampere, Finland
www.cs.uta.fi/~grse/
January – March, 2003
ACAI
information may be defined as the characteristics of the output of a
process; these characteristics enhance knowledge, being informative about
the process and its input
hierarchies of processes, linked together, provide a communication
channel between each of the corresponding functions and layers in the
hierarchies
fundamentally information is not modality-specific (i.e. not tied to its form of
presentation, be it visually, as sounds or whatever)
but the end user (destination, addressee) for whom the information exists,
is a person who is trained to manipulate through modality-specific notions
(which are formed by percepts)
therefore, within ‘information technology’ modal aspect of information
maybe more essential for development of relations between person and
computer [1]
http://www.fst.ch/FST2/al/default.htm
TAUCHI MMIG Evreinov G.
p 01_23
22.01.2003
ACAI
elements of communication process [3]
communication is defined as the transfer of information from a
communicator-source /originator to a communicator-receiver
the communication process allows people to share information, ideas, and
feelings; this is the transfer of meaning
where no meaning is transferred, no communication has taken place
the goal of communicators is to accomplish this process efficiently
(rationally) and effectively
TAUCHI MMIG Evreinov G.
p 02_23
22.01.2003
ACAI
human communication has no other purpose than to cause some kind of
action - to direct, to inform, to question, and to persuade [2]
the challenge is to come up with the right combination of codes, media, and
contexts in order to make the transfer of information fast, cost effective,
and accurate
ideas and feelings can be shared only if they are represented by symbols
symbols are things that stand for something else
signs are literal; symbols are not
a symbol means more than it literally says [4]
all of our communication messages are made up of two kinds of symbols:
verbal and nonverbal
symbols can have several kinds of association like personal, cultural or
universal
interpretation of symbols’ meaning depends on the set of parameters,
conditions and features of elements involved in communication process
TAUCHI MMIG Evreinov G.
p 03_23
22.01.2003
ACAI
the ideal message is one that is received & interpreted as it is intended
a receiver is a one or more individuals for whom the message is intended
to be an effective, communicator first gain and keep the receiver's attention
communicators must have similar knowledge /background with regards to
usage of the medium or get knowledge before during learning
communication conditions
noise is interference of signals with similar parameters in analyzers that
keeps a message from being understood or accurately interpreted
psychological noise occurs in the minds of the communicators and the
receivers when they are distracted by something
there are different types of communication: intrapersonal, interpersonal,
interviews, small group, and public
TAUCHI MMIG Evreinov G.
p 04_23
22.01.2003
Your Appearance and
Physical and Psychological Condition
Healthy
Good frame of mind
Neat
Clean
Stylish
Etc.
Your Social Traits
Outgoing
Assertive
Warm
Empathic
Etc.
Talents You
Possess or Lack
Artistic
Musical
Athletic
Writing
Speaking
Etc.
Your Moods and Feelings
Your Self
What you say
What you think
What you feel
How are you
ACAI
Humor
Anger
Hate
Love
Contentment
Etc.
Your Social Roles
Your Intellectual
Capacity
Parent
Family provider
Community leader
Professional person
Etc.
Your Strong
Beliefs
Logical
Reflective
Studious
Speculative
Etc.
intrapersonal communication occurs within us
it involves thoughts, feelings, and the way we look at ourselves [2, 5]
TAUCHI MMIG Evreinov G.
p 05_23
22.01.2003
ACAI
communication messages are made up of two kinds of symbols:
verbal and nonverbal
voice set (pitch, volume, rate and rhythm) is defined by the speaker’s
physical and emotional condition
certain kinds of voice sets are strongly classified with gender and age
there are three types of nonverbal vocalizations [7]
vocal characterizers (yawning or belching )
vocal qualifiers (momentary variations of speech or volume,
personality traits )
vocal segregates (sounds or silences made between expression
of words, such as ahh, hah, mmmh, and include pauses during speech,
which can convey an attitude of calmness and reflection) [6]
however, the information and richness contained in paralanguage, music,
and background sounds is lost to the deaf viewer [10, 11]
Demo 1
Demo 2
Demo 3
Demo 4
Demo 5
Demo 6
Demo 7
Demo 8
Demo 9
Demo10
[9] http://www.esl-lab.com/para.htm
TAUCHI MMIG Evreinov G.
p 06_23
22.01.2003
ACAI
Sonification of
facial expressions
on the contrary, for blind viewer… how to transform emotional
expression from the picture into sound/earcon or whatever [12]
there is not any system to reveal an emotional access to pictures for
blind persons
surveys show that it is very important for blind to transport emotions
independently of the content especially for modern images
the sonification system can enable an emotional access using the
similarity of feeling by watching a picture or hearing music and
transforming visual information in music compositions
results of psychological, physiological and emotional acoustic
researches, methods of image analysis and the rules of music
composition are basis for this translation and were describe by
Schwende in Auditory Emotional Access to Visual Information [33]
TAUCHI MMIG Evreinov G.
p 07_23
22.01.2003
ACAI
Sonification of
facial expressions
propose or design a system of very brief eARmoticons which could
describe a facial expression and evoke the same reaction
http://www.newag.org/funpics/weard/html/imagepages/image68.htm
http://www.berghuis.co.nz/abiator/weird/ears.html, http://www.velvet.net/~poppy/kitch/,
http://www.domainnamesanity.com/default.php,
http://www.dogandponysound.com/ico.htm
TAUCHI MMIG Evreinov G.
p 08_23
22.01.2003
ACAI
tactile diagrams are relatively easy to produce yet there are no
standard guidelines to assist during the design process
attempting to include as much information as possible can lead to
diagrams that are simply too confusing to be of use
research into a project on non-visual access to music notation
(Challis, “Weasel”) has shown that an interactive multimodal
approach
to
presentation
could
be
of
benefit
in
these
circumstances
Weasel, http://www.benchallis.com/research.htm#Weasel
National Centre for Tactile Diagrams http://www.nctd.org.uk/
GEWA Access Computer Keyboards - Remote Computer Access http://www.zygo-usa.com/accessir.html
Tellus und Mind Express http://www.weissenstein-bs.de/produkte/softw/mindexpr/mindexpr.htm
TAUCHI MMIG Evreinov G.
p 09_23
22.01.2003
ACAI
Access to Mathematics
communication of mathematics is usually visual - formulas, diagrams,
graphs etc.
that makes it very difficult for blind and partially sighted people /students
to do mathematics and is one of the biggest obstacles for them in school
and at the university [13-20]
6-dot Braille can represent 64 unique characters
through the use of special sequences Braille can support a much larger
set of characters, that is, the basic characters can have different meaning
in different contexts
while there is effectively no limit to the number of characters one can
represent in the fashion, the reading and understanding of these special
sequences becomes very involved [13]
the more new characters one needs, the more “escape” sequences one
requires
e.g., the letter “a” could be an “a”, an “A”, a “1”, the “first finger” in music
notation and so on
http://presentations.animfactory.com/pmpjumbos.html TAUCHI MMIG Evreinov G.
p 10_23
22.01.2003
ACAI
Access to Mathematics
most of the work reported in developing techniques and deal with
mathematics can be presented through the next categories:
tactile as in Braille and other raised representations
audio aids that read equations with tools to help in the reading
process
**
Y=X
tonal representations of equations and graphs
Y = |sin X|
Y = 5sinX
*** (sonification/audification)
haptic or forced feedback devices represent shapes of objects
and curves
integrated /multimodal approaches
[25]
** http://www.gfai.de/sachsen/blind_i.htm
*** http://www.seeingwithsound.com/winvoice.htm
[24]
TAUCHI MMIG Evreinov G.
p 11_23
22.01.2003
ACAI
Access to Mathematics
the Mathtalk command language is based on the concept of a action and a
target
that is the user might choose to hear the (whole) current expression, then
the current term (i.e. the first term of the expression), followed by the next
term (i.e. the second term of the expression) and so on
the command language is mnemonic, so that, the command to read an
expression is r e, to move to the next term it is n t, etc. [19]
[20]
TAUCHI MMIG Evreinov G.
p 12_23
22.01.2003
ACAI
Stuttgarter Mathematikschrift für Blinde
the Stuttgarter Mathematikschrift für Blinde (SMSB) is an 8-dot-Braille
which includes small and capital letters, the numbers and mathematical
symbols
SMSB provides three alphabets, the Latin, the old German and the Greek
alphabet, to write letters which are used in mathematics
capital letters differ from the small ones by the supplemented dot 7
each character of SMSB has a representation in Braille and one in print
no SMSB-character stands for a string, a group of characters or even a
word as it could be in contracted Braille
that means, that each SMSB-character can be shown on a Braille-output-
device as well as on a screen and may be printed [22, 23]
SMSB provides tactile characters for symbols like
Access to Mathematics
TAUCHI MMIG Evreinov G.
p 13_23
22.01.2003
ACAI
Access to Mathematics
SMSB uses characters, which can be understood intuitively by a person,
who is familiar with mathematics
symmetric Braille symbols are symmetric in meaning:
TAUCHI MMIG Evreinov G.
p 14_23
22.01.2003
ACAI
Access to Mathematics
DotsPlus is a tactile font, but unlike Braille, it requires no translation from
ordinary literature [27, 28]
any computer file that uses fonts with screen characters having the right
size can be printed directly in DotsPlus on the Tiger tactile graphics and
Braille embosser
DotsPlus has many more symbols than Braille
DotsPlus and the Tiger can make graphics such as maps, floor plans,
pictures of animals, etc.
http://dots.physics.orst.edu/dotsplus.html
Demo 11
TAUCHI MMIG Evreinov G.
p 15_23
22.01.2003
ACAI
Special Braille
Technical Drawings
the domain of the technical
drawings
includes
electronics,
architecture and mechanical eng.
they contain graphical items that
follow industrial norms, like ISO or
other standards
transforming visual info in technical
drawings into an understandable
form for blind people is very
important for vocational training
digitized technical drawings can be
presented and edited on standard
PCs with appropriate software
blind people need to access a user
interface and presentation tool
specially tailored for this type of
info
http://www.braille-scs.com/tech_code.html
http://www.tedub.org/Start_tab.html
TAUCHI MMIG Evreinov G.
p 16_23
22.01.2003
ACAI
Access to Mathematics
Demo12
TAUCHI MMIG Evreinov G.
p 17_23
22.01.2003
ACAI
Access to Mathematics
H
J
K
1
7
L
D
7
8
9
4
5
6
1
2
3
1
8
9
F
G
4
5
6
1
2
3
S
1
A
C
X
Z
1:1
2:1
3:1
input & graphics sonification based on numerical keyboard’s paradigm
Demo13
TAUCHI MMIG Evreinov G.
p 18_23
22.01.2003
ACAI
TL, a language for creating games for visually impaired and blind children
[17, 18]
this language is a part of the TiM (Tactile Interactive Multimedia computer
games for visually impaired children) project whose overall aim is to offer to
young visually impaired children the possibility to play with computer
games
TAUCHI MMIG Evreinov G.
p 19_23
22.01.2003
ACAI
Gaze Control
http://www.metrovision.fr/mv-vi-notice-us.html
http://www.delta7.asso.fr/Deltavision%202001/ecrire3.html
the Camera Mouse http://www.cs.bc.edu/~gips/CM/ or http://www.cameramouse.com/
TAUCHI MMIG Evreinov G.
p 20_23
22.01.2003
ACAI
References
[1] Edwards, A. D. N, Carey, K., Evreinov, G. et al. Information and Communication Technology in Special
Education: Analytical Survey. Moscow: UNESCO, Institute for Information Technology in Education,
2001
[2] Effective Communication Skills, http://www.srsd.org/jrotc/Text%20Books/LEII_U~1.doc
[3] The Diffusion of Web Icons. Communication,
http://chrissnider.com/academic/icons/Theories/Communication/communication.html
[4] Definition of Symbols, http://web.umr.edu/~gdoty/classes/concepts-practices/def-symbols.html
[5] Intrapersonal Communication Processes. Joan E. Aitken & Leonard J. Shedletsky (Eds.)
http://www.usm.maine.edu/com/intrap.htm
[6] Non Verbal Communication, https://webspace.utexas.edu/QUAHS/WWW/
[7] Eisenberg, A.M., & Smith, Jr., R.R. Nonverbal communication. N-Y: The Bobbs-Merrill Company, Inc.,
1971
[8] John Carey, Paralanguage in Computer Mediated Communication, http://acl.ldc.upenn.edu/P/P80/P801018.pdf
[9] Paralanguage, http://www.esl-lab.com/para.htm
[10] Towards Emotive Captioning for Interactive Television,
http://www.ryerson.ca/clt/towards_emotive_captionin_1.doc
[11] Charles Silverman and Deborah I. Fels, Emotive Captioning in a Digital World, 2002, LNCS 2398, p.
292, http://link.springer-ny.com/link/service/series/0558/papers/2398/23980292.pdf
[12] Frank Deconinck and Patricia Verschueren, A model of the understanding of graphical information by
blind people, TIDE103/VUB/930531-2, 1993 http://minf.vub.ac.be/~fdconinc/underst.rtf
[13] Karshmer, A.I., Gupta, G., Gillan, D. Architecting an Auditory Browser for Navigating Mathematical
Expressions, ICCHP 2002, LNCS 2398, p. 477
http://link.springer.de/link/service/series/0558/papers/2398/23980477.pdf
[14] Gaura, P. REMathEx - Reader and Editor of the Mathematical Expressions for Blind Students, 2002,
LNCS 2398, p. 486, http://link.springer-ny.com/link/service/series/0558/papers/2398/23980486.pdf
TAUCHI MMIG Evreinov G.
p 21_23
22.01.2003
ACAI
[15] Fitzpatrick D. Speaking Technical Documents: Using Prosody to Convey Textual and Mathematical
Material, ICCHP 2002, LNCS 2398, p. 494,
http://link.springer.de/link/service/series/0558/papers/2398/23980494.pdf
http://www.computing.dcu.ie/~dfitzpat/publications.html
[16] Math project, http://www.cs.york.ac.uk/maths/index.html
[17] Prosody in Mathtalk http://www.cs.york.ac.uk/maths/robert/prosody.html
[18] Mathematical Access for Technology and Science,
http://www.papenmeier.de/reha/research/mathe.htm
[19] Edwards, A. D. N., Stevens, R. D. and Pitt, I. J. Représentation non visuelle des mathématiques,
(translated by A. Assimacopoulos) in A. B. Safran and A. Assimacopoulos (editors) Le Déficit Visuel,
Éditions Masson, pp. 169–178 (1995),
http://www.cs.york.ac.uk/ftpdir/pub/alistair/publications/ps/geneva.ps
[20] Karshmer, A.I., Gupta, G., Geiger, S., and Weaver, C.: Reading and Writing Mathematics: The MAVIS
Project, BIT (Behaviour & Information Technology), January 1999
[21] Calle Sjöström, Using Haptics in Computer Interfaces for Blind People,
http://www.certec.lth.se/doc/usinghapticsin/usinghapticsin.pdf
[22] W. Schweikhardt: 8-Dot-Braille for Writing, Reading and Printing Texts which Include Mathematical
Characters. In: A.D.N. Edwards, A. Arato, W.L. Zagler: Computers and Assistive Technology, ICCHP98,
Proceedings of the XV. IFIP World Computer Congress, Wien and Budapest, 31. August - 4. September
1998
[23] W. Schweikhardt: Requirements on a mathematical Notation for the Blind. In: R. Vollmar, R. Wagner
(eds), CompurtersHelping Peoples with Special Needs ICCHP 2000, pp. 663-670, July, 17-21. 2000
[24] Typographic Semantics of Webpages Accessible for Visually Impaired Users - Mapping Layout and
Interaction Objects to an Auditory Interaction Space, in Proceedings ICCHP 2000 - International
Conference on Computers Helping with Special Needs. Karlsruhe 17-21 July 2000, http://www-cghci.informatik.uni-oldenburg.de/resources/ICCHP2000-gorny.pdf
TAUCHI MMIG Evreinov G.
p 22_23
22.01.2003
ACAI
[25] Yu, W., Ramloll, R., Brewster, S.A. and Riedel, B. Exploring computer-generated line graphs through
virtual touch. In Proceedings of IEEE ISSPA 2001 (Kuala-Lumpur, Malaysia), IEEE , pp 72-75,
http://www.dcs.gla.ac.uk/~stephen/papers/ISSPA2001.pdf
[26] Oakley, I., Brewster S.A. and Gray, P.D. (2001). Communicating with feeling. Haptic Human-Computer
Interaction. Brewster, S.A. and Murray-Smith, R. (Eds.), Springer LNCS, Vol 2058, pp 61-68,
http://link.springer.de/link/service/series/0558/papers/2058/20580061.pdf
[27] Equalizing Information Access, http://www.viewplustech.com/
[28] Tactile Graphics, An Overview and Resource Guide, http://dots.physics.orst.edu/tactile/tactile.html
[29] TIM Project http://inova.snv.jussieu.fr/tim/
[30] Tactile and Multimedia Tools for Young Visually Impaired People
http://inova.snv.jussieu.fr/colloques/BNet2001/uk/programme.php
[31] Hammarlund, J. Computer Play for Children who are Severely Visually Impaired, TRC Rapport nr. 20
(1999)
[32] Anita Hildén MUSSE version 2.0 http://www.sih.se/pdf/musse2_lathund.pdf
[33] Schwende, H., Auditory Emotional Access to Visual Information, http://link.springerny.com/link/service/series/0558/papers/2398/23980445.pdf
[34] Gips James, DiMattia Philip A., Curran Marialice, Lees Debra, Accessing Internet Courses by Eye
Movement and Head Movement, http://link.springerny.com/link/service/series/0558/papers/2398/23980236.pdf
[35] World of Gestures, http://nonverbal.ucsc.edu/gest.html
[36] Language Invention: Resources & Links,
http://www.linguistics.ucla.edu/people/grads/jforeman/collegium/resource.htm
TAUCHI MMIG Evreinov G.
p 23_23
22.01.2003
ACAI
All references & presentations are available at:
http://www.cs.uta.fi/~grse/ACAI_2003/
http://www.bjorkasen.vgs.no/bcd/bcd0-99/ears2.htm
TAUCHI MMIG Evreinov G.
p 00_00
22.01.2003