Transcript Approaches to Communication - California State University, Fresno

Information Theory
Claude Shannon
John A. Cagle, Ph.D.
Communication
California State University, Fresno
Shannon & Weaver (1947)
Norbert Weiner’s Cybernetics added the notion of
feedback to this communication model.
Information Theory




In a perfect communication system, the
sender and receiver have identical knowledge
of the code.
All possible messages are known in advance.
The source makes a choice to send a
message from the set of possible messages.
The receiver needs to know what choice the
sender made.
Information is not content



A message has information if it
reduces the uncertainty about what
choice the sender made.
If the choice is already known to the
receiver, the message is redundant.
Information is not content in
information theory.
Information is not meaning



Information is different from the
content and meaning of messages.
Information is not the interpretation of
information.
Peter Drucker wrote of the difference
between informating and
communicating in an organization.
Entropy




Entropy is randomness, chaos, the lack
of organization and predictability.
Entropy is uncertainty.
Information reduces entropy in a
communication system.
Entropy is variable in most situations.
Measurement of Information




The smallest unit of information
is a bit
Eight bits = one byte
Four bytes = one word
These terms are still at the core of
computer science

E.g., 32-bit word processors in the CPU
Measurement of Information

I = - log2 pi
is the formula for measuring the
information value of each message sent
against the probability of that message
in the field of all the messages that
could be sent.
Measurement of Information

H = - ∑ pi log2 pi
is the formula for measuring the
amount of information of all the
messages that could be sent in a
communication system.
Choices


Signal: we make choices about which signal
to send (sounds, letters, etc.)
Semantics: we make choices in a given
situation about which meaning to send.



Lexical choice
Meaning
Pragmatics: we make choices in a given
situation about which behaviors to enact.
Communication behavior is a specific form of
molar behavior _____1_____ occurs in a
situation or field possessing specified
properties, ____2_____ part of which are in
interdependent relationship with each
_____3_____. A theory of such behavior is
concerned with forces, _____4_____, social,
and physical, which determine the course of this
_____5_____ and its outcomes in relation to the
culture in _____6_____ it occurs.
Information theory analysis
choices
freq
rel freq
I
one
10
0.33
1.584963
-0.53
two
6
0.20
2.321928
-0.46
three
3
0.10
3.321928
-0.33
four
3
0.10
3.321928
-0.33
five
1
0.03
4.906891
-0.16
six
1
0.03
4.906891
-0.16
seven
1
0.03
4.906891
-0.16
eight
1
0.03
4.906891
-0.16
nine
1
0.03
4.906891
-0.16
ten
1
0.03
4.906891
-0.16
eleven
1
0.03
4.906891
-0.16
twelve
1
0.03
4.906891
-0.16
30
1.00
H= 2.97
Ring a-round the roses,
A pocket full of posies,
_____1_____! Ashes!
We all fall down!
Three blind mice,
See how they run!
They all _____2_____ after a farmer's wife,
Who cut off their tails _____3_____ a carving knife.
Did you ever see such a ____4______ in your life,
As three blind mice?
Brian está en el aeropuerto de Barajas en
Madrid. _____1_____ y otros estudiantes del
drupo esperan la llegada del _____2_____ para
ir a Leób. Deben esperar una hora. ¿Qué
____3______ hacer?
1 Alice
2 vuelo
3 deciden
Samuel Becker (1968)
We construct messages which "are, in effect, overlayed to
form the large and complex communication environment or
'mosaic' in which each of us exists. This mosaic consists of an
immense number of fragments or bits of information on an
immense number of topics. . . . These bits are scattered over
time and space and modes of communication. Each individual
must grasp from this mosaic those bits which serve his needs,
must group them into message sets which are relevant for him
at any given time, and within each message set must organize
the bits and close the gaps between them in order to arrive at
a coherent picture of the world to which he can respond."