Transcript Lecture11_Multimedia..

Lecture 11
Representing Multimedia Digitally
Digitizing Data
• Text is easy
• What about multimedia?
– Photos, audio, and video
• Same principles
Color and the Mystery of Light
• Color image
– Grid of pixels
• Pixel is formed from three primary colors
– RGB
Showing Colors
• Colors formed by using 3 intensities of
primaries
– Full intensity red, green, or blue
– Full intensity of red, green, and blue?
– No intensity of any color?
– Other combinations
LCD Display Technology
• Close-up of white
arrow pointer
– Note subpixels
Black and White Colors
• Intensity of light color stored in byte
• Needs 3 bytes/pixel
– Smallest intensity is 0000 0000
• Decimal?
– Largest value is 1111 1111
• Decimal?
Black and White Colors
• Black is absence of light:
– 0000 0000 0000 0000 0000 0000
• White is full intensity of each color:
– 1111 1111 1111 1111 1111 1111
Color Intensities
• Consider blue (0000 0000 0000 0000
• 8 bits have position values
1111 1111)
128
64
32
16
8
4
2
1
1
1
1
1
1
1
1
1
• To cut intensity in half
128
64
32
16
8
4
2
1
1
0
0
0
0
0
0
0
Color Intensities
Decimal to Binary
• How to convert decimal to binary?
– Look for powers of 2 and subtract
• E.g.: Convert 365 to binary
Lighten Up: Changing Color by
Addition
• What color does this represent?
1100 1000 1100 1000 1100 1000
• It’s RGB (200, 200, 200), a grey
– #C8 C8 C8 in hex
– All grays of form (x, x, x)
To Increase Intensity: Add in
Binary
• Increase common value to lighten
– E.g., add 0001 0000 (decimal 16) to each color
1101 1000 1101 1000 1101 1000
RGB (216,216,216)
Lightening
Adding another 16…
1101 1000
+ 0001 0000
-----------------
Check using decimal!
Lighter Still: Adding with Carry Digits
• Binary addition is similar to decimal addition
– Work from right to left
– 0 + 0 = 0, 0 + 1 = 1, 1 + 0 = 1
• No carry
– 1+1=0
• Carry of 1
Binary Addition
Binary Addition
Binary Addition
Computing on Representations
• Example: changing
the brightness and
contrast of a photo
Brightness and Contrast
• Brightness
– How close to white
• Contrast
– Difference b/w darkest and lightest portions of
image
• Photo manipulation software often gives
values of pixels in a Levels graph
Levels Graph
• 0 percent is black
point (0, 0, 0)
• 100 percent is white
point (ff, ff, ff)
• Midpoint of pixel
range is gamma point
Brightness
• Shift pixels closer to
white
• Add 16 to each pixel
• E.g.:
(197, 197, 197) =>
(213, 213, 213)
Contrast
• Scale pixel range
– Stretch toward right
• Add to each pixel, but
– add a smaller amount for
dark pixels
– add a larger amount for
light pixels
New Levels Graph
Adding Color
• Color => (x, y, z), all 3 differ
• Example
– Colorize image of moon
Making the Moon Orange
• Tint white regions
– Pick a shade of orange, say (255,213,132)
• Tint light gray
– Red byte: leave unchanged
– Green byte: reduce green slightly (subtract 42)
– Blue byte: reduce blue significantly (subtract 123)
Digitizing Sound
• Vibrating object creates sound
• Vibrations “push” air to form pressure
wave
• Wave vibrates our eardrums
Digitizing Sound
• Intensity of push
determines volume
• Frequency (# of
waves per second) of
pushes determines
pitch
continuous (analog)
representation of the wave
Analog to Digital
• Need to digitize to bits
• Use binary # for amplitude of wave
• At what point do you measure?
– Infinitely many possible
Analog to Digital
• Sample at regular
intervals
• Samples/second is
sampling rate
Nyquist Rule for Sampling
• Sampling rate is key
• Nyquist rule
– Sampling rate must 2x highest frequency
– Range of human hearing 20 Hz – 20 KHz
• Digital audio sampling rate is 44.1 KHz
Digitizing Process
Digitizing Process
• Recording (digitizing) process
– Sound => mic
– Signal sampled by ADC
• Samples encoded in binary
Digitizing Process
• Playing process
– Numbers read by DAC
– Electrical wave created by interpolation
– Electrical signal => speaker
How Many Bits per Sample?
• Perfect accuracy requires unlimited
bits/sample
• Must handle both +/- values
• More bits => more accuracy
How Many Bits per Sample?
• More bits yields a more
accurate digitization
• Digital audio uses 16 bits
Advantages of Digital Sound
• Key advantage is ability to compute on
representation
– Remove noise
– Compression
• Lossless
• Lossy
– Frequencies outside our range
Advantages of Digital Sound
• MP3 format
– Allows for compression ratio > 10:1
• Another key advantage of digital
representations is exact reproduction
Digital Images and Video
• Image is grid of RGB pixels
• Stored as linear sequence
• Can take up a lot of space
Digital Images and Video
• Example
– 8 × 10 image scanned at 300 ppi
– How many bytes to store?
– 8 𝑖𝑛 × 10𝑖𝑛 ×
3002 𝑝𝑥
1 𝑖𝑛2
×
3B
1 𝑝𝑥
= 21.6 𝑀𝐵
– Sending across 56 Kb/s phone connection
requires how many minutes?
Image Compression
• Typical monitor has fewer than 100 ppi
– 9x space saving over 300 ppi
• Still requires more than 5.5 min to send
Image Compression
• Compression changes rep to use fewer
bits
– Example: faxes
• Faxes are a sequences of 0’s and 1’s
• Use run-length encoding
Compression
• Run-length encoding is “lossless”
• Opposite is “lossy compression”
Compression
• MP-3
– Lossy scheme
– Highs and lows lost
• JPG (or JPEG)
– Lossy scheme for images
– Exploits limits of human perception
• Luminance sensitivity
• Chrominance insensitivity
JPEG Compression
• JPEG is capable of 10:1 compression w/o
detectable loss of clarity
JPEG Compression
• Ratios higher than 10:1
– Smaller files
– Possible artifacts (pixelation)
MPEG Compression
• MPEG used for video
• Video is sequence of stills
– Each image/frame is not seen for long
MPEG Compression
• JPEG used to compress frames
• “Interframe coherency” is used
– MPEG compression only transmits “deltas”
b/w frames
– Results in significant compression
Optical Character Recognition
• OCR
– Converts images of characters to characters
– Used by U.S.P.S.
– Used in banking
– Used in automatic license-plate recognition
Virtual Reality
• VR
– Simulates real-world environment
– Digitizes 3D space, sound
– Uses head-mounted displays
– Training: surgeons, military
Haptic Devices
• Haptic devices
– Used in VR
– Input/output technology for sense of touch
– Gloves can apply forces
Latency
• Challenges with VR
– Latency
• Time it takes for info to be recv’d
– Bandwidth
• Info per unit time
– Fundamental limit: speed of light
Bits Are It!
• 4 bytes can represent different info
– Number
– 4 characters
– Color
– etc.
• Bias-Free Universal Medium Principle
– Bits can represent all discrete info
– Bits have no inherent meaning
Bits: Bias-Free
• What does this bit sequence represent?
0000 0000 1111 0001 0000 1000 0010 0000
In hex: 00 F1 08 20
• Depends on context
Bits are bits…
Summary
• RGB color
• Manipulating images
• Binary numbers and arithmetic
• Digitizing sound, images, video
– Compression
Summary
• OCR
• Virtual Reality
• Latency and Bandwidth
• Bias-Free Universal Medium Principle
Quiz
• How many colors can 9 bits represent?
• 1010 + 0110 = ?
• Convert C7 to binary