Transcript ppt
Recap from Friday Pinhole camera model Perspective projections Lenses and their flaws Focus Depth of field Focal length and field of view What is wrong with this picture? What is wrong with this picture? Capturing Light… in man and machine Many slides by Alexei A. Efros CS 195g: Computational Photography James Hays, Brown, Spring 2010 Image Formation Digital Camera Film The Eye Digital camera A digital camera replaces film with a sensor array • • • Each cell in the array is light-sensitive diode that converts photons to electrons Two common types – Charge Coupled Device (CCD) – CMOS http://electronics.howstuffworks.com/digital-camera.htm Slide by Steve Seitz Sensor Array CMOS sensor Sampling and Quantization Interlace vs. progressive scan http://www.axis.com/products/video/camera/progressive_scan.htm Slide by Steve Seitz Progressive scan http://www.axis.com/products/video/camera/progressive_scan.htm Slide by Steve Seitz Interlace http://www.axis.com/products/video/camera/progressive_scan.htm Slide by Steve Seitz The Eye The human eye is a camera! • Iris - colored annulus with radial muscles • Pupil - the hole (aperture) whose size is controlled by the iris • What’s the “film”? – photoreceptor cells (rods and cones) in the retina Slide by Steve Seitz The Retina Cross-section of eye Cross section of retina Pigmented epithelium Ganglion axons Ganglion cell layer Bipolar cell layer Receptor layer Retina up-close Light Two types of light-sensitive receptors Cones cone-shaped less sensitive operate in high light color vision Rods rod-shaped highly sensitive operate at night gray-scale vision © Stephen E. Palmer, 2002 Rod / Cone sensitivity The famous sock-matching problem… Distribution of Rods and Cones # Receptors/mm2 . Fovea 150,000 Rods Blind Spot Rods 100,000 50,000 0 Cones Cones 80 60 40 20 0 20 40 60 80 Visual Angle (degrees from fovea) Night Sky: why are there more stars off-center? © Stephen E. Palmer, 2002 Eye Movements Saccades Microsaccades Ocular microtremor (OMT) Electromagnetic Spectrum Human Luminance Sensitivity Function http://www.yorku.ca/eye/photopik.htm Visible Light Why do we see light of these wavelengths? …because that’s where the Sun radiates EM energy © Stephen E. Palmer, 2002 The Physics of Light Any patch of light can be completely described physically by its spectrum: the number of photons (per time unit) at each wavelength 400 - 700 nm. # Photons (per ms.) 400 500 600 700 Wavelength (nm.) © Stephen E. Palmer, 2002 The Physics of Light Some examples of the spectra of light sources . B. Gallium Phosphide Crystal # Photons # Photons A. Ruby Laser 400 500 600 700 400 500 Wavelength (nm.) 700 Wavelength (nm.) D. Normal Daylight # Photons C. Tungsten Lightbulb # Photons 600 400 500 600 700 400 500 600 700 © Stephen E. Palmer, 2002 The Physics of Light % Photons Reflected Some examples of the reflectance spectra of surfaces Red 400 Yellow 700 400 Blue 700 400 Wavelength (nm) Purple 700 400 700 © Stephen E. Palmer, 2002 The Psychophysical Correspondence There is no simple functional description for the perceived color of all lights under all viewing conditions, but …... A helpful constraint: Consider only physical spectra with normal distributions mean area # Photons 400 500 variance 600 700 Wavelength (nm.) © Stephen E. Palmer, 2002 The Psychophysical Correspondence # Photons Mean blue Hue green yellow Wavelength © Stephen E. Palmer, 2002 The Psychophysical Correspondence # Photons Variance Saturation hi. high med. medium low low Wavelength © Stephen E. Palmer, 2002 The Psychophysical Correspondence Area Brightness # Photons B. Area Lightness bright dark Wavelength © Stephen E. Palmer, 2002 Physiology of Color Vision Three kinds of cones: 440 RELATIVE ABSORBANCE (%) . 530 560 nm. 100 S M L 50 400 450 500 550 600 650 WAVELENGTH (nm.) • Why are M and L cones so close? • Why are there 3? © Stephen E. Palmer, 2002 More Spectra metamers Color Sensing in Camera (RGB) 3-chip vs. 1-chip: quality vs. cost Why more green? Why 3 colors? http://www.cooldictionary.com/words/Bayer-filter.wikipedia Slide by Steve Seitz Practical Color Sensing: Bayer Grid Estimate RGB at ‘G’ cells from neighboring values http://www.cooldictionary.com/ words/Bayer-filter.wikipedia Slide by Steve Seitz RGB color space RGB cube • • • • Easy for devices But not perceptual Where do the grays live? Where is hue and saturation? Slide by Steve Seitz HSV Hue, Saturation, Value (Intensity) • RGB cube on its vertex Decouples the three components (a bit) Use rgb2hsv() and hsv2rgb() in Matlab Slide by Steve Seitz Project #1 • How to compare R,G,B channels? • No right answer • Sum of Squared Differences (SSD): • Normalized Correlation (NCC): Image half-sizing This image is too big to fit on the screen. How can we reduce it? How to generate a halfsized version? Image sub-sampling 1/8 1/4 Throw away every other row and column to create a 1/2 size image - called image sub-sampling Slide by Steve Seitz Image sub-sampling 1/2 1/4 (2x zoom) 1/8 (4x zoom) Aliasing! What do we do? Slide by Steve Seitz Gaussian (lowpass) pre-filtering G 1/8 G 1/4 Gaussian 1/2 Solution: filter the image, then subsample • Filter size should double for each ½ size reduction. Why? Slide by Steve Seitz Subsampling with Gaussian pre-filtering Gaussian 1/2 G 1/4 G 1/8 Slide by Steve Seitz Compare with... 1/2 1/4 (2x zoom) 1/8 (4x zoom) Slide by Steve Seitz Gaussian (lowpass) pre-filtering G 1/8 G 1/4 Gaussian 1/2 Solution: filter the image, then subsample • Filter size should double for each ½ size reduction. Why? Slide by Steve Seitz • How can we speed this up? Image Pyramids Known as a Gaussian Pyramid [Burt and Adelson, 1983] • In computer graphics, a mip map [Williams, 1983] • A precursor to wavelet transform Slide by Steve Seitz A bar in the big images is a hair on the zebra’s nose; in smaller images, a stripe; in the smallest, the animal’s nose Figure from David Forsyth What are they good for? Improve Search • Search over translations – Like project 1 – Classic coarse-to-fine strategy • Search over scale – Template matching – E.g. find a face at different scales Pre-computation • Need to access image at different blur levels • Useful for texture mapping at different resolutions (called mip-mapping) Gaussian pyramid construction filter mask Repeat • Filter • Subsample Until minimum resolution reached • can specify desired number of levels (e.g., 3-level pyramid) The whole pyramid is only 4/3 the size of the original image! Slide by Steve Seitz