Transcript Slide 1
3 D Display: Current and future technologies in Europe Part 2: 3D Display Research at DMU Phil Surman Wing Kai Lee Imaging and Displays Research Group De Montfort University Leicester UK NANJING UNIVERSITY 18 MARCH 2007 Presentation 1) 2) 3) 4) Principle of operation of DMU display ATTEST multi-user 3D prototype MUTED 3D display project Future work NANJING UNIVERSITY 18 MARCH 2007 PRINCIPLE OF OPERATION of DMU Display NANJING UNIVERSITY 18 MARCH 2007 DMU Display Side mirror Head tracker Screen Exit pupils Viewers NANJING UNIVERSITY 18 MARCH 2007 Exit Pupils PLAN VIEWS A SCREEN R VIEWER B L EXIT PUPIL PAIR NANJING UNIVERSITY 18 MARCH 2007 MULTIPLE EXIT PUPILS C Exit Pupil Formation Illumination source Lens Screen Exit pupil LENS NANJING UNIVERSITY 18 MARCH 2007 Multiple Exit Pupil Formation with a Lens Illumination source C Illumination source B Illumination source A Lens and vertical diffuser Viewer A VIEWING FIELD Viewer C Exit pupils NANJING UNIVERSITY 18 MARCH 2007 Viewer B Exit Pupil Formation with Array Illumination sources Exit pupil NANJING UNIVERSITY 18 MARCH 2007 Exit Pupil Steering Illumination sources Steering array lenses NANJING UNIVERSITY 18 MARCH 2007 Exit pupil Exit Pupil Steering Steering array lenses Illumination sources Exit pupil NANJING UNIVERSITY 18 MARCH 2007 Exit Pupil Steering Illumination sources Steering array lenses NANJING UNIVERSITY 18 MARCH 2007 Exit pupil Coaxial Optical Element Illumınatıon surface Aperture Lıght contaıned wıthın element by total ınternal reflectıon Refractıng surface To screen • Illumination and refracting surfaces both cylindrical with common vertical axis • • • Aperture centred at axis No off-axis aberrations Light contained in element by total internal reflection NANJING UNIVERSITY 18 MARCH 2007 Collimated Beam Formation Off-axis Rays Axial Rays To exit pupil Virtual image of aperture Virtual image of aperture TOP VIEWS COAXIAL OPTICS BEAM FORMATION NANJING UNIVERSITY 18 MARCH 2007 To exit pupil Spatial Multiplexing MUX screen LCD L From right source R To exit pupils From left source NANJING UNIVERSITY 18 MARCH 2007 To right exit pupil To left exit pupil Spatial MUX with Parallax Barrier Parallax barrier Illumination sources LCD L R L R SIDE VIEW NANJING UNIVERSITY 18 MARCH 2007 Spatial MUX with Lenticular Screen Lenticular screen Illumination sources R L L R SIDE VIEW NANJING UNIVERSITY 18 MARCH 2007 First Prototype Screen assembly Upper mirror R L Light sources L Lower mirror R Exit pupils This prototype has fixed pupils – its purpose is to demonstrate spatial multiplexing NANJING UNIVERSITY 18 MARCH 2007 First Prototype SCREEN UPPER MIRROR LOWER MIRROR NANJING UNIVERSITY 18 MARCH 2007 LAMPS Early Work: Schematic Diagram from PhD IR camera Multiplexing screen (Ch.12) LCD (Ch.5) (Ch.6) Illumination Head tracking processor (Ch.12) source (Chs.9&10) Vertical diffuser (Ch.7) Fresnel lens (Ch.3) R Exit pupils (Ch.3) L Folding mirrors (Ch.4) Viewing field (Ch.8) Viewer (Ch.8) Retroreflector (Ch.12) FIG.1.1 SCHEMATIC DIAGRAM OF PROTOTYPE 3D DISPLAY NANJING UNIVERSITY 18 MARCH 2007 Early Work: Head Tracker LED array showing head position IR diodes and camera lens Head Retro-reflector FIG.12.1 HEAD TRACKING SET-UP NANJING UNIVERSITY 18 MARCH 2007 Early Work: Head Tracker Retroreflector Head (a) View from Camera Lens (b) Red LED Array FIG.12.2 HEAD ‘SHADOW’ NANJING UNIVERSITY 18 MARCH 2007 Region imaged by IR array Early Work: Moving Illumination Source Stepper motor Pinion Magnet Track Rack Wheel Wheel Right halogen aperture Left halogen aperture Reed switch FIG.10.2 HALOGEN LAMP ILLUMINATION ASSEMBLY NANJING UNIVERSITY 18 MARCH 2007 ATTEST PROTOTYPE CONSTRUCTION AND RESULTS NANJING UNIVERSITY 18 MARCH 2007 ATTEST: Array Element and Illumination/Driver Board LED array Driver board Soft Aperture Aperture To viewer • • Aperture printed on strip of film (RH figure) 2 aperture components cemented together with aperture in between NANJING UNIVERSITY 18 MARCH 2007 ATTEST: Array Element and Illumination/Driver Board • • This shows first version with 90 x 3mm white LEDs. Exit pupils move in large increments (~30mm) NANJING UNIVERSITY 18 MARCH 2007 ATTEST:LCD Diffraction ATTEST: Illumination/driver Board Version 1 LED DRIVERS 90 x 3mm WHITE LEDs LIGHT NANJING UNIVERSITY 18 MARCH 2007 ATTEST: Illumination/driver Board Version 2 • • 256 x 1 mm surface-mount white LEDs Comprises 16 x 16-element modules NANJING UNIVERSITY 18 MARCH 2007 ATTEST: LED Module DRIVER CHIP HEAT SINK WHITE LED & LENS ARRAY MICROLENS ARRAY LIGHT LIGHT • • HEAT SINK DRIVER CHIP 16 x 1 mm surface-mount white LEDs Integral driver and heat sink NANJING UNIVERSITY 18 MARCH 2007 ATTEST: Illumination Sources • • This shows collimated beams formed in different directions Beam width can be increased by lighting more LEDs NANJING UNIVERSITY 18 MARCH 2007 ATTEST: Multiple Exit Beams • Multiple beams formed by lighting several sets of adjacent LEDs NANJING UNIVERSITY 18 MARCH 2007 ATTEST: DemonstratorArray • Constructed for demonstration of multiple exit pupil formation but without use of LCD NANJING UNIVERSITY 18 MARCH 2007 ATTEST: Demonstrator Exit Pupils • • Beams formed on targets. Polhemus electromagnetic tracker pickups located at targets NANJING UNIVERSITY 18 MARCH 2007 ATTEST: Array Configuration Lower layer PLAN VIEW Illumination surfaces Upper layer Upper layer Aperture Aperture Refracting surfaces • • One ten-element array is used for each of the left and right sets of exit pupils comprises two sets of five staggered elements NANJING UNIVERSITY 18 MARCH 2007 ATTEST: Appearance of Front of Array WI WS Continuous illumination over this width • • Aperture images are effective LCD backlight Vertical diffuser required to enable aperture images to illuminate full LCD height NANJING UNIVERSITY 18 MARCH 2007 ATTEST: Soft Apertures A&B A B B&C B C SOFT APERTURES (VIEWED FROM FRONT) • Soft apertures allow for constructional errors and aperture image width variation • Fading width determined from trials on perception of brightness variation NANJING UNIVERSITY 18 MARCH 2007 ATTEST: Aperture Intensity Variation (a) Appearance of aperture images Relative intensity (b) Intensity variation Distance across array NANJING UNIVERSITY 18 MARCH 2007 ATTEST: Folding Mirrors PLAN VIEWS Steering optics Virtual image Steering optics Virtual image (a) Without Folding • • Mirror Mirror (b) With Folding Virtual arrays formed either side of actual array Reduces housing size NANJING UNIVERSITY 18 MARCH 2007 ATTEST: Folding Multiplexing screen, LCD and vertical diffuser Folding mirror Steering optics Light path Folding mirrors • 5 Mirror folding enables same housing size as current rear projected displays (side mirrors not shown) NANJING UNIVERSITY 18 MARCH 2007 ATTEST: Prototype Side mirror Optical array Screen Side mirror NANJING UNIVERSITY 18 MARCH 2007 ATTEST: Plan view of Prototype Optical array TOP VIEW Side mirror Screen Side mirror NANJING UNIVERSITY 18 MARCH 2007 ATTEST: Prototype Side Elevation SIDE VIEW Side mirror Optical array Screen NANJING UNIVERSITY 18 MARCH 2007 ATTEST: Prototype STEERING ARRAY FOLDING MIRROR • • SCREEN ASSY. Incorporates same large optical elements as used in demonstrator Large cylindrical convex in front of LCD to increase brightness NANJING UNIVERSITY 18 MARCH 2007 ATTEST: Display Sub-pixels • 15µM structure within RGB sub-pixels RGB Sub-pixels • 100 milliradians Diffraction NANJING UNIVERSITY 18 MARCH 2007 Very high first-order component ATTEST: LCD Diffraction 1.0 INTENSITY RELATIVE Relative Intensity %(%) 100 HORIZONTAL PLOT 90 VERTICAL PLOT 80 70 60 0.5 50 40 30 20 10 0 0 0 50 100 150 200 -100 250 0 Distance(mm) / mm DISTANCE • large first order gives ~ 15% crosstalk NANJING UNIVERSITY 18 MARCH 2007 • Vertical diffraction << horizontal diffraction 100 ATTEST: Exit Pupil Profile L R 110 Scattering Diffraction INTENSITY RELATIVE Relative Intensity %(%) 100 90 • 80 70 60 • 50 40 • 30 20 10 0 0 50 100 150 200 250 300 350 Distance(mm) / mm DISTANCE • • NANJING UNIVERSITY 18 MARCH 2007 Maxima produced by use of discrete components Left eye located at position L Right eye located at position R Profile is convolution of aperture function with diffraction function (PSF) ATTEST: White LED Colour Variation • 0.5 Typical white LED variation • • Y 0 0 0.5 X NANJING UNIVERSITY 18 MARCH 2007 Blue region shows total variation from manufacturer This region divided into four Even with LEDs from one batch, variation still large ATTEST: Further Work Identified • • • • • • Use LCD with suitable sub-pixel structure to minimise diffraction Select appropriate material and manufacturing process to minimise scattering Use single illumination source to illuminate colour and brightness variation Use low etendue illumination source to reduce light loss Reduce housing size - consumer preference is for ‘hang-on-wall’ Develop multi-user non-intrusive head tracker NANJING UNIVERSITY 18 MARCH 2007 MUTED Multi-user Threedimensional Television Display NANJING UNIVERSITY 18 MARCH 2007 MUTED: Brief Summary • • • • • EU-funded Kicked-off July 2006 30 months duration 30 person years of effort 7 partners including SLE and Fraunhofer HHI NANJING UNIVERSITY 18 MARCH 2007 MUTED: Technical Summary • • • • • • • • RBG laser illumination source Provides wide colour gamut Holographic projector-controlled exit pupils Developing multi-user non-intrusive head tracker Human factors issues examined Investigation into low-diffraction LCD Investigation into temporal MUX Exploitation of display in medical applications NANJING UNIVERSITY 18 MARCH 2007 MUTED: Semi-coaxial Array Illumination Plane • • TOP VIEW Array elements have flat back surface – hence semi-coaxial Enables other means of illumination, for example projection NANJING UNIVERSITY 18 MARCH 2007 MUTED: Optical Array Light from projector Light to screen assy. 10CM SECTION OF ARRAY NANJING UNIVERSITY 18 MARCH 2007 MUTED: Illumination Plane VIEWER A VIEWER B • • Each exit pupil position can be mapped to a diagonal series of small sources Slope of diagonal determines exit pupil distance and lateral position the x-coordinate NANJING UNIVERSITY 18 MARCH 2007 LATERAL POSITION DISTANCE EXIT PUPIL POSITION MUTED: Optical Array X X Y Y ILLUMINATION PLANE HOLOGRAPHIC PROJECTION • Conventional projection blocks ~ of 95% of light • Use of CGH projector utilises complete wavefront on LCOS SLM • Binary phase hologram gives around 40% efficiency • Investigating use of conjugate image to double efficiency NANJING UNIVERSITY 18 MARCH 2007 MUTED: Schematic Diagram RGB LASER HEAD TRACKER LCOS LCD OPTICAL ARRAY MOBILE VIEWERS SIMPLIFIED SCHEMATIC DIAGRAM OF DISPLAY NANJING UNIVERSITY 18 MARCH 2007 MUTED: Current Status • Investigation into aperture-less optical elements for simplified construction • Refining LCOS algorithms • Measurement of suitable LCD panels wrt to speed and diffraction • Deciding multi-user tracker route • Low power monochromatic version under construction NANJING UNIVERSITY 18 MARCH 2007 MUTED: Enabling Technologies Fixed – non HT LCD OLED View-direct ing screen 2D/3D switching Single user HT Robust single-user HT Horizont al pixel LCD Multiuser HT Mult i-user HT St eering backlight unit Suit able LC panel (horiz. pixels) RGB laser Light valve Suit able SLM Multi-view Holoform Volumetric High-resolution LCDs (<20m) High-resolut ion OLED (<20m) Mult i-view video projector Direction select ive screen Multiple view rendering Very high res. OLED (~1m) Very fast LV (>10 4 FP S) Large (>1m x .6m) FE barrier Large (>1m x .6m) HOE Multi-layer LC screen (>50 layers) Very fast LV (>10 4 FP S) Voxel opacity NANJING UNIVERSITY 18 MARCH 2007 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 Enabling technolog y MUTED completed Binocular Ge neric dis play type 2006 Table 2 Time Scales for Development of Principal Enabling Technologies. available unavailable MUTED MUTED: Display Performance No. of viewers Viewer movement Motion parallax Acc./con. rivalry Image transparency Fixed – non HT Single user HT* Single No Yes No Single Very limited Adequate Possible Yes No Multi-user HT* Multiple Large Possible Yes No Multi-view Multiple Reasonable Yes Yes No Holoform Multiple Large Yes No No Volumetric Multiple Large Yes No Yes Holographic Multiple Large Yes No No Display type Binocular MUTED NANJING UNIVERSITY 18 MARCH 2007 FUTURE WORK NANJING UNIVERSITY 18 MARCH 2007 Future Work • European Union Framework 7 round of funding started in December 2006 • First Call closes 8th May 2007 • Multi-user 3D displays included in call • Also high colour gamut • Interactivity supported NANJING UNIVERSITY 18 MARCH 2007 Extract from EU Workplan Advanced visualisation systems and novel display technologies. Visualisation systems extending colour gamut and dynamic range beyond current state-of-theart, taking into account human vision and perceptual models. They should support multiviewer, unaided and unrestricted 3D viewing, as well as natural interaction modalities. This includes signal acquisition, processing and representation technologies for 3D-systems. NANJING UNIVERSITY 18 MARCH 2007 High Efficiency Laser-based Multiuser Multi-modal 3D Display (HELIUM3D) • Direct-view laser-based 3D display to be developed • Does not require LCD • Image information supplied by light valve • Illumination source is RGB lasers. • High colour gamut • Direct-view • Does not have light attenuation of LCD - energy efficient • Frees reliance on LCD fabrication plants NANJING UNIVERSITY 18 MARCH 2007 HELIUM 3D: Schematic Diagram Light valve RGB Laser Head tracker MEMS scanner SLM Screen Mobile viewers NANJING UNIVERSITY 18 MARCH 2007 HELIUM 3D: Colour Gamut NANJING UNIVERSITY 18 MARCH 2007 HELIUM 3D: Display Functionality • Display is functionally scalable • Fast light valve speed could enable a different • • • • image to be seen by each eye in viewing field Enables motion parallax Each viewer could choose their desired viewpoint if scene captured by a camera array Each viewer could see completely different images to other viewers Display will work in near field and far field modes NANJING UNIVERSITY 18 MARCH 2007 HELIUM 3D: Near Field Operation • Screen around 1 – 1.5 metre from viewer • Immerse hands into image – therefore image ~ 0.5 m from user • 1 or 2 users, single or collaborative working • Large disparities – up to I/O distance • Large convergence/accommodation rivalry (human factors work necessary) NANJING UNIVERSITY 18 MARCH 2007 HELIUM 3D: Near Field Tracking • Requires low tracker latency – high latency will affect task performance and could cause nausea • Requires high tracker accuracy (more than for just locating exit pupils) • Head tracking in x, y and z directions • Images rendered in accordance with head coordinates NANJING UNIVERSITY 18 MARCH 2007 HELIUM 3D: Near Field Example ‘Virtual Clay concept’ - ‘clay’ shaped with naked hands • • • • A virtual chunk of clay floats in front of screen Touching and shaping the ‘clay’ with the naked hands enables user to directly manipulate object Approach completely differs from existing techniques perceptual space matches interaction space This technique potentially useful in medical task applications NANJING UNIVERSITY 18 MARCH 2007 HELIUM 3D: Far Field Operation Viewing distance around 2 – 4 metres • • • Gaming Television Videoconferencing ……… NANJING UNIVERSITY 18 MARCH 2007