Transcript 幻灯片 1
Image Quilting for Texture Synthesis and Transfer Alexei A. Efros (UC Berkeley) William T. Freeman (MERL) Siggraph01’ About author? Alexei A. Efros Assistant Professor Computer Science Department & The Robotics Institute School of Computer Science Carnegie Mellon University From St. Petersburg, Russia Got PhD from UC Berkeley in 2003 Then a year as a Visiting Research Fellow in Visual Geometry Group of Oxford Joined in CSD and RI in autumn of 2004 Computer graphics & computer vision About author? William T. Freeman Professor of Electrical Engineering & Computer Science at the Artificial Intelligence Laboratory at MIT(September, 2001) Received a BS in physics and MS in electrical engineering from Stanford (1979), and an MS in applied physics from Cornell(1981) Got his PhD in 1992 from the MIT Worked at Mitsubishi Electric Research Labs (1992 – 2001, Cambridge) Computer vision Quilting? Transfer? + = Image vs. Texture Example-based Texture Synthesis Input Example The Goal of Texture Synthesis input image SYNTHESIS True (infinite) texture generated image Same in perceptual sense The Challenge Texture analysis: how to capture the essence of texture? Need to model the whole spectrum: from repeated to stochastic texture Repeated Stochastic Both? Related Work Local region-growing method -Pixel-based -Patch-based Global optimization-based method Heeger and Bergen sig95,Pyramid-based texture synthesis Paget and Longstaff IEEE Tran… 98,Texture synthesis via a noncausal nonparametric multiscale markov random field ….. Physical Simulation et al block Input texture B1 B2 Random placement of blocks B1 B2 Neighboring blocks constrained by overlap B1 B2 Minimal error boundary cut Minimum Error Boundary Cut overlapping blocks _ vertical boundary 2 = overlap error min. error boundary Minimum Error Boundary Cut e B B ov 1 ov 2 2 E Eij Eij eij min Ei 1, j 1 , Ei 1, j , Ei 1, j 1 The Image Quilting Algorithm – Pick size of block and size of overlap – Synthesize blocks in raster order – Search input texture for block that satisfies overlap constraints (above and left) • Easy to optimize using NN search [Liang et.al., ’01] – Paste new block into resulting texture • Compute minimal error boundary cut Synthesis Results Synthesis Results Synthesis Results Synthesis Results Synthesis Results Synthesis Results Portilla & Simoncelli Xu, Guo & Shum input image Wei & Levoy Image Quilting Portilla & Simoncelli Xu, Guo & Shum input image Wei & Levoy Image Quilting Portilla & Simoncelli Xu, Guo & Shum input image Wei & Levoy Image Quilting Failures Image Quilting vs. Graph Cut Input Image Quilting Graph Cut (siggraph 03’) Texture Transfer + Luminance Constraint parmesan + = rice + = Conclusion – No multi-scale, no one-pixel-at-a-time! – fast and very simple – Improved stability – Results are not bad Happy New Year! Pixel-based Methods Compare local causal neighbourhoods Efros and Leung (ICCV ’99) Wei and Levoy (Siggraph 2000) Ashikhmin (I3D 2001) Input Output Patch-based Methods Copy patches of pixels rather than single pixels Chaos Mosaic, Xu et al, 1997 Patch-Based Sampling, Liang et al(ACM 2001) Image Quilting, Efros and Williams(Siggraph 2001)