Texture Analysis and Synthesis

	Lazy Texture Selection Based on Active Learning Tian Xia, Qing Wu, Chun Chen, and Yizhou Yu The Visual Computer, 2010, PDF It imposes a great challenge to select desired textures and textured objects across both spatial and temporal domains with minimal user interaction. This paper presents a method for achieving this goal. With this method, the appearance of similar texture regions within an entire image or video can be simultaneously manipulated. The technique we developed applies the active learning methodology. The user only needs to label minimal initial training data and subsequent query data. An active learning algorithm uses these labeled data to obtain an initial classifier and iteratively improves it until its performance becomes satisfactory. A revised graph cut algorithm based on the trained classifier has also been developed to improve the spatial coherence of selected texture regions. A variety of operations, such as color editing, matting and texture cloning, can be applied to the selected textures to achieve interesting editing effects.
	Hierarchical Tensor Approximation of Multi-Dimensional Visual Data Qing Wu, Tian Xia, C. Chen, H.-Y. Lin, H. Wang and Yizhou Yu IEEE Transactions on Visualizationa and Computer Graphics, 2008, PDF Visual data comprise of multi-scale and inhomogeneous signals. In this paper, we exploit these characteristics and develop a compact data representation technique based on a hierarchical tensor-based transformation. In this technique, an original multi-dimensional dataset is transformed into a hierarchy of signals to expose its multi-scale structures. The signal at each level of the hierarchy is further divided into a number of smaller tensors to expose its spatially inhomogeneous structures. These smaller tensors are further transformed and pruned using a tensor approximation technique. Our hierarchical tensor approximation supports progressive transmission and partial decompression. Experimental results indicate that our technique can achieve higher compression ratios and quality than previous methods, including wavelet transforms, wavelet packet transforms, and single-level tensor approximation. We have successfully applied our technique to multiple tasks involving multi-dimensional visual data, including medical and scientific data visualization, data-driven rendering and texture synthesis.
	Laplacian Texture Synthesis and Mixing on Surfaces Qing Wu, Lin Shi, Stephen Bond, and Yizhou Yu Pacific Graphics 2006, PDF In neighborhood-based texture synthesis, adjacent local regions need to satisfy color continuity constraints in order to avoid visible seams. Such continuity constraints seriously restrict the variability of synthesized textures, making it impossible to generate new textures by mixing multiple input textures with very different base colors. In this paper, we propose to relax such restrictions and decompose synthesis into two relatively disjoint stages. In the first stage, an intermediate synthesized texture is generated by only considering the high frequency details during region search and matching. Such a scheme broadens the search space during texture synthesis, but may produce obvious seams due to large discontinuities in low frequency components. In the second stage, instead of performing local feathering along these discontinuities, we perform Laplacian texture reconstruction, which retains the high frequency details but computes new consistent low frequency components to eliminate the seams. It does not only affect texels close to the discontinuities, but also modifies the rest of the texels. Therefore, it can be viewed as a global feature-preserving smoothing step, and is more effective than local feathering. Experiments indicate that our two-stage synthesis can produce desired results for regular texture synthesis as well as texture mixing from multiple sources.
	Out-of-Core Tensor Approximation of Multi-Dimensional Matrices of Visual Data Hongcheng Wang, Qing Wu, Lin Shi, Yizhou Yu and Narendra Ahuja SIGGRAPH 2005, PDF Tensor approximation is necessary to obtain compact multilinear models for multi-dimensional visual datasets. Traditionally, each multi-dimensional data item is represented as a vector. Such a scheme flattens the data and partially destroys the internal structures established throughout the multiple dimensions. In this paper, we retain the original dimensionality of the data items to more effectively exploit existing spatial redundancy and allow more efficient computation. Since the size of visual datasets can easily exceed the memory capacity of a single machine, we also present an out-of-core algorithm for higher-order tensor approximation. The basic idea is to partition a tensor into smaller blocks and perform tensor-related operations blockwise. We have successfully applied our techniques to three graphics-related data-driven models, including 6D bidirectional texture functions, 7D dynamic BTFs and 4D volume simulation sequences. Experimental results indicate that our techniques can not only process out-of-core data, but also achieve higher compression ratios and quality than previous methods.
	Feature Matching and Deformation for Texture Synthesis Qing Wu and Yizhou Yu SIGGRAPH 2004, PDF One significant problem in patch-based texture synthesis is the presence of broken features at the boundary of adjacent patches. The reason is that optimization schemes for patch merging may fail when neighborhood search cannot find satisfactory candidates in the sample texture because of an inaccurate similarity measure. In this paper, we consider both curvilinear features and their deformation. We develop a novel algorithm to perform feature matching and alignment by measuring structural similarity. Our technique extracts a feature map from the sample texture, and produces both a new feature map and texture map. Texture synthesis guided by feature maps can significantly reduce the number of feature discontinuities and related artifacts, and gives rise to satisfactory results.
	Shadow Graphs and 3D Texture Reconstruction Yizhou Yu and Johnny Chang International Journal of Computer Vision, Vol. 62, No. 1-2, 2005, pp.35-60. PDF We present methods for recovering surface height fields such as geometric details of 3D textures by incorporating shadow constraints. We introduce shadow graphs which give a new graph-based representation for shadow constraints. It can be shown that the shadow graph alone is sufficient to solve the shape-from-shadow problem from a dense set of images. Shadow graphs provide a simpler and more systematic approach to represent and integrate shadow constraints from multiple images. To recover height fields from a sparse set of images, we propose a method for integrated shadow and shading constraints. Previous shape-from-shadow algorithms do not consider shading constraints while shape-from-shading usually assumes there is no shadow. Our method is based on collecting a set of images from a fixed viewpoint as a known light source changes its position. It first builds a shadow graph from shadow constraints from which an upper bound for each pixel can be derived if the height values of a small number of pixels are initialized correctly. Finally, a constrained optimization procedure is designed to make the results from shape-from-shading consistent with the height bounds derived from the shadow constraints. Our technique is demonstrated on both synthetic and real imagery.
	Pattern-Based Texture Metamorphosis Ziqiang Liu, Ce Liu, Harry Shum and Yizhou Yu Pacific Graphics 2002, PDF In this paper, we study texture metamorphosis, or how to generate texture samples that smoothly transform from a source texture image to a target. We propose a patternbased approach to specify the feature correspondence between two textures, based on the observation that many texture images have stochastically distributed patterns which are similar to each other. First, the user selects a pattern in the source and target textures, and establishes the "local feature correspondence" between these two patterns by specifying landmarks. Then, repeated patterns are automatically detected and localized in the source and target textures. The "pattern correspondence" between two textures is formulated as an integer programming problem and solved using the Hungarian algorithm. Finally, we obtain a warp function between two textures by combining "local feature correspondence" and "pattern correspondence". Experiments demonstrate that our technique produces visually appealing morphing sequences, with moderate amount of user interaction.
	Synthesizing Bidirectional Texture Functions for Real-World Surfaces Xinguo Liu, Yizhou Yu and Harry Shum SIGGRAPH 2001, PDF We present a novel approach to synthetically generating bidirectional texture functions (BTFs) of real-world surfaces. Unlike a conventional two-dimensional texture, a BTF is a six-dimensional function that describes the appearance of texture as a function of illumination and viewing directions. The BTF captures the appearance change caused by visible small-scale geometric details on surfaces. From a sparse set of images under different viewing/lighting settings, our approach generates BTFs in three steps. First, it recovers approximate 3D geometry of surface details using a shape-from-shading method. Then, it generates a novel version of the geometric details that has the same statistical properties as the sample surface with a non-parametric sampling method. Finally, it employs an appearance preserving procedure to synthesize novel images for the recovered or generated geometric details under various viewing/lighting settings, which then define a BTF. Our experimental results using the images from the CUReT database demonstrate the effectiveness of our approach.

Acknowledgment: the material on this webpage is based upon work partially supported by the National Science Foundation.