MATch: Differentiable Material Graphs for
Procedural Material Capture

SIGGRAPH Asia 2020

Liang Shi1     Beichen Li1     Miloš Hašan2     Kalyan Sunkavalli2    
Tamy Boubekeur3     Radomír Měch2     Wojciech Matusik1
    
1Massachusetts Institute of Technology      2Adobe Research      33D&I Research, Adobe

Every material in this rendered scene is a procedural material that was automatically created by MATch from a single flash photograph captured with a cellphone. All the target materials are visualized underneath.


Abstract


We present MATch, a method to automatically convert photographs of material samples into production-grade procedural material models. At the core of MATch is a new library DiffMat that provides differentiable building blocks for constructing procedural materials, which can be used to automatically translate large-scale procedural models, with hundreds to thousands of node parameters, into differentiable node graphs. Combining these translated node graphs with a rendering layer yields an end-to-end differentiable pipeline that maps node graph parameters to rendered images. This facilitates the use of gradient-based optimization to estimate the parameters such that the resulting material, when rendered, matches the target image appearance, as quantified by a style transfer loss. In addition, we propose a deep neural feature-based graph selection and parameter initialization method that efficiently scales to a large number of procedural graphs. We evaluate our method on both rendered synthetic materials and real materials captured as flash photographs. We demonstrate that MATch can reconstruct more accurate, general, and complex procedural materials compared to the state-of-the-art. Moreover, by producing a procedural output, we unlock capabilities such as constructing arbitrary-resolution material maps and parametrically editing the material appearance.


Paper


MATch: Differentiable Material Graphs for Procedural Material Capture
Liang Shi, Beichen Li, Miloš Hašan, Kalyan Sunkavalli, Tamy Boubekeur, Radomír Měch, Wojciech Matusik
ACM Transactions on Graphics 39(6) (Proc. SIGGRAPH Asia 2020)
[Paper]  [Supp]  [Code (coming soon)]  [BibTeX]


Video


Fast Forward


Overview Video


Application Video



High Resolution Procedural Texture


Click to view in full screen