Codes for Photometric Stereo

On this webpage you will find some codes in Matlab for photometric stereo. All these codes were developed in the VORTEX team of the IRIT lab, a lab of the Université de Toulouse in Toulouse, France.

To know more about each method, please refer to the corresponding publication.

For any questions/suggestions, feel free to send an email to: yvain \dot queau \at enseeiht \dot fr (webpage)

List of codes

Color Photometric stereo

Directional color PS

These scripts show how to solve the PS problem with RGB data through a linear PDE approach when the lightings are directional. The archive contain:

  • Two datasets: one Lambertian (banknote) and one with strong inter-reflection effects (coin). Both are provided along with the calibrated lighting directions and camera parameters (projection can also be reasonably approximated by orthographic model)
  • A least-squares depth estimaton from PS function
  • Two demo scripts for using the function

Source Code

The ".m" files, along with demo scripts and datasets, can be found here.

References

Unbiased Photometric Stereo for Colored Surfaces: a Variational Approach
Quéau, Y. and Mecca, R. and Durou, J.-D.
Computer Vision and Pattern Recognition (CVPR 2016)
Paper (pdf)

Pointwise color PS

These scripts show how to solve the PS problem with RGB data through a linear PDE approach when the lightings are pointwise and the camera is perspective. The archive contain:

  • One dataset: 8 images of a Buddha statue under nearby lightings produced by calibrated (positions and intensities) LEDs.
  • New dataset in V2 archive: 8 images of a plastic mask with important color changes and specularities.
  • New calibration of lights in V2 (quasi Lambertian pointwise model). The data contains the anisotropy coefficient of the sources, their orientations, RGB intensities and positions. They were calibrated using the method from our RFIA 16 paper.
  • A least-squares depth estimaton from PS function, able to handle the non-linear lighting model by fixed point iterations.
  • More stable version of the code in V2 (slightly changed the fixed point iterations so that regularization is no longer required)
  • One demo script for using the function

Source Code

The ".m" files of the V2, along with demo scripts and datasets, can be found here. Older version (V1) still available here.

References

Unbiased Photometric Stereo for Colored Surfaces: a Variational Approach
Quéau, Y. and Mecca, R. and Durou, J.-D.
Computer Vision and Pattern Recognition (CVPR 2016)
Paper (pdf)

Normal Field Integration

Variational fusion of rough depth map with high precision gradient estimation

These scripts show how to simply merge rough depth estimation with high precision gradient estimate, obtained for instance with using photometric stereo. The archive contains:

  • A script to perform photometric stereo to get surface normals
  • Least-squares integration of surface gradient on non-rectangular domain
  • Integration with sparse control points
  • Integration with dense depth prior

Source Code

The ".m" files, along with demo scripts and datasets, can be found here.

References

Edge-Preserving Integration of a Normal Field: Weighted Least-squares, TV and L1 Approaches
Quéau, Y. and Durou, J.-D.
Scale-Space and Variational Method in computer vision (SSVM 2015)
Paper (pdf)
Reconstruction tridimensionnelle par stéréophotométrie
Quéau, Y.
Ph.D thesis - chapter 2
Thesis (pdf)

Datasets

Here are some datasets, in Matlab format (.mat), which can be used for testing algorithms of shape-from-gradient, i.e. to recover the depth map from an estimation of its gradient, estimated from example by photometric stereo.

Real datasets include the photometric stereo images, a mask, and an estimation of the depth map. Synthethic datasets include a ground truth depth function and its gradient.

Edge-preserving Integration on Arbitrary Domains (WLS, TV and L1)

These are the sample codes associated to the three discrete functionals presented in our SSVM 2015 paper

  • Weighted least squares functionals can be used when it is easy to detect discontinuities through integrability evaluation
  • Isotropic TV can be minimised using reweighted least-squares
  • Anisotropic TV (L1 norm) can be minimised through split-Bregman iterations
  • Iterative integration provides a detail-preserving solution for surface edition

Demo of L1 (split-Bregman) iterations

Source Code

The ".m" files, along with demo scripts and datasets, can be found here.

Archives for each method can be downloaded separately:

Reference

Edge-Preserving Integration of a Normal Field: Weighted Least-squares, TV and L1 Approaches
Quéau, Y. and Durou, J.-D.
Scale-Space and Variational Method in computer vision (SSVM 2015)
Paper (pdf)

Fast Weighted Least Square Poisson Solver on Arbitrary connex Domains, with natural boundary condition

To reconstruct a depth map from an estimation of its gradient, a widely used method is to turn the problem into the resolution of Poisson's equation. When the domain is non-rectangular, FFT cannot be used. However, the problem remains linear, and the matrix considered in the system is very sparse. Thus, direct sparse solvers can be used. In this code, we perform 3D-reconstruction on arbitrary (connex) domains, along with the natural boundary condition, by using sparse matrix factorization. Furthermore, the original least square problem can be appropriately weighted, so as to exhibit depth discontinuities which are usually smoothed by the least square solution (see example in the archive). This method includes as a particular case the least square solution on rectangular or arbitrary domains.

Source Code

The ".m" file, along with a demo script and three datasets, can be found here.

Reference

Intégration d'un champ de gradient rapide et robuste aux discontinuités - Application à la stéréophotométrie
Quéau, Y. and Durou, J.-D.
Reconnaissance de Formes et Intelligence Artificielle (RFIA 2014)
Paper (pdf, in French)

Fast Least Square Poisson Solvers on Rectangular Domains

When the reconstruction domain is rectangular, fast solvers based on FFT can be considered. Depending on the boundary condition which is used, several implementations can be used:

Source Code

  1. Natural Boundary Condition: an implementation of Simchony and Chellappa's method is provided here. The natural boundary condition is the condition provided by the calculus of variations, and should provide more accurate results than homogeneous Neumann condition. Fast solution is provided by Cosine transform.
  2. Dirichlet Boundary Condition: If the depth is known (or, e.g., zero) on the boundary of the (rectangular) domain, an implementation of Simchony and Chellapa's method (via DST) is also provided here
  3. Periodic Boundary Condition: An implementation of Frankot and Chellappa's algorithm is provided by Peter Kovesi on his webpage. This is slightly faster than previous solvers, but periodic boundary conditions are usually not realistic.
  4. Homogeneous Neumann Boundary Condition: an implementation of such a solver is available on Amit Agrawal's webpage.

References

  1. A method for enforcing integrability in shape from shading algorithms
    Frankot, R.T. and Chellappa, R.
    IEEE T-PAMI (1988)
  2. Direct analytical methods for solving Poisson equations in computer vision problems
    Simchony, T. and Chellappa, R. and Shao, M.
    IEEE T-PAMI (1990)

Uncalibrated Photometric Stereo

Uncalibrated Photometric Stereo using Total Variation

This code performs Uncalibrated Photometric Stereo by minimizing the Total Variation of either the depth map or the M-field.

Given three or more different images of an object taken under the same viewpoint, but under different illumination conditions, a 3D-model of the object is estimated.

Source Code

The Matlab sources are available here.

References

  • Solving Uncalibrated Photometric Stereo using Total Variation
    Yvain Quéau, François Lauze, Jean-Denis Durou
    Journal of Mathematical Imaging and Vision (JMIV 2014 - Accepted)
    Paper (pdf)
  • Solving the Uncalibrated Photometric Stereo problem using Total Variation
    Yvain Quéau, François Lauze, Jean-Denis Durou
    Fourth International Conference on Scale Space and Variational Methods in Computer Vision (SSVM 2013)
    Paper (pdf)

Robust Photometric Stereo

Soon