RevaWiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
= Research Activities

**[[public:reva:vortexresearch_en#Image Processing]]**

**[[public:reva:vortexresearch_en#Video Processing]]**

**[[public:reva:vortexresearch_en#3D Modelling]]**

**[[public:reva:vortexresearch_en#Augmented Reality]]**

**[[public:reva:vortexresearch_en#Interactions]]**


----

== Image Processing


=== Urban scene understanding
{{public:reva:imajbox.jpg?100|}} 

{{public:reva:sylvie-1.jpg?400|}} {{public:reva:sylvie-2.jpg?400|}} {{public:reva:sylvie-3.jpg?400|}}

This work has been done in collaboration with {{:vortex:logoimajing.jpg?80|http://www.imajing.fr/}}

References: list of publications 

----

=== Image matching

----

=== Visual Inspection

----

=== Image Parsing

----

----

== Video Processing

=== 3D Visual Tracking

We introduced the use of point-based 3D models as a shape prior for real-time 3D tracking with a monocular camera. The joint use of point-based 3D models together with GPU allows to adapt and simplify an existing tracking algorithm originally designed for triangular meshes. Point-based models are of particular interest in this context, because they are the direct output of most laser scanners. We show that state-of-the-art techniques developed for point-based rendering can be used to compute in real-time intermediate values required for visual tracking. In particular, apparent motion predictors at each pixel are computed in parallel, and novel views of the tracked object are generated online to help wide-baseline matching. Both computations derive from the same general surface splatting technique which we implement, along with other low-level vision tasks, on the GPU, leading to a real-time tracking algorithm

{{htvid>http://ubee.enseeiht.fr/dokuwiki/lib/exe/fetch.php?media=public:reva:leopard_occlusions.ogv|http://ubee.enseeiht.fr/dokuwiki/lib/exe/fetch.php?media=public:reva:leopard_occlusions.mp4|320x240}}

{{htvid>http://ubee.enseeiht.fr/dokuwiki/lib/exe/fetch.php?media=public:reva:leopard_AR.ogv|http://ubee.enseeiht.fr/dokuwiki/lib/exe/fetch.php?media=public:reva:leopard_AR.mp4|320x240}}

Related Publications:
  * Christophe Dehais, Géraldine Morin, Vincent Charvillat. //From Rendering to Tracking Point-based 3D Models//. In : Image and Vision Computing, Elsevier Science, Vol. 28 N. 9, pp. 1386-1395, April 2010. http://dx.doi.org/10.1016/j.imavis.2010.03.001 

  * Christophe Dehais, Vincent Charvillat, Géraldine Morin. //3D Visual Tracking Using a Point-based Model//. In : Vision, Modeling and Visualization (VMV 2006), AAchen, 22/11/2006-24/11/2006, K Kobbelt (Eds.), Aka/ IOS Press, pp. 41-48, November 2006. 

----
=== Zoomable video players

----

=== Hypervideo

----


== 3D modelling

=== Plant Modeling from Images based on Analysis-by-Synthesis

Plants are essential elements of virtual worlds to get pleasant and realistic 3D environments. Even if mature computer vision techniques allow the reconstruction of challenging 3D objects from images, due to high complexity of plant topology, dedicated methods for generating 3D plant models must be devised. We propose an //analysis-by-synthesis// method which generates 3D models of a plant from both images and a priori knowledge of the plant species.
Our method is based on a skeletonisation algorithm which allows to generate a possible skeleton from a foliage segmentation. Then, a 3D generative model, based on a parametric model of branching systems that takes into account botanical knowledge is built. This method extends previous works by constraining the resulting skeleton to follow hierarchical organisation of natural branching structure. A rst instance of a 3D model is generated. A reprojection of this model is compared with the original image. Then, we show that selecting the model from multiple proposals for the main branching structure of the plant and for the foliage improves the quality of the generated 3D model. Varying parameter values of the generative model, we produce a series of candidate models. A criterion based on comparing 3D virtual plant reprojection with original image selects the best model. 

{{ public:reva:plant1.png?800 |}}

{{ public:reva:plant2.png?800 |}}

Relevant publications:
  * Jerome Guenard, Géraldine Morin, Frédéric Boudon, Vincent Charvillat. //Reconstructing Plants in 3D from a single Image using Analysis by Synthesis//. In : Proceedings of the International Symposium on Visual Computing, Greece, 29/07/2013-02/08/2013, ACM
  * Jerome Guenard, Géraldine Morin, Vincent Charvillat, Frédéric Boudon. //Realistic Plant Modeling from Images based on Analysis-by-Synthesis (regular paper)//. In : Curves and Surfaces, Oslo, 28/06/2012-03/07/2012, Springer. 

----
=== Structure From Motion
We introduced a new unified Structure- from-Motion (SfM) paradigm in which images of circular point-pairs can be combined with images of natural points. An imaged circular point-pair encodes the 2D Euclidean structure of a world plane and can easily be derived from the image of a planar shape, especially those including circles. A classical SfM method generally runs two steps: first a projective factorization of all matched image points (into projective cameras and points) and second a camera self- calibration that updates the obtained world from projective to Euclidean. This work shows how to introduce images of circular points in these two SfM steps while its key contribution is to provide the theoretical foundations for combining “classical” linear self-calibration constraints with additional ones derived from such images. 

{{public:reva:sfmcircle.png?400|}} {{public:reva:sfmcircle3d.png?200|}}

Related publications:
  * Lilian Calvet, Pierre Gurdjos. //An Enhanced Structure-from-Motion Paradigm based on the Absolute Dual Quadric and Images of Circular Points//. In Proceedings of the International Conference on Computer Vision (ICCV 2013), Sydney, Australia, 
  * Lilian Calvet, Pierre Gurdjos, Vincent Charvillat. //Camera tracking using concentric circle markers: paradigms and algorithms//. In : IEEE International Conference on Image Processing (ICIP 2012), Orlando, États-Unis, 30/09/2012-03/10/2012, IEEE, [[http://dx.doi.org/10.1109/ICIP.2012.6467121|DOI:10.1109/ICIP.2012.6467121]]

----
=== Photometric Stereo (Multiple lightings)

Estimating the shape and appearance of an object, given one or several images, is still an open and
challenging research problem called 3D-reconstruction. Among the different techniques available, photometric stereo produces highly accurate results when the lighting conditions have been identified. When these conditions are unknown, the problem becomes the so-called uncalibrated photometric stereo problem, which is ill-posed. We showed how total variation (TV) can be used to reduce the ambiguities of uncalibrated photometric stereo, and we will study two methods for estimating the parameters of the //generalized bas-relief ambiguity//. 

{{public:reva:dsc_0058.jpg?200|}} 
{{public:parcoursmm:3in:wiki_stereophotometrie.png?900|}}

This research work is undergoing a technology transfer project supported by {{public:reva:toulouse-tech-transfer.jpg?80|http://www.toulouse-tech-transfer.com/}}

Related publications:
  * Yvain Quéau, François Lauze, Jean-Denis Durou. //Solving Uncalibrated Photometric Stereo using Total Variation//. In : Journal of Mathematical Imaging and Vision, Springer-Verlag, 2014, [[http://dx.doi.org/10.1007/978-3-642-38267-3_23| DOI:10.1007/978-3-642-38267-3_23]]

----
=== Adaptive 3D streaming 

Online galleries of 3D models typically provide two ways to preview a model before the model is downloaded and viewed by the user: (i) by showing a set of thumbnail images of the 3D model taken from representative views (or keyviews); (ii) by showing a video of the 3D model as viewed from a moving virtual camera along a path determined by the content provider. We propose a third approach called preview streaming for mesh-based 3D objects: by streaming and showing parts of the mesh surfaces visible along the virtual camera path. This work focuses on the preview streaming architecture and framework and presents our investigation into how such a system would best handle network congestion effectively. We introduced three basic methods: //stop-and-wait//, where the camera pauses until sufficient data is buffered;  //reduce-speed//, where the camera slows down in accordance to reduce network bandwidth;  //reduce-quality//, where the camera continues to move at the same speed but fewer vertices are sent and displayed, leading to lower mesh quality. 
We further propose two advanced methods:  //keyview-aware//, which trades off mesh quality and camera speed appropriately depending on how close the current view is to the keyviews, and  //adaptive-zoom//, which improves visual quality by moving the virtual camera away from the original path. A user study reveals that our keyview-aware method is preferred over the basic methods. Moreover, the adaptive-zoom scheme compares favorably to the keyview-aware method, showing that path adaptation is a viable approach to handling bandwidth variation.

{{ public:reva:3dstreaming.png?800 |}}

Related publications:
  * Shanghong Zhao, Wei Tsang Ooi, Axel Carlier, Geraldine Morin, and Vincent Charvillat. 2014. //Bandwidth adaptation for 3D mesh preview streaming//. ACM Trans. Multimedia Comput. Commun. Appl. 10, 1s, Article 13 (January 2014), 20 pages. [[http://doi.acm.org/10.1145/2537854|DOI:10.1145/2537854]] 

----

----

== Augmented Reality

=== Augmented reality and relighting

----

=== Real-time Camera tracking for visual effects (ROM Project)
The aim of the project ROM ("Real-time On set Matchmoving") is to bridge the gap between the production and the post production process in filmaking. The main idea is to develop a system that allows the director of the movie to have at least a rough preview of the digital effects (3D rendering) that will be later added during the post-production process. This requires the development of tools for real-time camera tracking able to recover the position of the camera using either natural features or artificial reference markers, or even both of them. 
The novelty of the research relies on three main aspects:
* //Flexibility//: the system can work in many different scenarios, such as in a production studios, indoor scenes or even outdoor settings. In particular the system can work either in presence of artificial landmarks (e.g. AR markers) or natural features or with a mixed combination of the two. Therefore the use of markers can be reduced on the scene and less effort is required in the post-production step to digitally remove them.
* //Visual re-localization//: the innovative feature of the system is that a (visual) database is built during a pre-shooting step, in which the tracked features (artificial and/or natural) are stored by means of their descriptors (e.g. SIFT, SURF...) and their associated 3D points. Also, some key-frames of the video sequence are stored to create a visual-dictionary; the dictionary can be later queried during any subsequent shot to determine the most similar key-frame, and thus, using the features database, the initial position of the camera in the scene.
* //Real-Time preview//: using the database of collected features, during the final shooting the 3D artifacts are rendered in real-time thanks to an innovative architecture based on state-of-the-art hardware and software. Also, thanks to the modularity of the system, a plug-in version of the software can run in the Maya environment.

{{youtube>small:xp2hd-2cmJU}}  {{youtube>small:unybb0MFCOo}} 



----

=== TOURA project - Mobile Augmented Reality for Cultural heritage 

The spread of Smartphones in our society allows to imagine new services for users, who will take advantage of this technology. Many smartphone apps offer nowadays services based on geolocalization and some of them integrate augmented reality through the display of points of interest (restaurants, monuments, etc…) superimposed on the image caught by the camera. These apps use GPS and heading sensors integrated in the smartphone to determine the elements to be displayed.


{{public:reva:toura.png?100|}}

Web site of the project: [[http://www.apsat.eu/en/applications-en/toura-tourisme-en.html|Project TOURA]]

----

=== SICASSE project - Mobile AR for Environment monitoring

  * Regional project with OKTAL, M3Systems, IRIT, SEEM, PNRPA
  * Study of the anthropic impact over the environment
  * Mobile application for building an evolutionary image database
  * Assess the quality of the pictures
  * Compare the new picture with the database to check the evolution


{{public:reva:sicasse.png?800|}}
----
== Interactions

=== Visual Markers

We designed the  C<sup>2</sup>Tag, a marker composed of a set of concentric circles of different radii. Thanks to their photometric and geometric properties, the marker are easily detectable and very robust to occlusions. C<sup>2</sup>Tag can be used to encode information using the ratio between the different radii to code a small amount of information, thus being suitable for augmented reality applications. 
In computer vision, they can be used for camera calibration and for camera tracking, as the circles intrinsicly encode some projective properties that allows to compute the pose of the camera.

Robust marker detection for Augmented Ads
{{public:reva:ubleam-1.png?300|}}  

Shared patent with {{public:reva:ubleam_logo_horizontal_mantra_200web.png?80|}}

Marker detection and camera pose estimation for Augmented Applications
{{htvid>http://ubee.enseeiht.fr/dokuwiki/lib/exe/fetch.php?media=vortex:cones.ogv|http://ubee.enseeiht.fr/dokuwiki/lib/exe/fetch.php?media=public:reva:cones.mp4}}

Available code:
  * ELSD, Ellipse and Line Segment Detector, code available [[http://ubee.enseeiht.fr/vision/ELSD/|here]]

Related publications:
  * Jun-Sik Kim, Pierre Gurdjos, In-So Kweon. //Euclidean structure from confocal conics: Theory and application to camera calibration//. In : Computer Vision and Image Understanding, Elsevier, Vol. 114 N. 7, pp. 803-812, July 2010. [[http://dx.doi.org/10.1016/j.cviu.2010.03.004|DOI:10.1016/j.cviu.2010.03.004]]
  * Viorica Pătrăucean, Pierre Gurdjos, and Rafael Grompone von Gioi. 2012. //A parameterless line segment and elliptical arc detector with enhanced ellipse fitting//. In Proceedings of the 12th European conference on Computer Vision (ECCV'12) Springer-Verlag, Berlin, Heidelberg, 572-585. [[http://dx.doi.org/10.1007/978-3-642-33709-3_41|DOI:10.1007/978-3-642-33709-3_41 ]]
  * Lilian Calvet, Pierre Gurdjos, Vincent Charvillat. //Camera tracking using concentric circle markers: paradigms and algorithms//. In : IEEE International Conference on Image Processing (ICIP 2012), Orlando, États-Unis, 30/09/2012-03/10/2012, IEEE, [[http://dx.doi.org/10.1109/ICIP.2012.6467121|DOI:10.1109/ICIP.2012.6467121]]