Computer stereo vision

Computer stereo vision is the extraction of 3D information from digital images, such as those obtained by a CCD camera. By comparing information about a scene from two vantage points, 3D information can be extracted by examining the relative positions of objects in the two panels. This is similar to the biological process of stereopsis.

== Outline == In traditional stereo vision, two cameras, displaced horizontally from one another, are used to obtain two differing views on a scene, in a manner similar to human binocular vision. By comparing these two images, the relative depth information can be obtained in the form of a disparity map, which encodes the difference in horizontal coordinates of corresponding image points. The values in this disparity map are inversely proportional to the scene depth at the corresponding pixel location. For a human to compare the two images, they must be superimposed in a stereoscopic device, with the image from the right camera being shown to the observer's right eye and from the left one to the left eye. In a computer vision system, several pre-processing steps are required.

The image must first be undistorted, such that barrel distortion and tangential distortion are removed. This ensures that the observed image matches the projection of an ideal pinhole camera. The image must be projected back to a common plane to allow comparison of the image pairs, known as image rectification. An information measure which compares the two images is minimized. This gives the best estimate of the position of features in the two images, and creates a disparity map. Optionally, the received disparity map is projected into a 3d point cloud. By utilising the cameras' projective parameters, the point cloud can be computed such that it provides measurements at a known scale.

=== Active stereo vision === The active stereo vision is a form of stereo vision which actively employs a light such as a laser or a structured light to simplify the stereo matching problem. The opposed term is passive stereo vision.

Conventional structured-light vision (SLV) employs a structured light or laser, and finds projector-camera correspondences. Conventional active stereo vision (ASV) employs a structured light or laser, however, the stereo matching is performed only for camera-camera correspondences, in the same way as the passive stereo vision. Structured-light stereo (SLS) is a hybrid technique, which utilizes both camera-camera and projector-camera correspondences.

== Applications == 3D stereo displays find many applications in entertainment, information transfer and automated systems. Stereo vision is highly important in fields such as robotics to extract information about the relative position of 3D objects in the vicinity of autonomous systems. Other applications for robotics include object recognition, where depth information allows for the system to separate occluding image components, such as one chair in front of another, which the robot may otherwise not be able to distinguish as a separate object by any other criteria. Scientific applications for digital stereo vision include the extraction of information from aerial surveys, for calculation of contour maps or even geometry extraction for 3D building mapping, photogrammetric satellite mapping, or calculation of 3D heliographical information such as obtained by the NASA STEREO project.

== Detailed definition ==

A pixel records color at a position. The position is identified by position in the grid of pixels (x, y) and depth to the pixel z. Stereoscopic vision gives two images of the same scene, from different positions. In the adjacent diagram light from the point A is transmitted through the entry points of pinhole cameras at B and D, onto image screens at E and H. In the attached diagram the distance between the centers of the two camera lens is BD = BC + CD. The triangles are similar,

ACB and BFE ACD and DGH ${\begin{aligned}{\text{Therefore displacement }}d&=EF+GH\\&=BF({\frac {EF}{BF}}+{\frac {GH}{BF}})\\&=BF({\frac {EF}{BF}}+{\frac {GH}{DG}})\\&=BF({\frac {BC+CD}{AC}})\\&=BF{\frac {BD}{AC}}\\&={\frac {k}{z}}{\text{, where}}\\\end{aligned}}$ z = AC is the distance from the camera plane to the object. So assuming the cameras are level, and image planes are flat on the same plane, the displacement in the y axis between the same pixel in the two images is, $d={\frac {k}{z}}$ The depth component in the two images are $z_{2}$
, given by, $z_{1}(x,y)=\min \left\{v:v=z_{2}(x,y+{\frac {k}{z_{2}(x,y)}})\right\}$

=== Image rectification === Where the image planes are not co-planar, image rectification is required to adjust the images as if they were co-planar. This may be achieved by a linear transformation. The images may also need rectification to make each image equivalent to the image taken from a pinhole camera projecting to a flat plane.

=== Smoothness === Smoothness is a measure of the similarity of colors. Given the assumption that a distinct object has a small number of colors, similarly-colored pixels are more likely to belong to a single object than to multiple objects. The method described above for evaluating smoothness is based on information theory, and an assumption that the influence of the color of a voxel influences the color of nearby voxels according to the normal distribution on the distance between points. The model is based on approximate assumptions about the world. Another method based on prior assumptions of smoothness is auto-correlation. Smoothness is a property of the world rather than an intrinsic property of an image. An image comprising random dots would have no smoothness, and inferences about neighboring points would be useless. In principle, smoothness, as with other properties of the world, should be learned. This appears to be what the human vision system does.

== Information measure ==

=== Least squares information measure === The normal distribution is $L(x,\mu ,\sigma )=\log _{2}(\sigma {\sqrt {2\pi }})+I(x,\mu ,\sigma ){\frac {\log _{2}e}{2}}$

=== Information measure for stereoscopic images === The least squares measure may be used to measure the information content of the stereoscopic images, given depths at each point $z(x,y)$
. Firstly the information needed to express one image in terms of the other is derived. This is called $I_{m}$
. A color difference function should be used to fairly measure the difference between colors. The color difference function is written cd in the following. The measure of the information needed to record the color matching between the two images is, $I_{m}(z_{1},z_{2})={\frac {1}{\sigma _{m}^{2}}}\sum _{x,y}\operatorname {cd} (\operatorname {color} _{1}(x,y+{\frac {k}{z_{1}(x,y)}}),\operatorname {color} _{2}(x,y))^{2}$ The total measure of smoothness uses the distance between voxels as an estimate of the expected standard deviation of the color difference,

      I
      
        s
      
    
    (
    
      z
      
        1
      
    
    ,
    
      z
      
        2
      
    
    )
    =
    
      
        1
        
          2
          
            σ
            
              h
            
            
              2
            
          
        
      
    
    
      ∑
      
        i
        :
        {
        1
        ,
        2
        }
      
    
    
      ∑
      
        
          x
          
            1
          
        
        ,
        
          y
          
            1
          
        
      
    
    
      ∑
      
        
          x
          
            2
          
        
        ,
        
          y
          
            2
          
        
      
    
    
      
        
          cd
          ⁡
          (
          
            color
            
              i
            
          
          ⁡
          (
          
            x
            
              1
            
          
          ,
          
            y
            
              1
            
          
          )
          ,
          
            color
            
              i
            
          
          ⁡
          (
          
            x
            
              2
  ...

(Article truncated for display)

Source

This content is sourced from Wikipedia, the free encyclopedia. Read full article on Wikipedia

Category

Computer Vision - Computer Science