Algorithm of correction of error caused by perspective distortions of measuring mark images

– The algorithm of correction of error caused by perspective distortions of measuring mark images is discussed. Theoretical and experimental evaluation of error from image perspective distortions by the example of circular mark is conducted. The obtained results are prospective to be applied in the tracking systems, measuring devices and automatic robotic system. Algorithm with correction for decreasing of perspective distortions error will be useful for cost-cutting of developing systems as well as the existing systems could be improved to better accuracy level


Introduction
Up to date in a number of application engineering tasks a necessity of sighting marks recognition arise [1], as well as the necessity of calculation of mark center coordinates, e.g., for spatial measurements or provision of spatial orientation of information and measuring, robotic and other systems.
Despite a variety of well-known recognition algorithms for different marks centers coordinates, there is a common source of error for all algorithms of photographic image processing consisting in perspective distortion of objects projections in the plane of photographic camera image during work with images obtained by means of photographing of measured objects.An example of these distortions is shown in Figure 1.It is important to note that as affine ones, the perspective transformations are beyond the scope of Euclidean geometry and are stipulated with the application of central projecting.For example, parallel lines are no more parallel after transformations, circles are transformed to ellipses, and squares are transformed to rectangles.
As a rule, the measuring and tracking systems use a sighting mark in the form of figures possessing central symmetry [2], which simplifies tracking of their centers; they are: circles, rings, concentric circles [3,4].Thus, under influence of perspective distortions the circle transforms into ellipse, and for obtained ellipse a number of algorithms for tracking of center coordinates is still applicable, e.g., algorithm of center of mass tracking.However, it should be kept in mind that due to perspective distortions the geometric center of distorted figure does not coincide with the center of original figure dislocated with the same distortions (Fig. 2).For transformation of coordinates of 3D space points into 2D accounting for perspective a combination of expressions ( 1) and ( 2) is used: where x S , y S and z S are the initial point coordinates in 3D space; X and Y are the obtained coordinates in image plane accounting for perspective; x * and y * are the coordinates of point projections x S , y S and z S on plane z = 0 at parallel projecting; parameter u * contains the information on perspective distortions of coordinates x * , y * in compliance with the coordinates of convergence points x F , y F and z F located on respective axes.Points of convergence are the intercepts of lines that were parallel before distortion.Figure 1 shows an example of single-point convergence along z-axis, which is perpendicular to image plane.Points of convergence are located on axes and depending on values of x F , y F and z F projections with one, two or three points of convergence are possible.In the case of equality of infinity of all three values the transformation results in an ordinary parallel projection.In order to obtain coordinates X and Y the procedure of normalization of homogeneous coordinates (2), obtained as the result of (1), is used.
Similarly is possible the application of normalization of homogeneous coordinates (3) together with affine transformation (4) in plane for image perspective distortions.
where x n , y n are the initial (undistorted) coordinates of image points; x k , y k are the coordinates of image points (with distortions).Upon the results of affine transformation the coordinates of transformation the normalization of homogenous coordinates is applied by means of dividing of every term of the resulting vector by z * k .Herewith, a 00 , a 01 , a 02 , a 10 , a 11 , a 12 are the coefficients of affine transformation, a 20 and a 21 are the coefficients responsible for perspective distortions, and depending on signs and values of coefficients the direction and the "slope" of perspective will change [5,6].
Transformations (3) and ( 4) are reversible, so by selecting certain values of coefficients a ij it is possible to transform image without any perspective distortions.From expressions (3) and (4) for one point of plane there follows a pair of Equations (5).In order to perform the inverse transformation (exclusion of perspective distortions) it is necessary to select the values for eight coefficients a i j , taking coefficient a 22 (responsible for image scaling by two axes) equal to a unity.By using the coordinates of four pairs of initial and finite image points it is possible to solve the system (6) formed of ( 5) with application of four pairs of initial and finite points relative to eight coefficients a ij .
For obtaining of initial and finite coordinates of four points in image distorted with perspective it is necessary to include to initial image a priori information in the form of known shape of figure (e.g., square) described with four points of image.Thus, applying a square-shaped mark for exclusion of perspective distortions, the initial points in the system (6) are the apex points of square distorted with perspective, and finite points are any four points describing the square.

Experimental setup
For illustration of the degree of image perspective distortion the dependence of correlation between the semiaxes of ellipse, obtained with circle distortion (Fig. 3a), on perspective distortion coefficient a 20 (Fig. 3b) was plotted.For plotting of specified dependence the diametral points located horizontally and vertically were subjected to perspective distortions (3), (4) along horizontal axis with known values of coefficient a 20 , and then according to the formula (7) the correlation between the semi-axes of obtained ellipse was calculated.
where the subscript p designates the coordinates distorted with perspective, and the subscripts u, d, l, r designate upper, lower, left and right points of circle, respectively.In addition, the dependence of angle of deflection from normal direction of observation of measuring marks depending on correlation between semi-axes of ellipse obtained as the result of circle image distortion is plotted (Fig. 4).In this case, an effective and easy way of evaluation of angle of deflection of observation line from normal to the surface of measuring mark is the approximation with function (8), where ϕ is the angle of deflection of observation line from normal, and a e and b e are minor and major semi-axis of ellipse, respectively.
In order to obtain a theoretical dependence of error magnitude on the degree of perspective distortions (3), ( 4) there was plotted a function of distance between the points corresponding to the center of circle distorted with perspective, and geometric center of ellipse obtained as the result of the same perspective distortion of circle depending on coefficient a 20 .Further, for clarity of obtained results the values of angle of deflection from normal direction of observation of circle plane as the result of substitution of a 20 into ( 7) and (8) along abscissa axis instead of values a 20 were plotted.Values of coefficient a 20 varied within the range from 0 (corresponding to absence of distortions) to 0.05 in increments of 0.001.For obtaining the experimental dependence of perspective distortions error value on angle of deflection from normal observation direction of circle images the circles with different diameters in 8-bit format (256 levels of pixel brightness quantization) were created.Obtained images were distorted with perspective with different coefficients a 20 .The values of coordinates of center of mass of distorted figure were used as the coordinates of center of ellipse obtained during circle distortion.
Theoretical and experimental dependence of perspective error for different values of circle diameters are shown in Figure 5. Theoretical error is insensitive to the diameter, while the experimental error increases along with the diameter.

Experiment results
From plots, it is apparent that the error grows along with the increase of distortion coefficient (convergence point approaching zero), however at certain value of coefficient different for different circle diameters error stops growing and starts decreasing and asymptotically tending to zero.Error decrease is stipulated with the decrease of ellipse minor semi-axis and its asymptotical tending to zero.During work with the experimental data in the form of digitized images of circles, an error takes place, which is caused with discretization and is related to the fact that the points describing a circle upon the results of perspective transformation are nonuniformly distributed over discrete mesh of image matrix.Under the effect of this error (Fig. 5) there takes place a certain decrease of theoretical error from perspective distortions, however despite this the error of recognition remains relatively high and is the most demonstrative for ellipse obtained at angle deflection from normal direction of observation equal to 15-50 • .Thus, the area with elevated error is located in close vicinity to normal direction of object observation line.
In order to perform the inverse transformation (algorithm of exclusion of perspective distortions) a squareshaped mark was used, which was distorted with perspective along with the circle image.Prior to recognition of ellipse center the coordinates of square apexes were recognized, and according to obtained values the image was transformed based on a priori data on square mark shape.After described procedure of perspective distortions compensation the recognition of obtained figure center was conducted, which was compared to initial coordinates of circle center.
As initial points square apex points in distorted image were accepted, and the coordinates of finite points are chosen in an arbitrary manner with the condition that obtained points must describe a certain square in an image space [7].Transformation matrix utilizes only eight coefficients, while the 9th coefficient responsible for image scaling along two axes is set to a unity.Specified sequence of transformations was applied relative to image of circle against white background with the values of the coefficient a 20 from 0 to 0.05 in increments of 0.001, coefficient a 21 was always equal to 0 (in order to provide a single-point convergence of obtained image).The results of the experiment are resented in Figure 6; it  is apparent, that within the area of the most likely direction of sighting marks observation the error was decreased by several times.In plot (Fig. 6) is also noted the increase of error in the area of angular deflections from normal direction of observation of more than 55 • stipulated with significant influence of error caused with discretization combined with significant reduction of obtained ellipse size.

Conclusions
Considering error reduction within the most probable range of observation angles, the application of described method of compensation of error from perspective distortions of images is advisable, the main advantages of proposed method are: the possibility of work with images of sighting marks with the diameter of about 20 pixels obtained for angular deflections from normal direction of observation up to 55 • with error of recognition of coordinates of mark center within 0,1 pixel; analytical solution of equation system (5) allowing implementation of specified transformations as programs with high processing speed.
In this regard, the developed algorithm with perspective distortion errors correction could be implemented in the existing or developed measuring, robotic, tracking  systems in order to increase its accuracy and performance.The software could be developed and introduced as preprocessor or as a part of existing software or as additional dynamic software library.
The prospective use of the results of this work is in the modern 3D optical scanners for non-contact measuring and reverse engineering.Since, typically in 3D scanners there is a problem with an adequate description of surfaces by points cloud, it could be useful to collect more information to increase the accuracy of final 3D model.

Fig. 2 .
Fig. 2. Non-coincidence of ellipse center and the center of circle distorted with perspective.

Fig. 3 .
Fig. 3. Circle perspective distortion (a) and the dependence of ratio of ellipse axes on circle perspective distortion coefficient (b).

Fig. 6 .
Fig. 6.Evaluation of efficiency of error compensation method.