5.1. Calibration

There are three coordinate systems: those of the eye tracker (X, Y, Z), hand tracker (Xh, Yh, Zh), and virtual board (xb, yb), as shown in Figure 12. In practice, users have to calibrate these three coordinate systems to find connection the virtual board and hand tracker coordinates with the coordinates of the eye tracking sensor in advance. The eye tracking sensor is assumed to obtain the coordinates of the left and right pupil centers and the 2D eye gaze position on the virtual board. In the calibration, a board with some points that represent the priors of the measured 3D position (C1–C5) on the eye tracker coordinates is temporarily installed, and the 2D coordinates of those five points on the board [46] (C1–C5) are the outputs from the eye tracking sensors when an examiner looks at those five points. The conversion between the 2D virtual board and 3D eye tracking sensor coordinates is performed with the following equations:

where, N is a fixed distant value with real-world units (millimeters), and the resulting value R is the ratio of the converted eye gaze position in terms of pixels to millimeters.

where, G is the position (x, y) of the eye gaze in units of pixels. The resulting value J is the real-world value according to the ratio R and it is offset by C5.

System calibration.

Subsequently, the distances between the origins of the hand tracker and eye tracker (Hz and Hx) are manually measured, so that the transformation between the hand tracker and eye tracker coordinates can be obtained. During the testing state, the board that is used in calibrations is removed, and this is called the “virtual board” in this paper.

As another necessary calibration, the hand pointing template has to be established in advance. Because the hand and finger sizes of each user are different, the cell sizes have to be measured by a hand sensor, and the hand-pointing template has to be determined. The ranges between neighboring joints and tips (IL2−4, ML2−4, RL2−4, and PL2−4) are measured according to coordinates from a sensor, and rectangular cells are assumed to be established to cover all fingertips and joints, as shown in Figure 13. The sizes of the template cells (IT1−4, MT1−4, RT1−4, and PT1−4) can be simply obtained by the following equations:

where, F={I,M,R,P}.

Template range determination.

On the other hand, the ranges between neighboring finger joints on the baseline (W1–W4) are measured according to the coordinates from the hand motion, and the width of the template cells (Iw, Mw, Rw, and Pw) can be calculated by the following equations:

Finally, the bottom cells of those four fingers (I0, M0, R0, and P0) are originally for some bent fingers that possibly cross over the baseline, such that their sizes are assumed to be the same as those on the baseline (I1, M1, R1, and P1).