|
Digital Image Stabilization by
Adaptive Phase Correlation Motion Vectors Filtering
Bibhuti Bikramaditya , Ohyun Kwon1, Joonki Paik
Image Processing and Intelligent Systems Laboratory,
Department of Image Engineering,
Graduate School of Advanced Imaging Science,
Multimedia, and Film, Chung-Ang University,
221 Huksuk-Dong,
Tongjak-Ku, Seoul 156-756, Korea,
http://ipis.cau.ac.kr 2nSYSTechnologies Co.Ltd,
1301, Mario-tower,222-12 Guro-dong,
Guro-gu, Seoul 135-841, Korea,
/http://www.nsystech.com
Abstract
This paper proposes digital image stabilization with sub-image novel phase correlation based global motion estimation and maximum peak filtering based motion correction. Global motion is estimated from the local motion of four sub-images each of which is detected using phase correlation (PC) based motion estimation. The correlation surface using Phase correlation techniques determine local motion vector (LMV) and most peak amplitude from block of LMV decides its global motion vector (GMV), thereby Accumulating motion vector (AMV) for panning. The proposed algorithm can make a robust digital image stabilization when camera affected by vibration or unwanted movement. Experimental results show that the proposed digital image stabilization algorithm can efficiently remove the unwanted displacement.
1. INTRODUCTION
The unwanted movements and vibration caused by hand tremble and heartbeat may affect digital image while using digital camcorder and digital camera. An unwanted movement in the camera shot often causes an incorrect superposition of the current and reference images as well as destructive consequences for typical change-detection algorithms. Compact consumer video cameras with powerful zooms are likely to results in the fluctuation of images to the hand movements, and various digital image stabilizing systems have been developed to improve the visual quality through stabilization process [1].
The digital image stabilization system can be divided into two parts: the global motion estimation system and the motion correction system. The motion estimation system is responsible for the estimation of interframe global motion vectors, which are forwarded to the motion correction system. The motion correction system accomplishes the stabilization of the image sequence according to the global motion model or objective. A review and brief comparison of already
Figure 1 Block diagram of digital image stabilization system.
2. MOTION ESTIMATION BY PHASE CORRELATION
5.1. Phase correlation methods
The idea behind this method is quite simple and is based on the Fourier shift property, which states that a shift in the coordinate frames of two functions is transformed tin the Fourier domain as linear phase differences. This can be described as follows,
Where * indicates the complex conjugate. The normalized cross power spectrum may also be viewed as the cross power spectrum of whitened signals. There are two possible ways of solving (3) for. One way is to directly work in the Fourier domain. For this purpose, consider a three-dimensional (3-D) Euclidean space whose canonical reference frame is given by the two frequency axes and the phase difference between the two images. The second possible approach which is more practical and also more robust to noise is to first inverse Fourier transform the normalized cross power spectrum. It is then a simple matter to determine, since from (3) the result is ),(d1d2),(d1d2),d(x-d1,y-d2-- which a Dirac delta function is centered at, which corresponds to the displacement between the two images.
Figure 2 (a) and (b) Drawer images with displacements and (c) phase correlation.
these local motion vectors and the previous global motion vectors, and the integration unit using the accumulated global motion vector decides whether the motion of a frame is caused by undesirable fluctuation of the camera or intentional panning. Finally, the stabilized image is generated by reading out the proper block of fluctuated image in the frame memory.
Figure 4 Block diagram of motion estimation system.
3. MOTION CORRECTION
In general, motion vectors from the subimage with moving objects are used in the present algorithms but it doesn’t meet the proper need for the stabilization, preferably it should be excluded from the global motion decision process. Since the hand movement is relatively slower than the frame rate of the video camera, hence motion vectors of two successive frames fluctuated by camera’s jitter should be similar. Based on these properties of camera’s motion, we use a simple and robust motion correction scheme in which the global motion vector is determined by separately selecting the most maximum peak of each motion vector elements from subimage.
After determining the global motion vector, the motion correction system in Figure 5 decides whether camera’s motion or intentional panning affects the motion of a frame.
An important point in motion correction is the distinction between jiggling and panning. Jiggling is the oscillatory movement that has to be stabilized. It hasn’t a constant direction on consecutive frames and its amplitude is generally little. Panning is the wanted motion that user does to capture a wide area of the scene. It is directed in the same direction and displacements are bigger than jiggling. In [4] proposed distinction between jiggling and panning by threshold (T). If absolute of GMV is larger than T, panning occurs which is not correct .In the similar way ,even if absolute of GMV is smaller than T, jiggling starts and again it causes image to be destabilized. In this paper, we use AMV for panning [2]. The AMV computation procedure is given by
To show the performance of the proposed algorithm, Figure 7 show the original image and stable image in indoor and Figure 8 show the original image and stable image in outdoor
5. CONCLUSIONS
In this paper, we proposed a digital image stabilization algorithm to remove unwanted displacement by hand shaking, which is based on subimage phase correlation for motion estimation and maximum peak filtering for motion correction. For |
|
proposed global motion estimation and motion correction techniques are analyzed.
Various digital image-stabilizing systems have been developed for camcorders free from degradation in picture quality by hand movement. The digital image stabilizer consists of the motion estimating system and the motion correction system. Most often, the motion estimation based on block matching algorithm [2] [3] [4] and phase correlation algorithm are being proposed for the same [5] [6]. In block matching, for computationally efficient image stabilization algorithms that represent point matching [3], edge pattern matching [2], grey-coded bit-plane matching [1], and block motion vectors filtering have been developed. However, block matching is not good results on without pattern image. Therefore, we use phase correlation for motion estimation.
In this paper, we propose a digital image stabilization with sub-image novel phase correlation based global motion estimation and maximum peak filtering based motion correction. Global motion is estimated from the local motion of four sub-images each of which is detected using phase correlation (PC) based motion estimation. Concretely among local motion vector (LMV), filter find most peak from phase correlation each blocks and set this LMV to global motion vector (GMV). And then make accumulated motion vector (AMV) for panning.
This paper is organized as follows: In Sec. 2, we summarize the sub-image phase correlation based global motion estimation algorithm. Motion correction using good filter algorithm described in Sec. 3. Experimental results and conclusions are respectively given in Sec. 4 and 5.
5.2. Motion estimation
In each image frame of the sequence, four sub-images are designated as shown in Figure. These sub-images are used to determine local motion vectors using phase correlation. To enable FFT computation for phase correlation, sub-images are predetermined to be of square shape with horizontal and vertical pixel dimensions being a power of two. Typically a sub-image size of 32×32is preferred to keep the computation load of motion estimation low and at the same time include sufficient spatial image content for correct estimation.
Figure 3 Location of sub-images used for local motion estimation.
For all four sub-images of an image frame, local motion vectors are estimated from the respective sub-images of the previous frame based on phase correlation. For each sub-image the largest peak amplitude location of the corresponding phase correlation surface is assigned as the local motion vector and recorded together with the corresponding peak amplitude value. For each image frame, in total four local motion vectors corresponding to the sub-images are estimated and the peak amplitude of each vector is noted. The global motion vector of the image frame can then be decided based on the peak amplitude values of local motion vectors.
The configuration of the DIS system is shown in Figure. The local motion estimation unit produces local motion vectors from subimage in the different position of the frame. The global motion decision unit determines the global motion vectors by processing
Where t represents time index, the constant k, , is used for smooth panning and for virtually enlarging the effective AMV range, and a, 0<=a<=1, for filtering out the unexpected noise effect on the AMV. Our proposed motion correction system in Figure 5.
Figure 5 Block diagram of motion correction system.
4. EXPERIMENTAL RESULTS
In order to demonstrate the performance of the proposed algorithm, we used a 642x480 single image frame from SONY
Camcorder DCR-TRV900. We used a image in outdoor and indoor. Block diagram for experiment is in Figure. We connect camcorder and computer by grabber from Meteor II standard. Color signal change to grey signal for motion estimation. After process motion estimation and motion correction we can get motion vector. Then we show original and stabilized image in real-time.
Figure 6. Block diagram of experimental process. 7. REFERENCES
[6] H. Foroosh, J.B. Zerubia and M. Berthod, “Extension of phase correlation to subpixel registration,” IEEE Trans. Image Processing, vol. 11, pp. 188-200, 2002.
[4] Filippo Vella, Alfio Castorina, Massimo and Giuseppe Messina, “Digital image stabilization by adaptive block motion vectors filtering,” IEEE Trans. Consumer Electronics, vol.
48, pp. 796-801, 2002.
[3] K. Uomori, A. Morimura, H. Ishii and Y. Kitamura, “Automatic image stabilization system by full-digital signal processing,” IEEE Trans. Consumer Electronics, vol. 36, pp.
510-519, 1990. [5] S. Erturk, “Digital Image Stabilization with Sub-Image Phase Correlation Based Global Motion Estimation,” IEEE Trans. Consumer Electronics, vol. 49, pp. 1320-1325, 2003.
[2] J.K. Paik, Y.C. Park and D.W. Kim, “An adaptive motion decision system for digital image stabilizer based on edge pattern matching,” IEEE Trans. Consumer Electronics, vol
. 38, pp. 607-615, 1992. [1] S.J. Ko, S.H. Lee, S.W. Jeon and E. S. Kang, “Fast digital image stabilizer based on gray-coded bit-plane matching,” IEEE Trans. Consumer Electronics, vol. 45, pp. 598-603,
1999. |
|
Facebook
Twitter
Linkedin