B Bellers and G de Haan Philips Research Laboratories Television Systems Group Prof Holstlaan 4 5656 AA Eindhoven The Netherlands TEL31402744285 FAX31402742630 Key words deinterlacing motion compensation motion estimation sequential scan conversion g ID: 30036 Download Pdf

165K - views

Published bykarlyn-bohler

B Bellers and G de Haan Philips Research Laboratories Television Systems Group Prof Holstlaan 4 5656 AA Eindhoven The Netherlands TEL31402744285 FAX31402742630 Key words deinterlacing motion compensation motion estimation sequential scan conversion g

Download Pdf

Download Pdf - The PPT/PDF document "Advanced motion estimation and motion co..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Page 1

Advanced motion estimation and motion compensated de-interlacing E.B. Bellers and G. de Haan Philips Research Laboratories Television Systems Group Prof. Holstlaan 4 5656 AA Eindhoven The Netherlands TEL:+31–40–2744285 FAX:+31–40–2742630 Key words: de–interlacing, motion compensation, motion estimation, sequential scan conversion, generalized sampling theorem Abstract: This paper describes a new high quality de-interlacing algorithm applying motion estimation and compensation techniques. First, a comparison between two recently introduced de-interlacing concepts will be

presented. One method is based on a generalized sampling theorem and the other uses time- recursion. The new algorithm aims at combining the beneﬁts of both. 1 INTRODUCTION Historically, interlacing has been introduced to offer a compromise between quality and re- quired bandwidth. A major drawback of the interlaced scanning format on current bright high resolution displays is the line ﬂicker and serration of moving edges. In the literature, several de- interlacing algorithms have been proposed to eliminate these artifacts, or to serve as a base for other scan rate conversions.

Delogne et al. [1] recently proposed an advanced motion estimation and de-interlacing tech- nique based upon a generalization of the sampling theorem. For an assumed velocity, the motion compensated correlation between two successive frames is calculated applying motion compen- sated vertical-temporal ﬁlters. As this can be repeated for any assumed velocity, it is possible to calculate the velocity for which this correlation has a maximum. This method is elegant, but re- quires a constant velocity over a three ﬁeld period, which is a serious drawback for sequences with

acceleration or covering and uncovering, which are rather common case in natural sequenc- es. The problem has been noticed by the authors of the original algorithm already, and has re- sulted in an approach in which they maximize the correlation between a reconstructed frame and

Page 2

a successive ﬁeld as mentioned in [2] by Vanderdorpe et al.. The constant velocity constraint is reduced to a two ﬁeld period. Wang et al. [3], somewhat earlier, proposed an alternative high-quality, time-recursive, de- interlacing concept, which does not impose a constant velocity

constraint. The de-interlacing proposal also requires motion estimation and aims at the highest performance level. This paper compares the time-recursive technique proposed by Wang with the Vandendorpe method, applying the 3D Recursive Search (3D-RS) block matcher of [5] to both algorithms, and introduces an advanced new de-interlacing algorithm that aims at combining the advantages of the other methods. The motion estimator used for all algorithms is the same apart from the error function which depends on the algorithm under test. Section 2 starts with a description of this 3D-RS block

matcher. Section 3 focuses on the time-recursive and generalized sampling theorem based de-in- terlacing algorithms. In section 4, the new algorithm is presented, and section 5 compares the ex- perimental results with these algorithms. Finally, conclusions are drawn in section 6. 2 THE 3D RECURSIVE-SEARCH BLOCK MATCHER The high quality and efﬁcient 3D RS block matcher of [5] is used in the algorithms present- ed. This algorithm uses a small number of candidate vectors per block of pixels with a quarter pixel accuracy. Furthermore, due to the inherent smoothness constraint, it yields

very coherent vector ﬁelds that closely correspond to the true-motion of objects. This makes this method also suitable for scan rate conversion. This section brieﬂy summarizes its characteristics. In block-matching motion estimation algorithms, a displacement vector (or motion vector) is assigned to the center , with for transpose, of a block of pixels in the current ﬁeld by searching a similar block within a search area , also centered at , but in the previous ﬁeld . This similar block has a center which is shifted with respect to over the displacement vector . To

ﬁnd , a number of candidate vectors are evaluated applying an error measure to quantify block similarity. Figure 1 demon- strates the procedure. The block of pixels (positions) is deﬁned by: (1) with and the block width and block height respectively , and the spatial position in the image. The candidate vectors are selected from the candidate set , which is deﬁned by: (2) 1. In our experiments, was set to 8 pixels and to 8 frame lines. db () () tBb () nSAb () db () db () eCb ,, () Bb () xy () ----- xx ----- èø æö ---- yy ---- èø æö èø æö îþ íý ìü WH xy () WH CS b () CS b

() db èø æö èø æö () èø æö db èø æö èø æö () èø æö db èø æö èø æö èø æö ,,

Page 3

where the update vectors and are selected from an update set , deﬁned by: (3) with the integer updates deﬁned by: (4) and the fractional updates , necessary to realize sub-pixel accuracy, are deﬁned by: (5) From these equations it can be concluded that the candidate set consists of spatial and spa- tio-temporal ‘prediction vectors’ from a 3D neighbourhood and an updated prediction vector. This implicitly assumes spatial and/or temporal consistency. The updating process involves up-

dates added to either of the spatial predictions. Figure 2 shows where the spatial and spatio-tem- poral prediction vectors are located relative to the current block. The displacement vector resulting from the block-matching process, is a candidate vector which yields the minimum value of the error function : (6) The error function (which will be given in the following subsections) is a cost function of the luminance values, with spatial position of the pixels in the current block and those calculated with aid of the candidate vector. This error function is different for the Wang and the

Vandendorpe approach, called Time-Recursive (TR) error function and Transversal Gen- eralized-Sampling-Theorem (TGST) error function respectively. SA B(b n-1 image number Figure 1 Illustration of block-matching of the dark areas () () US US b () US () US () US () US () èø æö èø æö èø æö èø æö èø æö èø æö èø æö èø æö èø æö èø æö èø æö èø æö èø æö ,, ,, ,, ,, ,, ,, îþ íý ìü US () US () 0.25 èø æö 0.25 èø æö 0.25 èø æö 0.25 èø æö 0.5 èø æö 0.5 èø æö 0.5 èø æö 0.5 èø æö ,,,,,,, îþ íý ìü db () eCb ,, () db () CCSeCb ,, () eV b ,, () () VCSb () () {} fxn () xxy ()

Page 4

2.1 TR-error

function The TR-error function is a cost function of luminance values of the pixels in the current block and those of the shifted block from the previous ﬁeld, summed over the block . A common choice, which we also use, is the Sum of the Absolute Differences ( ). The TR-er- ror function is deﬁned by: (7) with the de-interlaced output frame . 2.2 TGST error function The TGST error function is calculated, using a generalization of the sampling theorem. From the sampling theorem, it is known that a bandlimited signal with maximum frequency can exactly be reconstructed if this signal

is sampled with a frequency of at least . Already in 1956, Yen [6] showed a generalization of this theorem. Yen proved that any signal that is band- limited by can exactly be reconstructed by independent sets of samples, sampled with frequency . This theorem can effectively be used to perform motion estimation and de-inter- lacing as also presented by Vandendorpe [2] and Delogne [1]. This method will be addressed as the TGST approach. An application of the generalized sampling theorem is to calculate an error-function for mo- tion estimation by comparing a generated ﬁeld from the

previous and pre-previous image with the most recent ﬁeld. Figure 3 shows that samples from the previous ﬁeld and pre-previous ﬁeld are shifted over the motion vector in order to create two sets of samples for ﬁeld number . An appropriate ﬁlter that matches the desired interpolator can reconstruct the expected samples. Mo- tion estimation tries to minimize the difference between the generated ﬁeld and the current ﬁeld The calculation of the expected samples is explained in the papers of Vanderdorpe [2] and Delogne [1]. Kalker [7] shows a

generalization of this concept which does not require the transla- tion via the Fourier domain. Figure 2 Positions, relative to the current block, from which the prediction vectors are taken in the 3D RS block-matcher --> spatial prediction --> spatio-temporal prediction current block block in current ﬁeld block in previous ﬁeld Bb () SAD TR Cb ,, () SAD TR Cb ,, () fxn () out xC () xBb () == out xn () ---- 0.5 ----

Page 5

According to Kalker [7], the expected samples for the odd ﬁeld , for vertical motion only , are calculated by: (8) with the sample from the

odd ﬁeld with ﬁeldnumber at vertical position , the sample from ﬁeld at vertical position , the desired ﬁlter impulse re- sponse which models the shift due to motion and the interpolator as well, and the odd ﬁeld of . Note that (9) In the z-domain, equation (8) is rewritten into: (10) with (11) Assuming that we have the complete frame at , available, of which ﬁeld is extracted. Consequently, the following equation is valid: (12) Field can be reconstructed by shifting the samples from frame over the motion vector, applying the desired interpolator, and

extracting the desired ﬁeld samples. So, 2. Horizontal motion is irrelevant for this explanation, since it can be solved with simple sample rate con- version theory. Therefore, it is set to zero for clarity. n-1 n-2 existing samples motion compensated samples expected samples vertical position y ﬁeld number Figure 3 Motion estimation with the generalized sampling theorem odd even odd y+1 y+2 y+3 y+4 y-1 xn () xn () fx èø æö èø æö () fx èø æö èø æö () xn () nxy () fxn () xy () hk () () èø æö èø æö mod 2 = 0 èø æö èø æö mod 2 = 1 zn () Fzn () () Fzn () () () () () () () () zn () zn

() zn () ()

Page 6

(13) where describes the motion over one ﬁeld period and the desired interpolation in the z-do- main. Field can now also be reconstructed by shifting the samples from over twice the motion vector. This results in: (14) Using the following set of characteristics: (15) equation (13) results in: (16) and equation (14) results in: (17) Substituting equation (16) for in (17) ﬁnally results in the desired expression: (18) with (19) Similar expressions can be extracted for the calculation of an even ﬁeld. If the motion vector and the desired

interpolator are known, expression (19) calculates the ﬁlter coefﬁcients. Assuming a bilinear interpolator and a shift over , is modelled as: (20) Consequently, (21) and (22) zn () zn () Hz () () Hz () nF zn () zn () zn () Hz () () Hz () () zn () () () == zn () xkn () is odd zn () xkn () is even Xzn () Yzn () () zn () zn () zn () zn () Xzn () Yzn () () zn () zn () zn () zn () zn () zn () () zn () () zn () zn () () () zn () () () ×× zn () zn () az () zn () bz () zn () az () () () bz () () Hz () a () () () () a az () a () bz () 21 ()

Page 7

If the velocity equals, e.g.

0.5 pixels per ﬁeld, then , and the luminance sample can be calculated according to: (23) This is illustrated in ﬁgure 4. So, for a given candidate vector, can be calculated, which can then be matched with the current ﬁeld. The same set of candidates as deﬁned in section 2 is used to ﬁnd the motion vectors. The best vector selection is based on minimization of the according to: (24) From ﬁgure 4 and expression (24), it is concluded that the motion vector is assumed con- stant for all pixels within the ﬁlter aperture. So, motion is considered to

be constant over a 2-ﬁeld period and spatially over the ﬁlter length, which deﬁnes the regional vertical-temporal uniform motion constraint of the TGST method. 3 THE DE-INTERLACING METHODS The de-interlacing algorithm used in the Wang approach will be called ‘TR de-interlacing in this paper. The Vandendorpe de-interlacer is referred to as the ‘TGST de-interlacer’. 3.1 The TR de-interlacer In [3], a time-recursive de-interlacing algorithm is proposed in which the interpolated pixels are determined by motion compensating the previously found de-interlaced output signal: --

fxn () fxn () fx èø æö èø æö -- fxn () -- fx èø æö èø æö n-1 n-2 existing samples motion compensated samples expected samples vertical position y ﬁeld number y+3 y+2 y+1 y-1 y+4 1/4 -1/4 1X calculated sample Figure 4 Calculation of the expected sample by applying a form of the generalized sampling theorem fxn () SAD TGST Cb ,, () SAD TGST Cb ,, () == fxn () () xCxn () èø æö èø æö () Cxn () èø æö èø æö xBb ()

Page 8

(25) where is the interlaced input (luminance) signal and the de-interlaced output. The motion vector is obtained by a ‘block erosion’ process [8], which uses

several block vectors in the direct environment of . Note that is true for original lines only. The pixels interpolated in the current frame are, generally, due to the sub-pixel accuracy partly based on pixels interpolated in the de-interlaced process of the previous ﬁeld. As an impli- cation, errors originating in an output frame, can propagate into later output frames. This is in- herent to the recursive approach, and the most important drawback of this method. To prevent serious propagation errors, several solutions have been described in [3]. Particu- larly, the median ﬁlter

is recommended to solve this problem. Consequently, equation (25) changes to: (26) Although this is a very effective method, it introduces alias in the de-interlaced image. 3.2 The TGST de-interlacing The de-interlacer proposed in [2] by Vandendorpe is illustrated in ﬁgure 5. The missing samples in ﬁeld of ﬁgure 5 can now be calculated according to: (27) Both the original samples from the current ﬁeld and the original samples from the previous ﬁeld are used to calculate the missing samples in the current ﬁeld. In the z-domain, the missing samples can

be calculated according to: outTR xn () fxn () mod 2 mod 2 outTR xdxn () () else fxn () out xn () dxn () db () mod 2 mod outTR xn () fxn () mod 2 mod median f outTR xdxn () () fx èø æö èø æö fx èø æö èø æö îþ íý ìü else n-1 n-2 existing samples motion compensated samples calculated samples vertical position y ﬁeld number odd even odd y+1 y+2 y+3 y+4 y-1 Figure 5 De-interlacing using a generalization of the sampling theorem xn () fx èø æö èø æö () fx èø æö èø æö ()

Page 9

(28) and (29) Solving from equations (28) and (29) results in: (30) with (31) If we again assume a bilinear

interpolator and a shift over : (32) the ﬁlter coefﬁcients are deﬁned by: (33) As an example, consider the situation of a motion of 0.5 pixels per ﬁeld and a bilinear inter- polator, then , and the missing samples can be calculated according to: (34) This is illustrated in ﬁgure 6. The missing samples are determined by the original samples of the current and the previous ﬁeld only. The motion estimator also uses original samples only. Therefore, errors, due to incor- rect motion vectors, will not propagate, which is a major advantage compared to the

TR-algo- rithm. zn () zn () Hz () () zn () () zn () () == zn () zn () Hz () () zn () () zn () () == zn () zn () az () zn () bz () zn () az () () () () --------------- bz () () () --------------- Hz () a () az () a () ------------------- bz () ------------ -- fxn () fx èø æö èø æö -- fx dxn () () -- fx dxn () èø æö èø æö n-1 n-2 existing samples motion compensated samples to be calculated samples vertical position y ﬁeld number y+2 y+1 y-1 y-2 y+3 1/2 -1/2 calculated sample Figure 6 Calculation of the missing samples using generalized sampling

Page 10

The de-interlaced output

is deﬁned by: (35) 4 RECURSIVE GST MOTION ESTIMATION AND DE-INTERLACING ALGORITHM Delogne et al. [1] proposed a motion estimation algorithm based on a generalization of the sampling theorem, which requires uniform motion over a 3 ﬁeld period. Vandendorpe et al. [2] proposed a similar algorithm which requires motion to be uniform over a 2 ﬁeld period. The 3D RS block matcher [5] is not restricted in this sense, which is an advantage as motion is generally not uniform. It is therefore expected that the TR approach is advantageous in case of non-uni- form motion. However, in

the case of uniform motion, the proposed algorithms of Delogne et al. and Vandendorpe et al. are expected to perform well. The de-interlacer as described in subsection 3.2 uses information from the past (ﬁeld ) and from the current ﬁeld for calculating the interpolated lines. Generally, the samples from the current ﬁeld are highly correlated with the samples from the missing lines. Consequently, it is advantageous to use information from the current ﬁeld. The TR de-interlacer does not use infor- mation from the current ﬁeld for calculating the interpolated

lines, but uses information from pre- vious ﬁelds only The combination of a ‘recursive’ motion estimator which uses the current ﬁeld and the pre- vious de-interlaced ﬁeld (which is generated using data from the previous ﬁeld and pre-previous ﬁeld only), and the de-interlacer of section 3.2 is expected to outperform the previous approaches and will be addressed as the recursive GST method (RGST). Note that the estimator uses infor- mation from 3 successive ﬁelds only (in contrast with the TR method which uses information from the complete history), due

to the choice of the de-interlacer. Consequently, severe error propagating as with the TR method can not occur. The de-interlaced image is median ﬁltered prior to the TR motion estimation. As a conse- quence, serious errors of the de-interlacer will not deteriorate the motion estimator. Without pro- tection of the estimator, the result is a ‘self-fulﬁlling prophecy’. Due to wrong or inaccurate motion vectors, the de-interlacer generates a wrong de-interlaced image, which is used by the motion estimator. The motion estimator estimate motion between this ‘wrong’ image and the in-

put, which results in a wrong motion vector. As a consequence, the de-interlaced image is incor- rect. The protection of the estimator by a median prevents these errors from propagating. 3. However, the median ﬁlter used to prevent errors for propagating can also introduce information from the current ﬁeld, but is only meant as an ‘escape’. outTGST xn () fxn () mod 2 mod fx dxn () èø æö èø æö () fx dxn () èø æö èø æö () else

Page 11

4.1 RGST with selective median The median protection of the estimator does not prevent de-interlace errors, however, pre- vents severe

errors from propagating. For near-critical velocities, shifted samples are mapped closely to original samples. The difference between these sample values greatly inﬂuences the in- terpolation. As a consequence, this difference becomes ‘boosted’, which also boosts the noise level. Therefore, inaccuracies can occur, which yield into undesired artefacts for which a remedy is required. The accuracy of the motion estimator is 0.25 pixel. Together with the proposed bilinear in- terpolator, ﬁlter coefﬁcients can be calculated according to Kalker et al. [7]. The ﬁlter

coefﬁ- cients that result from this are either {-2.25, 3, 0.25}, {-0.5, 1, 0.5}, {-0.083, 0.333, 0.75} or {0, 0, 1}, with the middle coefﬁcient referring to the current ﬁeld and the other coefﬁcients to the previous ﬁeld. The ﬁlter with coefﬁcients {-2.25, 3, 0.25} has a high gain in the high frequencies. As a con- sequence, the resulting samples might be ‘over-corrected’ due to a wrong motion vector or just peaked due to inaccuracies. Consequently, the de-interlaced sequence might show regions that are extremely boosted compared to the rest of

the image. The estimator is also negatively inﬂu- enced by this effect. As a remedy against this phenomenon, a median ﬁlter is activated when this ﬁlter is select- ed, according to: (36) This approach will be denoted as the RGST with selective median. In order to allow correct motion estimation of vertical high spatial frequencies, the selective median is also used for the estimator The architecture of this new algorithm is shown in ﬁgure 7. 5 EVALUATION Several tools can be used to evaluate the de-interlaced results ranging from objective mea- surements to

subjective evaluation. We preferred to use the objective measurement based on the mean-square-error, since it is used in the papers of the described de-interlacing methods. Howev- er, it is not always a reliable indicator. New tools which better reﬂect the relation between the measurement and the perception are still desired. 4. A proposal with a selective median for the estimator only was not found to be an interesting option. outRGSTseM xn () median f outTGST xn () fx èø æö èø æö fx èø æö èø æö ,, îþ íý ìü boosting filter selected outTGST xn () else

Page 12

5.1 Tools Several

sequences with different characteristics were processed in order to evaluate the dis- cussed methods. As an objective measure, the interlaced Mean-Square-Error, , is calculat- ed according to: (37) with indicating the Measurement Window, the number of samples within the mea- surement window, and the original samples in ﬁeld . All ﬁeld lines within con- tribute to the interlaced MSE. For sequences that originate from movie-material, the de-interlaced images can be com- pared with the original, resulting in a real MSE, and deﬁned by: (38) with the output of the chosen

de-interlacer; , , or . The real MSE can not be calculated for video-camera material, since no progressive original is available. The MSE scans all frame lines within instead of the ﬁeld lines only for the in- terlaced MSE. The MSE has the advantage that it can be applied to judge also the performance in case of critical velocities, whereas this is not reﬂected in the . as an additional criterion, the Motion Trajectory Inconsistency, MTI [4], will be calculated: (39) As with the MSE, all frame lines within contribute to the MTI. ﬁeld memory ﬁeld memory deinterlacer

median motion estimator GST mux de-interlaced output interlaced input Figure 7 Proposed de-interlacing architecture = line memory MSE MSE () MW ------------- orig xn () fx dxn () () () xMW MW N MW orig xn () nMW MSE n () MW ------------- orig xn () out xn () () xMW out xn () outTR xn () outTGST xn () outRGST xn () outRGSTseM xn () MW MSE MTI n () MW ------------- fx dxn () () out xn () () xMW MW

Page 13

A low MTI score indicates a high correlation between the previous de-interlaced image and the currently calculated de-interlaced image, or more speciﬁc, it is a measure for the

temporal consistency along the motion trajectory. As indicated in [4], a problem with this measure is that a good score is a necessary but insuf- ﬁcient constraint. Switching the output to zero, forces the MTI to low values, while the picture is seriously degraded. However, a lower MTI coupled to an almost stable (interlaced) MSE is a strong indication for quality improvement. These measurements together form a useful tool to evaluate the alternative algorithms. 5.2 Results The sequences used in the evaluation are Renata (ample vertical detail, horizontal veloci- ties), Mobile (both

horizontal and vertical velocities, including critical velocities), Shopping (ample vertical and horizontal detail, with critical velocities), RenataSpeed (same as Renata but accelerated 3 times), Tokyo (slow vertical and horizontal motion) and Bicycle (rotation). For calculation of the real MSE, the sequences Tokyo and Bicycle are used. A snapshot of these se- quences are shown in ﬁgure 8. The results are categorized into two groups; one with a near uniform motion ( Mobile Shopping and Tokyo ) and one with non-uniform motion ( Renata RenataSpeed and Bicy- cle ). The transversal

de-interlacer of [2] was expected to perform well for images with uniform motion, since the constraint of uniform motion over a 2 ﬁeld period is in this case valid. The re- cursive de-interlacer was expected to perform better in situations where this constraint is invalid. The results are shown in the plots for MSE and MTI in the ﬁgures 9, 10 and 11. Table 1 shows the results in both MSE and MTI improvement with respect to the TR algorithm. Some observations from these results: • In all situations, the RGST with selective median outperforms the one without selective me- dian both

for the MSE as well as the MTI. The improvement is partly due to the elimina- tion of the median in the estimator, which allows vertical high frequency to be tracked, and partly caused by the protection at the output due to the selective median. Figure 10 Percentage MSE improvement compared to the TR method; a) interlaced MSE, b) real MSE. ( R=Renata, M=Mobile, S=Shopping, RS=RenataSpeed, B=Bicycle, T=Tokyo ). TGST RGST RGST sel med

Page 14

• Similarly, the RGST outperforms in all situations the TGST, which is mainly due to remov- ing the uniform motion constraint. 5. The trend in

the real MSE’s resembles that of the interlaced MSE’s, which indicates that the interlaced MSE is also a valuable measure indicating performance improvements. Figure 8 Images from the used sequences. Images b,c,e have nearly uniform motion, whereas a,d,f are typical non-uniform a) Renata b) Mobile c) Shopping d) RenataSpeed e) Tokyo f) Bicycle

Page 15

Figure 9 MSE (a,b) and MTI (c,d) results for TR (- -), TGST (-.-), RGST (...) and RGST with selective median (-), measured for the frames 5 to 29 Uniform motion Non-uniform motion MSE MSE MTI MTI frame number frame number frame number

frame number Figure 11 Percentage MTI improvement compared to the TR method. ( R=Renata, M=Mobile, S=Shopping, RS=RenataSpeed, B=Bicycle, T=Tokyo ). TGST RGST RGST sel med

Page 16

• In the plots of ﬁgure 10 and 11 and table 1 it can be observed that the sequence Bicycle per- forms worst in all situations compared to the TR method. Bicycle is a sequence with com- plex motion, but without high vertical frequencies. It is therefore not surprising that even the RGST with selective median has no advantage without the median (in most cases), but has the disadvantage of not having a

median for all velocities. Consequently, there is not enough protection available. So, for complex motion sequences without vertical high frequencies, the TR method is advantageous. • The MTI performance indicator as plotted in ﬁgures 9c and 9d shows a signiﬁcant improve- ment in favor of the RGST with selective median for uniform motion sequences and on av- erage a similar score in case of non-uniform motion sequences. Since the MSE for the RGST with selective median outperforms the TR method, it can be concluded that this algo- rithm results in a signiﬁcant improvement

for de-interlacing. • It is also interesting to note that for the TR approach the MTI value is about 75% of the value, which indicates that the median ﬁlter, which is used to prevent errors for prop- agating, seems to be a must. (If the median ﬁlter would not be used, the MTI value would be half the value). • As shown in table 1, some sequences show an improvement in MSE and a worse MTI com- pared to the TR method. Since no thorough subjective evaluation has been conducted yet, it is too early for conclusions in these situations. • Generally, the TR method outperforms the TGST

method in case of non-uniform motion. • The TR algorithm performs, generally, also very well in terms of MTI, which was also ex- pected, since the recursive algorithm stimulates temporal consistency along the motion tra- jectory. This algorithm inherently also performs noise reduction, which also contributes positively to the MTI. • The ‘boosting ﬁlter’ as mentioned in section 4 is selected for near critical velocities, which are detected in the Mobile and Shopping sequence. As a result, the MTI increases. The se- lective median is a solution to solve this problem as also indicated in

ﬁgure 11. 6 CONCLUSIONS Two interesting algorithms for motion estimation and de-interlacing of Wang et al. [3] and Vandendorpe et al. [2] have been compared. The TGST was found to be superior (in terms of MSE ) for sequences with uniform motion, whereas the recursive algorithm of Wang was found to be superior for sequences with non-uniform motion. The MTI score is for the Wang approach in Table 1 MSE / MTI with respect to the TR method Sequence TGST RGST RGST sel median Renata + / ++ ++ / ++ ++ / ++ Mobile + / - ++ / - ++ / + Shopping + / --- +++ / - +++ / + RenataSpeed --- / - - / - ++

/ + Bicycle --- / --- --- / --- - / - Tokyo ++ / ++ ++ / ++ +++ / ++ MSE MSE

Page 17

both cases the best. The impact on subjective perception is yet unclear. The mutual weight of the MSE and MTI to the subjective perception remains to be investigated. The best aspects of both algorithms were joined in order to create a new motion estimation and de-interlacing algorithm, which does not demand for uniform motion nor suffer from error propagation. The RGST with selective median is found to be superior over the earlier methods, since the interlaced MSE was decreased in both cases with a

similar MTI for sequences with non-uniform motion (compared to the TR method) and a lower MTI for sequences with uniform motion. The robust approach of TR algorithm of Wang combined with the GST de-interlacing, which uses original samples only, seems an very interesting basis for further improvements. References [1] P. Delogne, L. Cuvelier, B. Maison, B. Van Caillie and L. Vandendorpe, ‘Improved Interpo- lation, Motion Estimation and Compensation for Interlaced Pictures’, IEEE Tr. on Image Processing , Vol. 3, No. 5, September 1994, pp. 482-491. [2] L. Vanderdorpe, L. Cuvelier, B. Maison, P.

Quelez and P. Delogne, ‘Motion-compensated conversion from interlaced to progressive formats’, Signal Processing: Image Communica- tion 6 , Elsevier 1994, pp. 193-211. [3] F.M. Wang, D. Anastassiou and A.N. Netravali, ‘Time-Recursive Deinterlacing for IDTV and Pyramid Coding’, Signal processing: Image Communications 2 , Elsevier 1990, pp. 365- 374. [4] G. de Haan and P.W.A.C. Biezen, Time-recursive de-interlacing for high-quality television receivers’, Proc. of the Int. Workshop on HDTV and the Evolution of Television , November 1995, Taipai, Taiwan, pp. 8B25-8B33. [5] G. de Haan and P.W.A.C.

Biezen, ‘Sub-pixel motion estimation with 3-D recursive search block matching’, Signal Processing: Image Communication 6 , Elsevier 1994, pp. 229-239. [6] J.L. Yen, ‘On Nonuniform Sampling of Bandwidth-Limited Signals’, IRE Tr. on Circuit Theory , vol. CT-3, December 1956, pp. 251-257. [7] A.A.C. Kalker, ‘Motion Estimation and Compensation for Interlaced Video’, to be pub- lished in IEEE Tr. on Image Processing. [8] G. de Haan, P.W.A.C. Biezen, H. Huijgen and O.A. Ojo, ‘True-Motion Estimation with 3-D Recursive Search Block Matching’, IEEE Tr. on circuits and systems for video technology Vol.

3, No. 5, October 1993, pp. 368-379.

Â© 2020 docslides.com Inc.

All rights reserved.