# MVTools

## Abstract

author: Manao
version: 0.9.5
download: http:/ /jourdan.madism.org/~manao/
category: Misc Plugins
requirements: YV12 Colorspace

## Table of contents

- I) About MVTools
- II) Function descriptions
- III) Examples
- IV) Disclaimer (don't skip that part, but I don't force you to learn it either)
- V) Revisions

## I) About MVTools

Collection of filters (MVBlur, MVConvertFPS, MVDenoise, MVInterpolate, MVMask and others) which uses motion vectors generated by this plugin.

Note. There are more recent MVTools versions with some new and improved functions, but without motion interpolation functions (MVBlur, MVConvertFPS, MVInterpolate).

## II) Function descriptions

### Common parameters

The filters which use motion vectors have common parameters. They allow for example to set the scene change detection thresholds. Moreover, filters may need one or two motion vectors stream. When two are needed, one is backward ( obtained by `MVAnalyse`(isb = true) ), and the other is forward ( `MVAnalyse`(isb = false) ). By convention, the backward one has to be given first.

mvsbw : the backword motion vector

mvsfw : the forword motion vector

int thSCD1 : threshold which decides whether a block has changed between the previous frame and the current one. When a block has changed, it means for me that motion estimation for him isn't relevant at all. It occurs for example at scene changes. So it is one of the thresholds used to tweak the scene changes detection engine. Default is 300. Raising it will lower the number of blocks detected as changed.

int thSCD2 : threshold which sets how many blocks have to change for the frame to be considered as a scene change. It is ranged from 0 to 255, 0 meaning 0 %, 255 meaning 100 %. Default is 130 ( which means 51 % ).

### MVAnalyse

`MVAnalyse` (clip, int "bw", int "bh", int "pel", int "lv", int "fth", int "sadt", int "mpt", int "st", int "stp", bool "isb", int "lambda")

bw, bh : They are the width and the height of a block. Value can only be 4 or 8. Larger blocks are less sensitive to noise, are faster, but also less accurate. Defaults are 8 for both.

pel : it is the accuracy of the motion estimation. 1 means a precision to the pixel. 2 means a precision to half a pixel, 4 to a quarter of pixel and so on. Value can only be 1, 2, 4 or 8 ( beyond 8, it is useless and waste a lot of memory ). Default : 1. For the moment, it's the only possible value.

lv : it is the number of levels not used in the hierarchal analysis made while searching for motion vectors. The lower the better. It is kept variable for study's purposes only. Default : 0.

fth : Not used anymore, made useless by lambda.

sadt : Not used anymore, made useless by lambda and the speed gain.

st, stp : st decides the type of search, and stp is a setting for this search :

- st = 0 : 'OneTimeSearch'. stp is the step between each vectors tried ( if stp is superior to 1, step will be progressively refined ). It's the default search, with stp = 1.
- st = 1 : 'NStepSearch'. N is set by stp. It's the most well known of the MV search algorithm.
- st = 2 : Logarithmic search, also named Diamond Search. stp is the initial step search, there again, it is refined progressively.
- st = 3 : Exhaustive search, stp is the radius. It is slow, but it gives the best results, SADwise.

isb : allows to choose between a forward search ( between the current frame and the previous one ) and a backward one ( between the current frame and the following one ). isb = false means forward search.

lambda : set the coherence of the field of vectors. The higher, the more coherent. However, if set too high, some 'true' motion vectors can be missed. Default is 0, but values around 2000 are strongly recommended and will be made defaults in the next release.

### MVBlur

`MVBlur` (clip, clip mvsbw, clip mvsfw, int "nb", float "rl", int "fbw", string "wf")

Allows to create a motion blur. The motion blur is created by interpolating frames between the current frame and both next and previous frames, and then make a weighted mean of these interpolations.

nb decides how many interpolations are made. The more, the better, and the slower. rl is equivalent to the speed of a camera's shutter. The higher, the stronger the motion blur. It's unit is in fps. fbw allows to increase the size of the blocks that are moved along the motion vectors. For example fbw = 2 will increase the block by a 2-pixels margin. This allows less artifacting, but leads to a slightly less imprecise motion blur ( it will be less directionnal ). Finally, wf controls the weighting of the interpolations. Severals strings are possible here :

- wf = "uniform" : Uniform weighting ( all the weights are equal to 1 ).
- wf = "ramp" : the weight is proportionnal to the time between the interpolation and the current frame.
- wf = "revramp" : it's the contrary.
- wf = "hat" : it begins with a ramp, and finish like a revramp.
- else : it's a revramp.

Defaults are : nb = 10, rl = 0.5, fbw = 2 and wf = "uniform".

### MVCompensate

`MVCompensate` (clip, clip vectors)

Do a full motion compensation of the frame. It means that the blocks pointed by MV in the previous frame ( or following, it depends on the setting of `MVAnalyse` ) will be moved along the vectors to reach their places in the current frame.

It differs from filters like MVinterpolate because here, the rebuilding is complete ( meaning by that that the blocks once moved aren't overlapping, and cover the whole picture ).

For the moment, it doesn't move the chroma ( because it needs a spatial interpolation algorithm to do so ).

### MVConvertFPS

`MVConvertFPS` (clip, clip mvbw, clip mvfw, int "nb", float "rl", int "fbw", int "sbw", int "rbw", int "bvt" [, int num, int den], float "fps", string "wf")

Will change the framerate of the clip, without changing its length. As with `MVBlur`, nb is the number of frames built in order to create one interpolate frame. rl camera's shutter speed and wf is the normalization function. fbw, sbw and rbw allows to increase the size of the blocks moved. bvt is a threshold which decides whether a motion vector is valid or not. The default value makes all vectors valid. num and den are the numerator and the denominator of the new framerate. You can also give this number using the fps parameters.

Defaults are : nb = 1, rl = 0, fbw = 4, sbw = 0, rbw = 0, bvt = 35000 and wf = "uniform".

### MVDenoise

`MVDenoise` (clip, clip mvbw, clip mvfw, int "thT", int "sadT")

Makes a basic temporal denoising with motion compensation. Both previous and following frames are motion compensated and then merged into the current frame. The two thresholds are used to prevent artifacts.

The first one, thT, decides whether the pixel which comes from the previous or the following frame has to be taken into account. If this pixel differs more than thT from the pixel of the current frame, it is not used.

The second one, sadT, decides whether the block has changed or not ( same principle as thSCD1 ). If it has changed, the pixel aren't merged with those of the previous or following frame.

Defaults are : thT = 10 and sadT = 160.

I would say ( without much testing ) that :

- thT = 5-10, sadt = 100-200 : weak denoising
- thT = 10-15, sadt = 200-300 : medium denoising
- thT = 15-25, sadt = 300-500 : strong denoising

### MVInterpolate

`MVInterpolate` (clip, clip mvbw, clip mvfw, int "nb", float "bl", float "el", int "fbw", int "sbw", int "rbw", int "bvt", string "wf")

Allows to shift temporally the video from a non integer number of frames, between 0.0 and 1.0. Works the same way `MVBlur` and `MVConvertFPS` do, which means nb, fbw, sbw, rbw, bvt and wf work the same way. bl and el define the beginning and the end of the interpolation. If bl is different from el, the created frame is slightly blured. The interpolation is made between the current frame and the previous one.

Defaults are nb = 5, bl = 0.4, el = 0.6, fbw = 4, sbw = 0, rbw = 0, bvt = 35000 and wf = "uniform".

### MVMask

`MVMask` (clip, clip vectors, int "ml", float "gamma", bool "showsad", int "Ysc")

Creates a motion mask from the motion vectors' length. It builds a better mask than MotionMask ( MaskTools ) because motion vectors are a lot more reliable than the algorithm of MotionMask.

The mask is output only on the luma plane. 0 means no motion at all ( the length of the motion vector is null ), whereas 255 means maximum motion, and is defined by ml. When the vector's length is superior or equal to ml, the output value is 255. gamma is used to defined the relation length - value. gamma = 1.0 implies a linear relation, whereas gamma = 2.0 gives a quadratic relation.

showsad allows to build a mask of the SAD values instead of the vectors' length. And finally, Ysc is the value taken by the mask on scene change

Defaults are : ml = 100, gamma = 1.0, showsad = false and Ysc = 0.

### MVShow

`MVShow` (clip, clip vectors, int "scale", int "sil", int "tol', bool "showsad")

Shows the motion vectors. scale allows to enlarge the motion vectors, in order for example to gain in accuracy ( when pel > 1 and scale = 1, you can't see a variation of less than one pixel ). sil allows to see a different level of analysis ( when searching for motion vectors, a hierarchal analysis is done, and it may be interesting to see what happens at higher levels ). tol is a tolerance threshold. If the distorsion induced by the motion vector is over tol the vector isn't shown. Finally, showsad allows to show the mean SAD after compensating the picture.

Defaults are : scale = 1, sil = 0, tol = 20000 and showsad = false ( which shows all vectors ).

## III) Examples

To show the motion vectors ( forward ) :

    vectors = source.MVAnalyse(isb = false)
    return source.MVShow(vectors)

To show the backward one :

    vectors = source.MVAnalyse(isb = true)
    return source.MVShow(vectors)

To use MVMask :

    vectors = source.MVAnalyse(isb = false)
    return source.MVMask(vectors)

To use MVBlur :

    vectorsfw = source.MVAnalyse(isb = false)
    vectorsbw = source.MVAnalyse(isb = true)
    return source.MVBlur(vectorsbw, vectorsfw)

It's the same for `MVInterpolate`, `MVDenoise` and `MVConvertFPS`.

To denoise :

    backward_vectors = source.MVAnalyse(isb = true, lambda = 2000)
    forward_vectors = source.MVAnalyse(isb = false, lambda = 2000)
    return source.MVDenoise(backward_vectors, forward_vectors, tht = 10, sadt = 300)

A very slow, but also very good motion blur:

    # Assuming 25fps material. Works very well for my CG material.
    vec_back = source.MVAnalyse(lambda=2000, isb=true)
    vec_forw = source.MVAnalyse(lambda=2000, isb=false)
    MVConvertFPS(last, vec_back, vec_forw, fps=100, fbw=2, thSCD2=255, thSCD1=200)
    TemporalSoften(2, 255, 255, 0, 2)
    SelectEvery(4,0)

Motion compensated 60i to 24p conversion:

    function KernelBob(clip a, int "ord")
    {
      f = a.KernelDeint(order=ord)
      e = a.SeparateFields.Trim(1,0).Weave().KernelDeint(order=1-ord)
      Interleave(f,e)
    }

    function MVConvert60ito24p(clip x, int "mode")
    {
    mode = default(mode, 2)
    mbl = 0.1

    vectorsforward = x.MVAnalyse(isb = false)
    vectorsbackward = x.MVAnalyse(isb = true)

    y = x.MVInterpolate(vectorsbackward, vectorsforward, nb = 4, bl = 0.5 - mbl, el = 0.5 + mbl, wf = "hat")

    Interleave(y, x)
    mode0 = SelectEvery(5, 2)
    mode1 = Overlay(SelectEvery(5,3),SelectEvery(5,2),opacity=0.5)
    mode2 = Overlay(Overlay(SelectEvery(5,1),SelectEvery(5,3),opacity=0.5),SelectEve
    ry(5,2),opacity=0.3)
    mode3 = Overlay(Overlay(SelectEvery(5,0),SelectEvery(5,3),opacity=0.5),Overlay(S
    electEvery(5,1),SelectEvery(5,2),opacity=0.5),opacity=0.5)

    (mode==0) ? mode0 : (mode==1) ? mode1 : (mode==2) ? mode2 : mode3
    }

    x = source.KernelBob(ord=0)  # or any other bobber
    MVConvert60ito24p(x)

## IV) Disclaimer

This plugin is released under the GPL license. You must agree to the terms of 'Copying.txt' before using the plugin or its source code.

## V) Revisions

0.9.5

- Huge rewritting of the core engine. Vectors are now searched with a precision up to the pixel ( because no other filters can use yet more precise vectors, except MVShow ). The search engine is now fast ( which doesn't mean necessarily that the filters which use it are fast )
- A new parameter for MVAnalyse : lambda. See the documentation of the filter to see how it works
- MVDenoise works better.

0.9.4

- Vectors can be saved to a file. In order to do so, add the parameter filename="C:\foo.bar" to the filter MVAnalyse. If the file doesn't exist, vectors will be saved into it. If it exists, vectors will be read from it. But, be warned :
  - The file for a whole movie will be around 500 MB
  - Saving / reading from a file need for the moment a linear access to the frames, so it has to be used only when encoding the movie, not when doing random access in it.
  - The speed gain is not as great as one may hope, because SADs can't be saved ( it would take too much space ) and so have to be recomputed.
- The filter MVDenoise now works on 5 frames, and its parameters are now "thT" and "sadT" ( have a look in the documentation to see how they work ). It works nice ( very good for heavy denoising )
- The scene change detection thresholds have slightly changed. Now, a block has changed if its SAD it over thSCD1. The default for thSCD1 is 300, and for thSCD2 it is 130. It orks well ( better than the previous SCD engine ).

0.9.3

- Last cleanings in the search of the motion vectors. It should be slightly faster
- More search parameters can be set by the user, especially the search algorithm. See the documentation
- Server / client implemented. You now first have to use MVAnalyse, and then the filter you want. Look at the documentation and at the examples I'll give alter.
- MVCompensate is separated from MVShow ( it's more logic that way ). For the moment, it doesn't move the chroma ( same behavior as MVShow in the latest releases )
- Some cleaning in MVBlur / MVInterpolate / MVConvertFPS, but still some work to do. Now, MVBlur blurs around the frame, not between the frame and the previous one.
- Half of the work is done for writing vectors to a file. But the resulting file will be large ( around 500 MB - 1 GB I guess ).
- MVDenoise is slightly faster ( at least it should )
- Copies are optimized inside the filter, thanks to avisynth's copy functions.
- MVShow can display the mean SAD of the compensation ( using showsad = true )

0.9.2.1

- MVInterpolate makes its come back.
- MVConvertFPS should work on the last few frames of the clip

0.9.2

- MVInterpolate doesn't exist anymore.
- MVBlur and MVConvertFPS have been improved. They also have got new parameters, have a look at the documentation.
- MVShow gets back its compensate mode ( MVShow(cm = true) )

$Date: 2005/10/03 16:31:31 $