[vtk-developers] GSOC "VTK Implementation of Eulerian Magnification for Revealing Subtle Changes" Proposal

Wed Mar 18 18:27:30 EDT 2015

Hi,

My name is Yingjie Luan (https://github.com/y1275963) and I am current a transfer student from University of Leeds in UK and back at home in China my home university is Lanzhou university. I am currently a year 3 undergraduate student studying electronic engineering. Since I was enrolled in the university I have been keen on programming and back at my home I was the elect member of open source society in Lanzhou University(https://oss.lzu.edu.cn/).   

I am currently targeting at the project called “Eulerian Magnification for revealing subtle changes”. The reasons why I love this project are the the following:

1. I am really interested in implementing all sorts of artificial algorithms. Just one year ago, I was luckily allowed to work at the mathematical laboratory at my home university during the winter holiday. And at that time I spent a great deal of time studying implementing artificial algorithms like BP Neural network, Generic Algorithm and simulating annealing. I love to read papers and documentations and write code.
2. I have C++ background. Just one year ago, I implemented a laboratory management system using C++ and Qt.(https://github.com/y1275963/FinalTerm) So I think I am comfortable with C++.
3. I have VTK background. Just at this semester, I was luckily enrolled in Pro. David Duck(https://www.engineering.leeds.ac.uk/people/computing/staff/d.j.duke)’s module called “Scientific Visualization” within which we are discussing a way to visualize data from early universe using VTK. And Pro. David Duke is the developer of the VTK class called “vtkSpanTreeLayoutStrategy”. 
4. The most significant reason why I think I am suitable for this position is that I think I am a dedicated person and I am self-motivated. I asked Pro Wang to allow me to work at his mathematical laboratory. Although I am not allowed to be enrolled in the C++ module back at my home university, I managed spent my spare time to attend all my professor’s classes and finish his final project just as same as his enrolled student( Or even better because I leant Qt at the same time). The module called “Scientific Visualization” is for master student but after consulting Pro. David I was allowed to be enrolled and I think my performance is not worse than those master students. I think as long as I am willing to devote all my time onto this project I will be fine.

The rough idea about the implementation:

After read though the paper and its related resources, I have got the idea about this algorithm. It goes like this:

1. First, I need to develop a way to construct spatial filter like Gaussian pyramid or Laplace pyramid. 
2. Second, after I have decomposed the video in spatial domain I then apply temporal filter onto it. (like ideal filter, Butterworth filter or second-order IIR filter) 
3. Third, I then amplify the filtered signal according to the description in the document. Basically what I need to do is to avoid huge magnification errors by setting approximation limit.
4. After adding the original signal and the amplified signal back together and collapsing the the image pyramid we should get the desired video.

The technical difficulties:

1. We only have a Gaussian low pass filter and we don’t have filter to construct Gaussian pyramid or Laplace pyramid.( We have one in ITK)
2. The paper suggested us to use YIQ as color space, we need a filter to transfer the image into YIQ space.
3. This filter is the combination of temporal and spatial decomposition and I need to find a way to link them. 
4. For visualizing motion the writer suggested a method called "phase based video motion processing” which is another topic.
5. Provide users some guide lines in determining filter parameters.
6. VTK requires threading process over images and hopefully I can do some things with parallelize.
7. To be honest, my biggest concern is how to build VTK filter and that is area I am going to work really hard onto(Like how to implement Requestdata function, how to work with images or even time varying images, where should my filter inherent from)

My rough time table goes like this,
1.  I decide to go before the start of the project and I hope by the 25th May I can get a functioning model using scripting language like MATLAB and I can get myself familiar with the VTK environment(Especially, writing VTK filter for time-varying processing)

2.  I will do my best during the first term of the GSOC. And within that period I first will do as best as I can to implement the Gaussian pyramid, Laplace pyramid and YIQ converter. After that, I will combine the designed spatial filter with the pre-existing temporal filter and FFT in VTK to build a functioning model in VTK. And I guarantee I can finish this before 3rd July. 

3. After I have got the functioning model, I then can do some advanced topics in the second part of my term, like supporting parallelize, implementing phase-based pipeline or write a example about this filter( This is important because we need to give user guide line in terms of how to select the filter parameters, like amplify ratio and what kind of schema). 

My questions in terms of this paper:

1. I dug into the MATLAB code provided and I am right now trying to implement the simplest one which is gaussian pyramid along with ideal filter, and I saw it goes like this:

   % Temporal filtering
    disp('Temporal filtering...')
    filtered_stack = ideal_bandpassing(Gdown_stack, 1, fl, fh, samplingRate);
    disp('Finished')

    %% amplify
    filtered_stack(:,:,:,1) = filtered_stack(:,:,:,1) .* alpha;
    filtered_stack(:,:,:,2) = filtered_stack(:,:,:,2) .* alpha .* chromAttenuation;
    filtered_stack(:,:,:,3) = filtered_stack(:,:,:,3) .* alpha .* chromAttenuation;

That is, the writer just filter it out and directly amplified them, but in the paper we should do .  Is this true we just assume motion signal is within the passband of the temporal bandpass filter that after applying the filter what we get is ? So we don’t need to explicitly calculate the spatial difference and to find the motion function.

2. The color magnification is fairly easy, I still need some work to understand the maximum application ratio theory for movement magnification.( Like applying different ratio for each spectral components of the movement signal)

3. There is no ideal bandpass filter in VTK should I use vtkWindowedSincPolyDataFilter or vtkSmoothPolyDataFilter or implementing my own?( I think those two filter are all for smoothing and is not for frequency filtering)

4. Am I on the right track?

In summery, I hope I can be enrolled in this project and I will do my best not to let my mentor down. I have set up my blog at http://y1275963.github.io/ and my current progress is that I have finished the paper and I am digging into how to implement the pyramids. And I will write a post on my blog as soon as I got the result. I appreciate your valuable time in reading my proposal and I am looking for your feedback.

Yingjie

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20150318/29a3676f/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 2015-03-18_22-00-52.png
Type: image/png
Size: 7578 bytes
Desc: not available
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20150318/29a3676f/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 2015-03-18_22-02-24.png
Type: image/png
Size: 6710 bytes
Desc: not available
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20150318/29a3676f/attachment-0003.png>