Video segmentation and Color space conversion!

Its been 10 days since my last post. Since then, I have started on the implementation of my project.

First thing I did was pull up a sample OpenGL-Cg implementation, using pass through vertex and fragment shaders in order to display an input image, rendered as a texture. This was done using the basic assignments that I had performed during the course and from the Cg Tutorial book.

Next step was to decide whether I should continue working on a single image and apply the various steps to this image to get the final stylized look, or should I get my hands dirty with the video file and get the segmentation component over with? By the segmentation component, I mean the framework to break up the video file into individual bitmaps, each of which will then be acted upon as single images and the process applied upon those.

So I figured that, since I had a simple application now, I should get it over with rather than worry over it once things get too complicated. So I started looking into the file format of an AVI file. I have chosen the input video file as an AVI file, since these are made up of bitmaps or more specifically DIBs (Device Independent Bitmaps). Another restriction was that the AVI file had to be uncompressed so that I didn't have to worry about compression and uncompression of the file. So after struggling through some pretty comprehensive information on the internet, I did manage to get my system in place. I used the inbuilt Vfw (VideoForWindows) C++ library for manipulating the AVI files and retrieving the individual frames from the file and getting the actual pixel data from the frames.

The next challenge was the conversion from RGB space to the CIELab space mentioned by the authors. This is a device independent color space designed to represent human vision more accurately.

This again was a nice thing to learn and it did involve some level of complexity. But I have managed to deal with it and gotten both the conversions working, i.e. RGB to CIELab and vice versa, which would be needed while converting the output back for display.

One issue that came up is that I have implemented this conversion on the CPU for now, and it has slowed down things. So as a next task, I will see if I can take this conversion onto the GPU using a fragment shader. It does seem possible. That will definitely speed up this step and aid all future stages.

In the next few days, I hope to get that done with soon and then start off on the actual implementation of the algorithms mentioned in the paper.

Until next time then!

The Amazing Cartoon Maker!!

Thats what I have named my final project for the CIS 665 - GPU Programming course. This is an implementation of the paper 'Real Time Video Abstraction' by Holgen Winnemoller, Sven Olsen and Bruce Gooch. This paper applies techniques from the fields of Non-Photorealistic Rendering and Image Processing for the purpose of realtime video and image abstraction and cartoon stylization. You can check out some more information on this here .

Ever since I took up this subject and started reading up on it, I have been pretty fascinated by the area of NPR and the way advances in GPUs have really led to better and efficient techniques of NPR. Hence I was always going to pick up on a topic related to this field for the subject of my final project here.

The choice of this name for the project came as a natural extension to what the final product promises to be... a tool which can be used for creating cartoon style videos from normal video files and even cartoon style images for normal image input.

The design doc has been created. It will be fairly important to lay a good foundation while starting off with the coding, so that the final packaged product looks good.

I am planning to implement this in OpenGL and Cg, because of its ease of use and flexibility and efficiency. For the UI, I am planning to use Qt from Trolltech, though I haven't used it before. I feel this will be a good time to learn this API, starting off with a simplistic interface for my product.

Signing off and wishing the very best of luck to all the other groups and individuals in their implementations too!!