Progress

This describes the progress of the project

Week 1 12 - 18 August

This week the principal work done was reading and understanding the reading provided to me. Also I have started working on the white paper and am currently searching for the configuration of a new powerful desktop which is needed in this project. Below is the list of readings I managed to finish during this week.

1. 3WNews: Who, Where and When in News Video
2. AUTOMATIC CONTENT-BASED ORGANIZATION OF VIDEO SEQUENCES FOR MULTIMEDIA APPLICATIONS
3. A systems architecture for ubiquitous video
4. An image and video search engine for the world wide web
5. An indexing and browsing system for the home video
6. i.plot
7. Media Mirror
8. MediaMetro: Browsing Multimedia Documents collections with a 3d city metaphor

Also while browsing through the internet I came across Microsoft's latest project Microsoft Surface which can be browsed at the website
http://www.microsoft.com/surface/

I will be updating the summaries of the articles over time and try to co-relate it to this project.

Also I was thinking over the steps i need to follow for the completion of the project and these are the steps which i think i need to follow

Step 1: Create a 3D virtual universe
Step 2: Embed Pictures and Enable Zooming
Step 3: Make the motion of the Pictures Dynamic
Step 4: Make the Search Real time

After this step I should just be able to replace Pictures with Videos and the System would be 80% ready
Step 5: Replace photos with videos ( OOP will help in this step)

Step 6: Build a user interface
Step 7: Try to link the search to an online website like youtube/google/flicker
Step 8: Testing and report writing

The steps mentioned above are just what came to my mind during this week. I might have missed something or might have stated something which might take more time than I have so the steps will be modified surely during the course of this project.

Week 2 19-25 August

I continued the readings this week and here are the readings that I completed this week

1. Collages as Dynamic summary for news video
2. Image Mosaics
3. Jigsaw Image Mosaics
4. Interactive Mosaic generation for video navigation
5. Mediating photo Collage authoring
6. Mosaic based representation of Video sequences and their applications
7. Photo Tourism: Exploring Photo collections in 3D
8. Quantum treemaps and bubblemaps for Zommable Image Browser
9. Rapid Serial Visual Presentation Techniques for consumer Digital Video devices
10. Simulating Decorative Mosaics

Also during the week I went to Sim Lim and found out more about the recent machine configuration which I might need for the project. I got a quotation from various shops there and the one which was the cheapest I have listed it down already in the desktop configuration system. Also I installed Visual Studio on my laptop but it has been hanging my laptop and not working properly probably because i have all the video and photo editing software on my laptop. I will uninstall the other softwares and then reinstall Visual Studio again over the weekend to see if it works properly.

Week 3 26th Aug- 1st Sep

This week I have finished all the readings except one and will be starting the programming for the project on my Laptop. Visual studio seems to be working fine but sometimes it still hangs my system. Also while searching on the internet I found this interesting article on MPEG 4 and its Visual representation

The article is the one at the link below
www.img.lx.it.pt/~fp/artigos/PCS96.doc

This week I could not work that much on the FYP programming as have been attending Recruitment talks nearly every evening.

Week 4 2nd Sep - 8th Sep

This week i started the revision of OPEN GL and tried programming a few simple programs. While trying out the problems i developed a program for photo manipulations. The code of the program is in the program snippets page. However my computer gets hung upon executing the program.

During this week I revised the following Open Gl topics
1. Introduction
2. Graphics output primitives

Week 5 9th Sep - 15th Sep

This week i continued my Open GL revision and made another few programs. I also have developed one basic program which allows user to move in space using keys. In this program the camera is continuously moving. I am aiming to build on this for this program as this would give an impression of the user moving continuously through space. The program is in the code section

Also the topics i managed to revise this week are
3. Attributes of Graphics Primitives
4. Geometric transformations
5. 2-D viewing (partially)

Week 6 16th Sep - 22nd Sep

I continued my revision of Open Gl and worked on the algorithm for my program for the project. I drew a rough flowchart with which I can break the program into smaller programs and try them out first and perfect them before I integrate everything together. Also I have started writing my thesis, just wrote a bit of introduction for it this week. Will work on the thesis and the presentation during the mid term break

Week 7 1st Oct - 6th Oct

The presentation was ok. However I realized that there is a lot to be done still. Also I will now be working with Anne till December and we will be Co-ordinating to complete the project. I continued my revision of Open GL and tried to find out about rendering video in open GL. I have started working on creating a 3D world of Photos which should be ready in about two weeks. I will be updating the content of the program in the code section.

Week 8 8th Oct - 13th Oct
I finally completed my Open Gl revision and now am focusing solely on rendering videos. Over the week I found out that there is a language called Delphi which has been used to program Quake 3 and some other games. Although the language looks powerful and has Open GL interface but it is not that well know. So I will be consulting people next week whether it is probable to program in Delphi. I also was going through the Nehe game development website and found out a code for rendering videos. However it is not executing on my system and is crashing.

I also found out a few examples in which DirectX has been use to render videos. I will be researching more in DirectX as it appears that it reduces the computational requirements as compared to OpenGl.

Also I have been in touch with Anne and she will be beginning coding this week.

I have been a bit loaded with tests and submissions of other modules so have not made great leaps in programming but will be covering all up in the next week

Week 9 15th Oct - 20th Oct

Finally the code rendering videos on nehe website runs on my laptop though i have no idea why it still crashes. While playing around with the code i found out that the code is stable for large videos and can play large videos. However there is one problem it does not plays sound. So I have been researching to check how to play audio now. Till now have not found much but hopefully i will be able to find something which will play audio otherwise will have to use directx to code which would mean learning the new language and coding within the next 3-4 weeks.

Week 10 22nd Oct - 27th Oct

I have been trying to edit the code on Nehe website but on editing it simply does not work i think i am bungling up with opening the graphics streams to play AVI. I was also trying to learn a bit of directx this week and read 2 chapters.
It took me the entire week to figure out how to actually open and close streams and i feel that the nehe code is not well written. On further research I came across a research paper which explains in detail how windows OS plays video. It also has the explanation how C++ can be used to play videos. I will be focusing on that to write a new code which will help me play both audio and video for a video at the same time. Hopefully this way i will be able to use OpenGL only and will not have to switch to directX.

Week 11 29th Oct -3rd Nov

I wrote the code for rendering a video stream this week. And it works Finally after working for 3 days continuously I managed to render a video. I now understand a bit how the basic video rendering works in opengl. It requires a bit of assembly coding to do so. Also I found out that there are other methods to do the rendering but I am not sure which one is the best. For now I will concentrate on the one i have developed and will try to render multiple videos together.

Week 12 4th Nov - 9th Nov

This week i came accross a major block. I am unable to render multiple videos. I open the video stream for one video and when i try opening it for more than one it crashes because the frame rate is different. The pointers are probably not able to simultaneously open multiple videos. I have talked to Janaka and requested him if he find out more about rendering multiple videos simultaneously. Also as I have exams from next week so I will be unable to give as much time for FYP as i will be concentrating more on my modules. However I will continue researching about opening multiple videos simultaneously.

Week 13 11th Nov - 16th Nov
Week 14 18th Nov - 23th Nov
Week 15 25th Nov - 30th Nov

The past three weeks have been examz and interviews so have not been able to work on my Final Year project. Will start from December 2nd

Week 16 2nd Dec - 7th Dec

This week i resumed working on rendering videos. I brought books on the windows operating system to understand how windows plays videos.

Week 17 9th Dec - 14th Dec

This week i spent time mostly on compiling and integrating all the programs and preparing for my CA2 presentation.

BREAK

I am going back to India for sometime so will be working from back home in India on the comments i received during my CA2. My main target for this break is to work on the interaction interface of the video browser.

Week 1 Semester 2

I just came back a few days ago and started work from where i left. Over the holidays i wrote the code for the basic interface window of the browser and tinkered with the playing of videos. I finally found out that even though the directx might be easier to implement it is more memory intensive for the video rendering. As a result i will stick with C++ method using inbuilt AVI32 application of windows.

Week 2 Semester 2

Damn my laptop crashed last weekend and I lost the previous 3 weeks of work (my last backup was in 3rd week of Dec 2007). I went to Simlim to check for the recovery of data but it is too expensive and as a result i will redo the work i did over the winter break…. Anyways i did the CA2 for the lab the other day and have got a few more ideas and comments to work on for my project

Week 3 Semester 2

The video rendering code was behaving strangely this week. Sometimes it was playing multiple videos and sometimes it was crashing. I passed my code to Janaka and he finally managed to find the problem which has now enabled me to render multiple videos stably. I had to declare a few variables globally in one of the classes for the code to function properly. I will now concentrate on the optimization. Will check how many simultaneous videos can I play.

I also found one more thing which disheartened me. I cannot separately use the 4 cores of the Quad core processor. The windows OS wont allow me to do that (got this info from one of my other professors) Now my main target is Optimization for the video rendering.

I was thinking that if I am unable to render too many videos using the current method then i will create a structure in which i will pull few frames from start, middle and end of the video and render them. It should solve the problem.

Also Anne sent me her code last week and I am studying that and will try to combine it into my existing code.

Will upload a few pictures and probably a video of the project the entire thing gels together. Should be done within February I guess.

Week 4 Semester 2

With Janaka's help I finally managed to render many videos simultaneously. Now I will try and see how many videos and how big videos can I render simultaneously.

Also I created a basic layout of the screen. It is looking ok but I will improve it in the coming weeks.

Week 5 Semester 2

I found out that despite the size of the videos maximum 10 videos can be played using this program. After 10 videos it gives a memory overflow error.

I talked to professor and he told me that perhaps a video server can be created to solve my problem.

I will be looking into the optimization and the use of a video server or perhaps of a structure to render the videos.

Week 6 Semester 2

Being the mid-term week I was a bit occupied with the Citibank case challenge, however I managed to create another version of 3D world where the user is not continuously moving but moves using the keys awsd like in a video game.

Also there are many video server technologies available and from what i can gather it is the codec that is important. So I will be trying to understand and get some open source video server program to try out.

Also Janaka showed me a prototype of a video browser being developed in which the video only plays when the mouse is over the video. I was wondering perhaps this could be done.

Week 7 Semester 2

Update 1:

Finally I can render as many videos as I want. It was a bit of a bug in my code. However there is a problem the physical memory usage is equal to the size of the videos being played. So the system becomes very slow when I use about 20 videos. Thus either I have to use a server to pass on the videos or perhaps use a container to render a part of the videos.

Also I made modifications to the 3D world. Now the user view is not automatically moving but instead it is stationary with the user having full control over the movement. the movement is as below

a- Left
d- right
w- forward
s- backward
left arrow - move view left
right arrow - move view right
up arrow- move view up
down arrow- movie view down

Also the user can use the mouse to move his view, by pressing left click and the moving the view to the desired location!!

Update 2:

For whatever servers i have found I can create a server but the thing is there is no clue in it how to integrate it with a c++ code. I am lost as to how to do that….I have emailed one of my professors to discuss the problem, and hopefully he will get back to me.

Also I thought of using a different program to do the container creation for me..and Janaka passed me the mutex code, which I will be studying over the weekend. Also I will be working on the portion of selection of a particular object and zooming out this weekend.

The 3D browser which Janaka showed me has been built in flash and I think i can also create that in 2 weeks as its much much easier to create a 3D browser in flash but I am unsure about its capability and feel that my current project is a better manifestation of the idea of the 3D browser

Week 8 Semester 2

The professor was unable to help me solve the problem as he hasn't done multiple video rendering. Also I worked with janaka to display videos using a different method. This method though a bit better than the previous one has the same problem of very high memory usage.

I also noticed that the amount of virtual memory is equivalent to the the size of the video loaded in the memory. So the bigger the videos loaded the bigger the memory usage and slower the system.

Also i implemented the selection of videos this week.

Week 9 Semester2

Update 1:
After the discussion with professor, we realised that this method of video rendering is not appropriate since it does not makes the browser scalable. So instead this week my focus is on using vlc to try and render videos using a server.

Update 2:
I am unable to integrate the VLC server into my code because of its difference output methodology. Alternatively I will be converting videos into a stream of frames and render videos using frame rendering. The only problem being that vlc converts video to images in .png formats

Update3:
Janaka and I managed to find a way to render .png images in openGL. I created a mosaic of multiple videos being played by continuous image rendering. In this scenario I can render around 35 videos before the video frame rate goes down…i.e the virtual memory usage is not high but the video is being played at a slower pace at sometime.

Update4:
After discussion with professor I tried finding a way to cache frames to keep the video playing speed normal. However there is a hurdle in buidling a cache. DevIL the library i am using to render .png images has a restriction that it only binds and unbind one image at a time. If i bind multiple images then only the latest one is accessible and the earlier ones cannot be accessed. Thus I would discuss with professor and Janaka on how to solve our problem.

Update5:
I was unable to go to lab yesterday but Janaka discussed our problem with the professor and we have come up with an alternative to solve our problems. I will be coding this method and should get it done by Tuesday.

Week 10 Semester2

Update1:
I am now rendering the videos as discussed with Janaka, I render videos in zones. The first zone has videos being displayed using the normal win32 API. The second zone is the zone in which the videos are displayed using continuous images. I convert the videos into images using the vlc player and then use those images to display video in the 3D world.
As soon as the user passes the mid-point of the second zone the videos of zone one stop rendering and new videos using the win32 api are displayed in new zone after to zone 2.

Also the videos have slow motion and are not static in nature.

Update2:

After discussion with professor and Janaka, I realized that I had visualized the hybrid method differently. Instead of having zonal videos I need to map the rendering of videos using the distance from user. A particular video when far away from the user is displayed using the .png method and when near the user is displayed using the win32 API. I will be using flags in my functions to implement this method.

Also we had a presentation today for the year 2 and year 3 engineering students where I got feedback on my project. People were quite interested in the project and wanted to inquire more about it. Also they asked about the computer usage and were quite impressed with the new hybrid method we are using for displaying the video.

I will be working on the aesthetics of the project next week and on the Thesis which is due next Fri.

COMPLETION

Dear All,

This project has been completed and submitted. Thus I will not be updating this website further.

Regards

Anshul

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License