Hypervideo

In the process of producing DVD videos, I found the way DVD menus are organised to be, frankly, rather annoying. DVD videos consist of nothing but MPEG-2 program streams (combinations of video, audio, subtitles, etc.) and some small amount of metadata about those streams. Menus are encoded as program streams that have changeable subpictures (these are a generalisation of subtitles) and bytecoded routines that are called when the user presses a button. There are various special tools for designing ("authoring") DVDs, which abstract away some of the detail of this, but I don't think any of them can easily be scripted. I pondered aloud on #debconf-video that, given that this is basically hypermedia, it ought to be possible to design a DVD as HTML pages with links to video files and to each other, then run a conversion program to deal with the fiddly details. There are many tools available for writing HTML, and far more people know how to use them.

So I tried writing that conversion program. I have no intention of writing my own HTML renderer, so I decided to embed Mozilla. Unfortunately Mozilla insists on rendering to a window rather than an off-screen bitmap, which meant I also needed to work out how to spawn and kill Xvfb (X server using an off-screen bitmap) and direct Mozilla to create its window on that server. Then I had to work out how to find and follow links, tweak them into their "hover" and "active" state and record their changes in appearance, and do some basic image processing to convert those changes down to the small number of colours that a subpicture can use.

This involved a fair amount of learning (Mozilla interfaces, Gtk, Gtkmm, Gdk-pixbuf, and X authority files), but with my existing knowledge of C++ and some past experience with image processing it's gone quite fast. This evening I finally got some HTML pages showing on my DVD player. The links don't work properly, and I need to do some serious work on default styles (for instance, there needs to be a lot of padding around the BODY since video frames extend beyond the edge of the visible TV screen). But I actually have something to show after only a week of spare time.