I spent some weekend time working on one of those Crazy Things That Just Might Work: a JavaScript cross-compile of the Ogg Theora video decoder. Very primitive proof of concept demo is live (no audio, no sync, no color, …) but it works better than I figured after a day’s worth of hacking!
Larger files run… veerrryyyy sslllooww on my test iPod Touches, but this certainly seems fast enough on desktop to one day replace our old Java fallback for Safari and newer IE…
How it works
The C libraries libogg, libvorbis, and libtheora are cross-compiled from C to JavaScript using emscripten, a super-awesome tool that builds via the LLVM clang compiler and provides a mostly C-compatible environment within JavaScript with surprisingly good performance.
A thin C/JS wrapper layer accepts input data from the JavaScript side, lets it be processed by the converted C code, and then outputs to an HTML canvas element on the web page.
Only a couple of tiny tweaks to the libraries are needed to make them build; I started with build scripts for just the audio codecs from this project, added in libtheora, and started adapting parts of one of the Theora data dump examples.
Possibilities
Finish up the YCbCr->RGB conversion, add audio decoding & output, and some kind of sync and seeking, and …… this could replace the old Java Cortado app we use as a fallback player on Wikipedia for browsers that don’t run WebM or Theora.
Web Workers could be used to push decoding to a background thread, depending on whether overhead is problematic.
Crazy idea: provide an HTML5 <video>-style DOM interface, integrate into TimedMediaHandler as a drop-in replacement
Limitations
- Audio sync may be difficult to achieve.
- Audio output APIs — need to confirm what’s consistently available.
- Performance is surprisingly good on a desktop; I have no doubt this will be sufficient for playback in Safari and IE if audio & sync can be managed.
- Performance is not so good on my test iPod Touches; it might be fun to tune and optimize but I would expect to get much better results from a native app on iOS.
- There doesn’t seem to be a good universal way to do progressive data reads from an XMLHttpRequest; it may be necessary to buffer portions of the file by running multiple requests for subranges of the file, which is NOT pretty.
- IE 10 doesn’t support ArrayBuffer.slice(). Currently this prevents the demo from running, but it’s not actually needed.