I’ve often wished that for ogv.js I could send my raw video and audio output directly to a “real” <video> element for rendering instead of drawing on a <canvas> and playing sound separately to a Web Audio context.
In particular, things I want:
- Not having to convert YUV to RGB myself
- Not having to replicate the behavior of a <video> element’s sizing!
- The warm fuzzy feeling of semantic correctness
- Making use of browser extensions like control buttons for an active video element
- Being able to use browser extensions like sending output to ChromeCast or AirPlay
- Disabling screen dimming/lock during playback
This last is especially important for videos of non-trivial length, especially on phones which often have very aggressive screen dimming timeouts.
Well, in some browsers (Chrome and Firefox) now you can do at least some of this. :)
I’ve done a quick experiment using the <canvas> element’s captureStream() method to capture the video output — plus a capture node on the Web Audio graph — combining the two separate streams into a single MediaStream, and then piping that into a <video> for playback. Still have to do YUV to RGB conversion myself, but final output goes into an honest-to-gosh <video> element.
To my great pleasure it works! Though in Firefox I have some flickering that may be a bug, I’ll have to track it down.
Some issues:
- Flickering on Firefox. Might just be my GPU, might be something else.
- The <video> doesn’t have insight to things like duration, seeking, etc, so can’t rely on native controls or API of the <video> alone acting like a native <video> with a file source.
- Pretty sure there are inefficiencies. Have not tested performance or checked if there’s double YUV->RGB->YUV->RGB going on.
Of course, Chrome and Firefox are the browsers I don’t need ogv.js for for Wikipedia’s current usage, since they play WebM and Ogg natively already. But if Safari and Edge adopt the necessary interfaces and WebRTC-related infrastructure for MediaStreams, it might become possible to use Safari’s full screen view, AirPlay mirroring, and picture-in-picture with ogv.js-driven playback of Ogg, WebM, and potentially other custom or legacy or niche formats.
Unfortunately I can’t test whether casting to a ChromeCast works in Chrome as I’m traveling and don’t have one handy just now. Hoping to find out soon! :D