ogv.js now with synchronized sound

My ogv.js JavaScript Theora video player project now has synchronized audio and video, which makes it possible to watch videos of people talking without going mad. 😀

Screen Shot 2014-03-06 at 4.57.30 AM

Try it out!

This works both with Web Audio in Firefox, Chrome, and Safari and with the Flash audio shim for IE; basically we have to keep track of the audio playback position and match up decoding frames with that. Took a little poking in the ActionScript code but it’s now working!

This brings us much closer to being able to integrate ogv.js as a fallback video player for Wikimedia on IE 10/11 and Safari 6/7. Thanks to the guys hanging out in #xiph channel who encouraged me to keep poking at this! 😀

Additionally there’s now an override selector for the video size, so you can try decoding larger than 360p versions, or switch a slow machine down to the little 160p versions.

I’ve also started investigating an all-Flash version using Adobe’s Crossbridge, which if it works would be a suitable replacement for the Cortado Java applet on old browsers (think of all those IE 6/7/8/9 systems out there!). I seem to be able to build the ogg libraries but haven’t gone beyond that yet… will be interesting to poke at.

ogv.js sound and video now working on iOS

Thanks to some audio-related fixes and improvements from a contributor, I’ve started doing some new work on ogv.js, the emscripten-powered JavaScript Ogg Theora/Vorbis decoder.

In addition to Internet Explorer 10/11 (via Maik’s Flash shim), I now have audio working on iOS — and smaller video sizes actually play pretty decently on a current iPhone 5s as well!

See demo:

Try it out on your computer or phone!

Older iOS 7 devices and the last generation of iPod Touch are just too slow to play video reliably but still play audio just fine. The latest 64-bit CPU is pretty powerful though, and could probably handle slightly larger transcodes than 160p too.

Wikipedia new Android, iOS apps in development: current demo

Latest demo screencast of Android and iOS new Wikipedia mobile apps in development — now with login and basic editing! I think folks are really going to like what we’ve done once we get these polished up and out in the stores replacing our old mobile app.

If the on-wiki embedded video player doesn’t work for you, try a mirror on YouTube.

Thanks to Android 4.4’s built-in screen recording feature this didn’t require any custom hardware kit to record, but I had to jump through some hoops to edit it in PiTiVi… I’ll write up some notes on-wiki.

ogv.js now with sound in Internet Explorer thanks to… Flash?

One of my fun little side projects is ogv.js, a prototype audio/video decoder in JavaScript to allow playback of Wikimedia Commons’ media files on browsers that don’t natively support the royalty-free Ogg Vorbis, Theora or WebM codecs, but may have trouble with our old ‘Cortado’ Java applet compatibility solution thanks to the modern trend of disabling plugins (and Java in particular).

Mostly this means Internet Explorer and Safari — Chrome and Firefox handle the files natively. However Internet Explorer was limited by the lack of support for the Web Audio API, so could not play any sound. I’d hypothesized that a Flash shim could be used — Windows 8 ships with the Flash plugin by default and it’s widely installed on Windows 7 — but had no idea where to start.

Open source to the rescue!

One of the old maintainers of the Cortado applet, maikmerten, took an interest. After some brief fixes to get the build scripts working on Ubuntu, he scrounged up a simple ActionScript audio shim, with source and .swf output, and rigged up the ogv.js player to output audio through that if there was no native Web Audio API.

It woooooorks!

Screen Shot 2014-02-18 at 6.54.53 AM

The ActionScript of the Flash shim is pretty straightforward, and it compiles into a nice, approx 1kb .swf file. Luckily, you can rebuild the .swf with the open-source Apache Flex SDK, so it doesn’t even rely on proprietary Flash Builder or anything. We could do with some further cleanup (for instance I don’t think we’re disposing of the Flash plugin when shutting down audio, but that’s easy to fix in a bit…) but the basics are in place. And of course getting proper audio/video sync will be complicated by the shim layer — the current code drives the clock based on the video and has choppy audio anyway, so there’s some ways to go before we reach that problem. 😉

It even works on Windows RT, the limited ARM version of Windows 8 — though the video decoding is much too slow on a first-gen Surface tablet’s Tegra 3 CPU, audio-only files play nicely.

Thanks maikmerten!

 

Veterans Day

Today is Veterans Day in the US, known as Armistice Day in some other countries. Whatever you call it, it’s the day we remember the sacrifices of soldiers and other military veterans, both present and past.

I’ve never been in a war; I’ve never served in any army; I didn’t even grow up in a military family. But war and its human element have always loomed in the background, both informing the history of the world around me and shaping what comes next.

 

At least one of my great grandfathers served in World War I, a war now nearly a century old that America barely remembers. But it was this war, the “War to End All Wars”, that brought us the 11/11 Armistice Day.

 

My grandfathers both volunteered during World War II back in the 1940s — the last time the US was on a “total war” footing. My father’s father served in Europe as an Army supply sergeant, and my mother’s father, too young for the Navy, joined the Merchant Marine and helped with trans-Atlantic shipping. They worked hard for their country, and the work they did on the supply lines helped keep front-line soldiers alive and fighting against the Nazis.

 

As we get closer to the present, America’s wars have gotten smaller, and fewer people have been directly involved on “our” side. My father was in university during Vietnam, so avoided the draft and instead had great learning and job opportunities here at home. War became something remote and theoretical to our family.

 

When I was a child, Vietnam was over, the draft had been abandoned, but the Cold War was still alive and well; we were more worried about mutual nuclear annihilation with the Soviet Union than conventional war. It almost seemed an anachronism that we had two Marine Corps air bases nearby (MCAS Tustin and MCAS El Toro, which put on a wonderful air show every year for the kids). Tanks and missiles and machine guns and attack helicopters were “cool” things you mostly saw in movies and video games.

But real wars kept happening, even if they weren’t quite as globe-spanning, and real people were still living and dying in them.

Desert Shield / Desert Storm brought active war into focus for me when I was about 12. I didn’t know anybody directly in the military, but I knew they were real people — a lot of people and equipment shipped out from the local bases, and the news would report on casualties from the local area.

One of my most vivid memories from Desert Storm was reading a newspaper account of a friendly fire incident in which a member of a tank crew was decapitated. I’ve seen far more gruesome things simulated in movies and for real in pictures on the internet (unfortunately), but the shock of a 12-year old reading about a soldier suddenly finding himself holding his crewmate’s lifeless head will never quite fade.

“My” generation’s war didn’t come until 9/11 sparked a US invasion of Afghanistan, followed later by another Iraq invasion. There was no longer a draft, but like my father during Vietnam I was old enough I could have served, but chose instead to stick with university and a career.

 

I don’t like the idea of war. I hate the idea of people being hurt, displaced, killed, or losing loved ones. But war is a real thing that’s part of the human condition, and for better or worse we have to have people who get involved in them to try to bring the fighting to a close.

Anyway, I’m not really sure I had a point. But please, if you have the day off today, spend some time thinking about the people who go off to war, whether deliberately as soldiers or with no choice as noncombatants, who never come back. And the ones that do come back, often don’t come back the same.

Think about this when you make decisions about your life.

ogv.js update: color, sound

Last week I posted about a proof of concept experiment cross-compiling the Theora video decoder to JavaScript: ogv.js.

Well, I did some more hacking on it this weekend:

  • Color output? Check.
  • Streaming input? Check. (But no seeking yet, and buffering is aggressive on some browsers.)
  • Sound? Check. (But it’s not synced, choppy, and usually the wrong sample rate. And no IE or iOS support yet.)
  • Pretty interface to select any file from Wikimedia Commons’ Media of the Day archive? Check.
  • Laid some groundwork for separating the codec to a background Worker thread (not yet implemented)
  • See the readme for more info.

Try out the live demo with this pretty video of a Montreal subway station:

Jarry metro video screenshot

Feel free to try building or hacking the source for fun. :)

ogv.js proof of concept

I spent some weekend time working on one of those Crazy Things That Just Might Work: a JavaScript cross-compile of the Ogg Theora video decoder. Very primitive proof of concept demo is live (no audio, no sync, no color, …) but it works better than I figured after a day’s worth of hacking!

Larger files run… veerrryyyy sslllooww on my test iPod Touches, but this certainly seems fast enough on desktop to one day replace our old Java fallback for Safari and newer IE…

How it works

The C libraries libogg, libvorbis, and libtheora are cross-compiled from C to JavaScript using emscripten, a super-awesome tool that builds via the LLVM clang compiler and provides a mostly C-compatible environment within JavaScript with surprisingly good performance.

A thin C/JS wrapper layer accepts input data from the JavaScript side, lets it be processed by the converted C code, and then outputs to an HTML canvas element on the web page.

Only a couple of tiny tweaks to the libraries are needed to make them build; I started with build scripts for just the audio codecs from this project, added in libtheora, and started adapting parts of one of the Theora data dump examples.

Possibilities

Finish up the YCbCr->RGB conversion, add audio decoding & output, and some kind of sync and seeking, and …… this could replace the old Java Cortado app we use as a fallback player on Wikipedia for browsers that don’t run WebM or Theora.

Web Workers could be used to push decoding to a background thread, depending on whether overhead is problematic.

Crazy idea: provide an HTML5 <video>-style DOM interface, integrate into TimedMediaHandler as a drop-in replacement

Limitations

  • Audio sync may be difficult to achieve.
  • Audio output APIs — need to confirm what’s consistently available.
  • Performance is surprisingly good on a desktop; I have no doubt this will be sufficient for playback in Safari and IE if audio & sync can be managed.
  • Performance is not so good on my test iPod Touches; it might be fun to tune and optimize but I would expect to get much better results from a native app on iOS.
  • There doesn’t seem to be a good universal way to do progressive data reads from an XMLHttpRequest; it may be necessary to buffer portions of the file by running multiple requests for subranges of the file, which is NOT pretty.
  • IE 10 doesn’t support ArrayBuffer.slice(). Currently this prevents the demo from running, but it’s not actually needed.

 

Ada Initiative: help support women (and everybody else) in Open Source!

The Ada Initiative is raising money for their programs supporting women in open source, open culture, and geekdom in general. They’ve reached about 70% of their fundraising goal… can you help them reach $100k by Saturday?

Like it or not, there are widespread issues with poor behavior, outright harassment, and creepy misogynistic tendencies in our beloved nerd communities:

  • free/open source software
  • gaming
  • Wikipedia
  • science-fiction fandom
  • atheism/skepticism
  • etc

Having grown up, worked, or dabbled in all of those communities, I’m often saddened by the continued negative experiences that many women have had and continue to have. The issues that Ada Initiative deal with affect many men and other social subgroups of all sorts, too — LGBTQ folks, people with depression or other mental illness, etc — which matters to me because so many of my coworkers and friends fall into some of those categories.

I’d like to see us move away from glorifying douchebaggery in all forms, and towards respectful participation for all! Care to help?

 

Donate now

HDMI 1080p60 capture or decimation needed for mobile demos

Dear Lazyweb: anyone know a device that can decimate an HDMI 1080p60 signal to 1080p30 or 1080i60 – OR record HDMI directly at 1080p60 for a reasonable price?

I’ve got a Thunderbolt-connected HDMI recorder I use for demos and screencasts of Wikipedia on smartphones and tablets… but some of the newer devices output at 1080p60, which my low-end capture box doesn’t grok.

The devices I’m seeing that can record 1080p60 are ~$1k and/or are PCIe cards which isn’t useful in a laptop-dominated workplace.

Any recommendations?

Update 2013-08-30: I’m getting the impression that there’s actually a combination of two problems: things wanting to default to 1080p60 and HDCP encryption — which apparently is turned on by default for the HDMI outputs on the Nexus 4 and 10. I found a widget that allegedly strips the HDCP, but I’m still not sure if it’s trying to pump 1080p through there in which case it’s still not working. I’m picking up an EDID sniffer/emulator which should be able to detect the EDID from the capture box (which should say ‘no 1080p60′) and put that in front of the HDCP stripper…. we’ll see if that works. Sigh.

Samsung Galaxy S4 Mini Duos first look (GT-i9192)

So last week was Wikimania 2013 in Hong Kong — it was as always good smashing fun, but it was also one of the first trips I’ve been on where I used a local data SIM in my primary smartphone.

This turned out to be really useful for communicating via online tools (email and Facebook mostly, though some of the other folks used Google Hangouts for group planning), and also let me post some photos during the trip — still have to go over them and do a final push to Flickr! (Previously I’d used an old spare phone for data but it wasn’t able to take decent photos etc.) But on the downside, with the local SIM in nobody could reach me at my regular phone number…

This got me thinking about dual-SIM card phones that can be online on two mobile networks at once — such as to use local data while roaming on your regular home number. Though I travel fairly regularly for conferences, I’d never gotten one before as I’d only seen Chinese brands that I don’t know and am not quite sure I’m willing to trust for software updates. :) At our last hackathon in Amsterdam I carried two phones, one for calls and one for data… effective but bulky.

Turns out there’s a dual-SIM version of the Samsung Galaxy S4 Mini, which I was able to easily obtain in the US via Amazon.

First impressions: size matters

The S4 Mini is definitely smaller than is the trend for the more top-of-the-line phones these days… IMO this is a good thing, as I’ve been whinging about the Nexus 4 being slightly too large to comfortably use in one hand, and the “regular” versions of the S4 and HTC’s One are even huger. Instead, it’s about the size of the older Nexus S; a little wider than an iPhone 5 but about the same length.

This is waaaay more comfortable to use in one hand, though the S4 Mini is also very lightweight — so much so that I’m having to get used to not flipping it over by mistake by being too rough with it. :)

I may actually like using this as a primary phone for the compactness; certainly it’ll be a win for travel — especially compared to carrying two phones!

Display: meh

AMOLED 960×540 screen. Not super great, but functional and sharp enough to read comfortably… the screen is one of the biggest complaints I’ve seen in other reviews of the S4 mini family, though. Personally I’d have preferred a 720p LCD panel if I could make an ‘ideal’ phone in this size to my specifications.

User interface: why, Samsung, why?

Most of the Android devices I’ve personally owned have been Nexus models, running a stock Google user interface, which is to say, a really nice interface since the 4.0 release. This …. is not one of those phones.

The S4 Mini sports Android 4.2.2 Jelly Bean with Samsung’s customized “TouchWiz” interface modifications. It also has hardware home/menu/back buttons which means slightly different gestures versus the software buttons on the Nexus 4.

(My personal pet peeve with the hardware buttons: Android 4.x doesn’t show an overflow menu icon on apps’ action bars. You’re back to the Android 1/2 paradigm of “there might be a menu, try the button and find out!” Bleh.)

Luckily many of the custom behaviors can be disabled, and the standard Google on-screen keyboard can be installed to replace Samsung’s slightly-different version. This helps immensely in making me comfortable using the device.

The dual SIM feature

I was of course most interested in that dual-SIM feature. To test, I put my regular T-Mobile SIM in slot 1, and my Hong Kong temporary data plan’s SIM in slot 2. There’s a ‘SIM Manager’ app which lets you easily select whether you want one, the other, or both SIMs active, and switch the primary data and voice networks between them. The HK SIM came up on roaming, and I was able to easily switch it off until next time it becomes relevant. :)

Unfortunately this feature may also be the thing preventing me from a stock Android experience — there currently isn’t a CyanogenMod build available for this particular model, and I don’t know whether dual-SIM features are available in CM or other more ‘stock’ alternate ROMs. :(

Storage

8 GB of total internal storage (about 5 GB available after the OS), plus a micro-SD card slot (welcome back to manual storage management!) Adequate core for a casual phone and extensible for media storage, hey that ain’t bad.

Optional encryption is supported, enabled separately for the core storage and the SD card.

CPU/GPU/RAM/Blah

Fast enough for my basic needs. See benchmarks in generic S4 mini reviews on the interweb if you like that sort of thing. :)

Next steps

I’ll poke at this phone for a couple weeks and see how I like it; may or may not switch back to the Nexus 4 as a primary phone, but definitely this is going to be coming with me on international trips!