Category Archives: wiki

ios-ogvjs-header

ogv.js sound and video now working on iOS

Thanks to some audio-related fixes and improvements from a contributor, I’ve started doing some new work on ogv.js, the emscripten-powered JavaScript Ogg Theora/Vorbis decoder.

In addition to Internet Explorer 10/11 (via Maik’s Flash shim), I now have audio working on iOS — and smaller video sizes actually play pretty decently on a current iPhone 5s as well!

See demo:

Try it out on your computer or phone!

Older iOS 7 devices and the last generation of iPod Touch are just too slow to play video reliably but still play audio just fine. The latest 64-bit CPU is pretty powerful though, and could probably handle slightly larger transcodes than 160p too.

Wikipedia new Android, iOS apps in development: current demo

Latest demo screencast of Android and iOS new Wikipedia mobile apps in development — now with login and basic editing! I think folks are really going to like what we’ve done once we get these polished up and out in the stores replacing our old mobile app.

If the on-wiki embedded video player doesn’t work for you, try a mirror on YouTube.

Thanks to Android 4.4’s built-in screen recording feature this didn’t require any custom hardware kit to record, but I had to jump through some hoops to edit it in PiTiVi… I’ll write up some notes on-wiki.

Screen Shot 2014-02-18 at 6.54.53 AM

ogv.js now with sound in Internet Explorer thanks to… Flash?

One of my fun little side projects is ogv.js, a prototype audio/video decoder in JavaScript to allow playback of Wikimedia Commons’ media files on browsers that don’t natively support the royalty-free Ogg Vorbis, Theora or WebM codecs, but may have trouble with our old ‘Cortado’ Java applet compatibility solution thanks to the modern trend of disabling plugins (and Java in particular).

Mostly this means Internet Explorer and Safari — Chrome and Firefox handle the files natively. However Internet Explorer was limited by the lack of support for the Web Audio API, so could not play any sound. I’d hypothesized that a Flash shim could be used — Windows 8 ships with the Flash plugin by default and it’s widely installed on Windows 7 — but had no idea where to start.

Open source to the rescue!

One of the old maintainers of the Cortado applet, maikmerten, took an interest. After some brief fixes to get the build scripts working on Ubuntu, he scrounged up a simple ActionScript audio shim, with source and .swf output, and rigged up the ogv.js player to output audio through that if there was no native Web Audio API.

It woooooorks!

Screen Shot 2014-02-18 at 6.54.53 AM

The ActionScript of the Flash shim is pretty straightforward, and it compiles into a nice, approx 1kb .swf file. Luckily, you can rebuild the .swf with the open-source Apache Flex SDK, so it doesn’t even rely on proprietary Flash Builder or anything. We could do with some further cleanup (for instance I don’t think we’re disposing of the Flash plugin when shutting down audio, but that’s easy to fix in a bit…) but the basics are in place. And of course getting proper audio/video sync will be complicated by the shim layer — the current code drives the clock based on the video and has choppy audio anyway, so there’s some ways to go before we reach that problem. ;)

It even works on Windows RT, the limited ARM version of Windows 8 — though the video decoding is much too slow on a first-gen Surface tablet’s Tegra 3 CPU, audio-only files play nicely.

Thanks maikmerten!

 

ogv.js update: color, sound

Last week I posted about a proof of concept experiment cross-compiling the Theora video decoder to JavaScript: ogv.js.

Well, I did some more hacking on it this weekend:

  • Color output? Check.
  • Streaming input? Check. (But no seeking yet, and buffering is aggressive on some browsers.)
  • Sound? Check. (But it’s not synced, choppy, and usually the wrong sample rate. And no IE or iOS support yet.)
  • Pretty interface to select any file from Wikimedia Commons’ Media of the Day archive? Check.
  • Laid some groundwork for separating the codec to a background Worker thread (not yet implemented)
  • See the readme for more info.

Try out the live demo with this pretty video of a Montreal subway station:

Jarry metro video screenshot

Feel free to try building or hacking the source for fun. :)

ogv.js proof of concept

I spent some weekend time working on one of those Crazy Things That Just Might Work: a JavaScript cross-compile of the Ogg Theora video decoder. Very primitive proof of concept demo is live (no audio, no sync, no color, …) but it works better than I figured after a day’s worth of hacking!

Larger files run… veerrryyyy sslllooww on my test iPod Touches, but this certainly seems fast enough on desktop to one day replace our old Java fallback for Safari and newer IE…

How it works

The C libraries libogg, libvorbis, and libtheora are cross-compiled from C to JavaScript using emscripten, a super-awesome tool that builds via the LLVM clang compiler and provides a mostly C-compatible environment within JavaScript with surprisingly good performance.

A thin C/JS wrapper layer accepts input data from the JavaScript side, lets it be processed by the converted C code, and then outputs to an HTML canvas element on the web page.

Only a couple of tiny tweaks to the libraries are needed to make them build; I started with build scripts for just the audio codecs from this project, added in libtheora, and started adapting parts of one of the Theora data dump examples.

Possibilities

Finish up the YCbCr->RGB conversion, add audio decoding & output, and some kind of sync and seeking, and …… this could replace the old Java Cortado app we use as a fallback player on Wikipedia for browsers that don’t run WebM or Theora.

Web Workers could be used to push decoding to a background thread, depending on whether overhead is problematic.

Crazy idea: provide an HTML5 <video>-style DOM interface, integrate into TimedMediaHandler as a drop-in replacement

Limitations

  • Audio sync may be difficult to achieve.
  • Audio output APIs — need to confirm what’s consistently available.
  • Performance is surprisingly good on a desktop; I have no doubt this will be sufficient for playback in Safari and IE if audio & sync can be managed.
  • Performance is not so good on my test iPod Touches; it might be fun to tune and optimize but I would expect to get much better results from a native app on iOS.
  • There doesn’t seem to be a good universal way to do progressive data reads from an XMLHttpRequest; it may be necessary to buffer portions of the file by running multiple requests for subranges of the file, which is NOT pretty.
  • IE 10 doesn’t support ArrayBuffer.slice(). Currently this prevents the demo from running, but it’s not actually needed.

 

Ada Initiative: help support women (and everybody else) in Open Source!

The Ada Initiative is raising money for their programs supporting women in open source, open culture, and geekdom in general. They’ve reached about 70% of their fundraising goal… can you help them reach $100k by Saturday?

Like it or not, there are widespread issues with poor behavior, outright harassment, and creepy misogynistic tendencies in our beloved nerd communities:

  • free/open source software
  • gaming
  • Wikipedia
  • science-fiction fandom
  • atheism/skepticism
  • etc

Having grown up, worked, or dabbled in all of those communities, I’m often saddened by the continued negative experiences that many women have had and continue to have. The issues that Ada Initiative deal with affect many men and other social subgroups of all sorts, too — LGBTQ folks, people with depression or other mental illness, etc — which matters to me because so many of my coworkers and friends fall into some of those categories.

I’d like to see us move away from glorifying douchebaggery in all forms, and towards respectful participation for all! Care to help?

 

Donate now

HDMI 1080p60 capture or decimation needed for mobile demos

Dear Lazyweb: anyone know a device that can decimate an HDMI 1080p60 signal to 1080p30 or 1080i60 – OR record HDMI directly at 1080p60 for a reasonable price?

I’ve got a Thunderbolt-connected HDMI recorder I use for demos and screencasts of Wikipedia on smartphones and tablets… but some of the newer devices output at 1080p60, which my low-end capture box doesn’t grok.

The devices I’m seeing that can record 1080p60 are ~$1k and/or are PCIe cards which isn’t useful in a laptop-dominated workplace.

Any recommendations?

Update 2013-08-30: I’m getting the impression that there’s actually a combination of two problems: things wanting to default to 1080p60 and HDCP encryption — which apparently is turned on by default for the HDMI outputs on the Nexus 4 and 10. I found a widget that allegedly strips the HDCP, but I’m still not sure if it’s trying to pump 1080p through there in which case it’s still not working. I’m picking up an EDID sniffer/emulator which should be able to detect the EDID from the capture box (which should say ‘no 1080p60′) and put that in front of the HDCP stripper…. we’ll see if that works. Sigh.

Chromebook Pixel first look

Screenshot 2013-03-01 at 6.25.18 PM

So, I gave in and picked up a Chromebook Pixel. I admit, I’m seduced by the high-resolution 2560×1700 screen. Nom nom nom so many tiny pixels!

The browser works like you’d expect — all the usual web stuff seems to work, just like Chrome on Linux or Mac or Windows. Like the newer MacBook Pros it has a very high-density display, which looks fantastic. Wikipedia looks great; we properly detect the density and load enhanced-resolution images. (We still have to make the logo high-res, we know that’s a problem. :)

The machine also correctly handles mixed-resolution situations when you plug in an external monitor. (The plug is mini-DVI, conveniently compatible with your existing MacBook VGA, DVI, or HDMI adapters. Yay!) Hook up a regular 1080p monitor and drag a browser window over — it’ll automatically switch to low-density and everything appears the correct size. Move the window back to the main screen, and it pops back into beautiful high-resolution. The main limitation is that windows can’t span screens; except during the move operation itself they display only on one monitor or the other.

But of course you’re all wondering about Chrome OS and its suitability for a medium-high-end laptop. Is it good or bad? Hard to say so far, I’m still exploring it… but be aware the machine isn’t limited to Chrome: it’s easy to unlock to developer mode and either mess with the underlying Linux system or install a stock OS distribution like Ubuntu.

Just to prove it to myself, I went ahead and followed the directions on switching to developer mode, enabling USB and legacy booting, and was able to boot an Ubuntu installer stick into the GUI. (I was stuck for a while unable to boot, but it turned out to be because I had an incomplete .iso download. Whoops!) Unfortunately the trackpad isn’t supported in the stock distro yet; some people have been working on drivers, but I might wait a bit for it to be better integrated. Ubuntu’s Unity desktop also isn’t quite “retina-ready”, and needs some more loving for high-density screens.

In the meantime, I’m trying out Chrome OS as she was meant to be spoken. As a fan of Firefox OS, the idea of a browser-centric OS already appeals to me (though they are very differently implemented under the hood)… but I also know that there are limitations.

In regular (non-developer) mode, you don’t have access to a low-level Linux shell. There is a terminal emulator (press control+alt+T) which can do ssh, so if you do all your development on a server in a shell, that might be good enough for you. :)

But, you can’t install anything that doesn’t run in the browser… but it’s a pretty good browser, and is extended in several ways:

  • all the usual pure HTML5 suspects we know and love
  • Flash plugin
  • special plugin for Netflix — you don’t get that on stock Linux :(
  • PDF viewer
  • NaCl plugin

NaCl is interesting because it allows running sandboxed native code at full speed, within the existing HTML/JavaScript security model. Sorta like Java applets but precompiled on the server. Downside is that it requires compiling to multiple platforms (x86, x86-64, and ARM), but the upside is you can apparently run some pretty fast stuff, including access to OpenGL ES for graphics. This should be pretty good for games, if developers are willing to port… A low-end example is NaClBox which is a port of DosBox to run in the NaCl environment.

(Mozilla meanwhile is pushing emscripten as a platform-neutral alternative to NaCl. This compiles C and C++ programs to JavaScript through a clang/LLVM layer. The overhead of JavaScript compilation and type-safety slows it down compared to NaCl, but it achieves reasonably good performance on modern JS engines and works in more browsers. Combined with WebGL, this is also a way to port C/C++ games to the web. There are some nice examples like the BananaBread FPS demo, which *almost* works on the Chromebook… graphics are lovely but the mouse movement seems to be misdetected.)

As for getting “real work” done… thanks to Apple’s limitations I can’t do iOS development on anything but an actual Mac OS X machine, so I won’t be using it for my current main project. But it can serve well for secondary tasks: poking the wikis, email, calendering, chat, Google Docs and Hangouts, notes in Etherpad, etc. If I can rig up an SSH key, I should be able to ssh into my own or work servers in the terminal to do some maintenance there. In theory, I can do web development through an in-browser IDE like Cloud9 — I’ll try it out on MediaWiki and see what I can report.

I’m having trouble finding a good web-based IRC chat. Freenode’s web chat interface is usable but just …. not very good. I tried Kiwi IRC which has a better UI, but I’m still not quite satisfied with it. Maybe I’ll go back to the terminal. ;)

To be continued…

Ubuntu Touch first look

An early developer preview of Ubuntu Touch for Nexus phones and tablets has been released! Since I’m still waiting on the Nexus 4 I’m reserving for Ubuntu testing, I temporarily installed a tablet image on my Nexus 10.

device-2013-02-22-144057

Be warned: this is a very early preview and is more demo-ware than usable operating system at this point. Don’t flash it on your primary phone/tablet or you’ll be mighty disappointed, I suspect. :)

The good:

  • It looks very pretty. They’ve got some good UI designers. :)
  • Edge swipe gestures leave more of the screen available for applications than Android does, as with some other new OSs like Windows 8/RT and BlackBerry 10.
  • Seems to have a WebKit-based browser as default. Renders Wikipedia nicely — we get the desktop version by default on the Nexus 10 as it masquerades as an iPad, which we also send to the desktop site.
  • Some of the hardware actually works, like wifi networking and the cameras. :)
  • There seems to be infrastructure for inter-application data sharing.
  • It handles high-density screens from the get-go; the 2560×1600 Nexus 10 screen looks awesome. Text and photos are crisp and beautiful.

The bad:

  • Very unstable. The UI has crashed on me several times in just a couple of hours of testing. This kicks you back to the login screen.
  • No screen rotation on orientation change.
  • On-screen keyboard is pretty flaky; while it comes up and goes away much more reliably than the GNOME 3 or Ubuntu desktop versions, it fairly often fails to register keypresses.
  • Scrolling and similar UI operations are sometimes a bit sluggish, especially in the browser. This may be due to insufficient graphics acceleration at this stage.
  • WebM and Ogg video don’t appear to fully work in the browser… at least I can’t get anything to play on Wikimedia Commons.

The ugly:

  • Outside the browser and anything you install yourself with Qt Creator, there aren’t a lot of useful apps at this stage. There’s an “Available for download” section on the apps home screen, but tapping the icons does nothing. Aggravating as one of them is ‘Wikipedia’ and I want to know what it does. :)
  • Not clear how traditional Linux CLI and GUI programs will work. Will they run on the tablet-style “desktop” or will it fire up a classic desktop-style UI when you dock? There’s no terminal app yet; you can get a shell via the Android debug bridge tools and can set up SSH from there, but that doesn’t get you into the GUI.
  • No mobile data support for the phone builds yet; data is wifi only.
  • The current images include non-free drivers and sample data, so isn’t fully open source yet. Ick!

I haven’t tested HDMI output yet.

Development

I haven’t done more yet than firing up Qt Creator and fiddling with the demos on the desktop, but it’s interesting to see the development landscape shaping up.

Ubuntu Phone/Touch’s “native” UI toolkit is Qt, with emphasis on using ‘Qt Quick’ and QML+JavaScript for rapid development. It also allows for C++ Qt apps, including low-level OpenGLish things. There’s a sample port of some game that… doesn’t seem to work yet. ;)

Qt is a bit surprising as Ubuntu’s traditionally been GNOME/GTK+-focused. But it makes sense; it’s capable, developer-friendly, now has the declarative stuff like QML markup and first-class JavaScript support, and most importantly it’s also in use on other platforms like BlackBerry 10 and the (mostly dead or is it?) Meego. This could help with code sharing on ‘alternate platforms’… OpenGL-ish games probably will see this as no worse than Android, or possibly easier since there’s no Java/JNI stuff to play with for native apps.

Compared to Android

At this stage there’s no comparison to Android; you just can’t get much done. But we’ll see as it matures…

One of the selling points is that Ubuntu Touch is supposed to be dockable and usable as a “real computer” when connected to a mouse and keyboard and external screen. However this doesn’t seem to be in the current versions, and it’s unclear yet how much better it will be than Android when docked. (Android also supports keyboards, mice, and external screens… but doesn’t as conveniently run existing Linux apps.)

Compared to Firefox OS

The ability to write “native code” apps is probably going to be a big mark in favor of Ubuntu for things like games, which tend to be C/C++ and OpenGL-heavy. Though Mozilla’s done a lot of work with things like emscripten and WebGL making it possible to port C/C++ game engines to JavaScript, I suspect performance will always lag a bit on Firefox OS.

Ubuntu Touch also definitely supports high-DPI displays and tablet-size screens at this stage; it’s unclear to me whether Firefox OS is ready for high-DPI displays or larger screens. Even if it runs all the images in the apps are low-resolution so a lot of stuff might look blurry.

However, Firefox OS is much further along. It’s got a working app marketplace, a browser that doesn’t crash quite as constantly, and most importantly… has hardware partners who are already committed to shipping phones this year.

It looks like this version of Ubuntu Touch was kept as a skunkworks project for a while, then rushed out so it could be demoed at Mobile World Congress. Hopefully development will be more open from here out and it will improve rapidly.