Hotel MediaWiki: You can check out, but you can never leave

As announced today, I’ll be returning to the Wikimedia Foundation to work on MediaWiki full-time at the end of the month, after about a year and a half working on StatusNet‘s open-source social networking systems.

Of course in open source, leaving doesn’t mean good-bye… expect to still see SN bug reports and code commits from me in April and beyond! ;)

Never stop learning

First I want to give a shout-out to StatusNet founder and awesome guy Evan Prodromou — the flagship identi.ca community he started has been a major influence on how the freedom-loving end of geekdom sees social networking and its possibilities, and was what drew me to work on the project. I’ve had a great time working with the StatusNet devs & user community, and had chances to delve into new worlds both on the server (fun with queueing!) and the client (I certainly have a whole new appreciation for what JavaScript can do!)

The thing that has always distinguished StatusNet from its more commercial competitors has been respect for users’ autonomy. (The original name of the company was even “Control Yourself, Inc”!) As more and more applications move into “the cloud”, more and more of the “stuff” that makes up our personal lives is living on computers thousands of miles away that belong to some company whose only claimed responsibility is to its shareholders. Maybe your app works like you want, maybe it doesn’t. Maybe you can move your data to another web site, maybe you can’t. Maybe your private data stays private, maybe it doesn’t…

Not only is StatusNet’s code free to use for your own server, but you can actually do so without being cut off from the rest of the network — a consumer-friendly goal that the monolithic Twitters and Facebooks of the world are still fighting against. Evan, Zach, James, myself, and other StatusNetters have put real work into avoiding consumer-unfriendly lock-in and instead promoting interoperability with things like our work on the OStatus protocol and organizing last year’s Federated Social Web Summit at OSCON.

I’m darn proud to have been a part of that, and will continue to apply the “autonomous eye for the cloud guy” to the other web apps I work on in future…

StatusNet goings-on

Things have seemed quiet from StatusNet-land lately because the company’s been re-focusing for our 1.0 push (see Evan’s announcement); there’s actually some really cool stuff coming up:

• A final 0.9.x update should come out as StatusNet 0.9.7 this week or so — currently in final shakedown on identi.ca and status.net hosted sites. This fixes a lot of old bugs from 0.9.6, adds some AJAX-y love, and has improved ActivityStreams output including initial JSON support.
  • Users can now back up and restore their full posting history & friends list (links on settings page sidebar): this should stay future-proof in 1.0.x and beyond.
  • Lots of new plugins abound!
• StatusNet’s Android and iPhone mobile clients are being brought up to date and fixed up by Ed Finkler of Spaz fame. There should be new beta releases soon which improve performance and fix some of the crashing bugs from the last release.
• StatusNet development on gitorious and localization on TranslateWiki.net have moved to the 1.0.x branch, which picks up some existing work on improved database-independence and is moving on to a major visual refresh (making it easier to theme) and better extensibility with new data types.
  • A prototype of the new “micro-apps” can be seen in the Bookmark plugin in 0.9.7, which allows you to save Delicious-style bookmarks into your notice stream, including compatible import/export! Bookmarks are stored as a special notice type in the 1.0.x version, and the MicroAppPlugin base class helps take care of some of the work for you.
  • It will become much, MUCH easier to pass arbitrary data for “micro-apps” and for those trying to use StatusNet as a sort of custom message bus.

The backup/restore system, moving accounts from server to server, and the new micro-apps are all built on top of ActivityStreams, the industry-standard(ish) way of marking up social activities in Atom (and now JSON) streams. This follows StatusNet’s established practice of building open, interoperable systems by combining existing open standards where possible — our OStatus site-to-site protocol runs ActivityStreams data in Atom feeds over PubSubHubbub and Salmon transports (themselves using nice standard XML and HTTP).

MediaWiki plans

One of the things that has me super excited about MediaWiki again is the recent 1.17 migration on Wikimedia’s sites. With this new version’s ResourceLoader system we can be a lot more flexible with client-side extensions: jQuery libraries as a standard component, plus loading code modules dynamically makes it a lot easier to move beyond the traditional “post form and reload” interactivity that MediaWiki was originally built for in 2002.

My favorite pet project recently has been integrating an embeddable in-browser vector graphics editor, so SVG drawings and maps can be created and changed as easily as a simple text page. I’m very interested in getting similar sorts of advanced extensions workable with less administrator intervention — it would actually be possible to do this embedding system through user JavaScript or a Gadget, and I hope to make it easier for individual users to create and share this sort of advanced UI extension safely.

Parser ahoy!

At the core of the Wiki ideal is the notion that editing the documents on the site is quick and easy.  Early wikis used very limited markup, reserving just a few characters for formatting and using special patterns for linking (“CamelCase”).

Over the years, the capabilities of wiki systems grew to meet the demands of their users. Wikipedia being a …. serious use case, MediaWiki’s markup syntax got more and more complex. More and more funny characters, subsets of HTML, squiggle brackets, and finally the unholy terrors that can be created mixing tables and templates… Wikipedia’s gotten a reputation as being difficult to edit for newbies and even old hands are often reduced to quivering blobs of brain-goo when faced with a particularly complex template situation.

I won’t go to far into detail, but I’ve been suckered into ….. volunteered for the next-generation parser work. We’ve thrown around some ideas before on this, but it looks like there’s enough interest now to really start hammering things together. Basically what we want to do is:

1. solidify a sane, but flexible document structure that handles everything needed when templates are used sanely
2. be able to identify structures that don’t work well (existing weird template edge cases) so people and/or bots can help restructure them
3. as edge cases get marked out, start using the document structure directly in more places than the raw source — such as for rich inline editing, section extraction, etc

We don’t expect that wiki text-style markup will 100% go away for the forseeable future… this will be a journey, and it’s going to involve a lot of folks testing mass parsers on millions of articles. :) (See some notes from the 2011 Wikimedia Data Summit.)

More to come

That’s about it for now, but there’ll be plenty more fun stuff to come from all those projects and others… I’ll see you all on the internets!