Part of the Wikipedia Usability Initiative‘s work plan is to do some genuine real end-user testing — this’ll give us a more solid idea of a) what problems to prioritize and b) a solid measure of progress as improvements get made.
An initial test run is starting now: a banner is running for a small percentage of English Wikipedia anonymous page views inviting folks to participate in the study:
This leads to an online survey thingy which the testing firm we’ve contracted uses to pick out candidates for remote or in-person testing:
A subset of the respondants will get a callback from the testing firm, and in a couple weeks we’ll get folks in a lab and smack ourselves with how confusing our site is while we tape them. :)
Currently we’re mostly targeting San Francisco Bay Area locals; next time we think we can trim down the notice a little better to ensure it’s not showing to non-local visitors.