Beneath the surface of things


Whatever you think of the Microsoft Surface device, it’s been fascinating to watch the first steps of development of apps for the new style of interface.

It’s a radically different world now from when we started out web development in the last decade, but it will be just as hard to develop the new ways of thinking about how we interact with machines in this new setting.

When the web was in its infancy, we saw lots of ideas being transplanted from the world of print publishing, and later from the world of advertising. That resulted in some pretty horrible sites being built for many years.

Remember all those years of non-scrolling pages, or the multitudes of ‘single idea’ sites where the navigation was in the style of a cockroach, or a pizza, or a brain, or whatever? That was people taking what they knew about another subject and misappropriating it for a new and much more interactive medium.

But the fundamental design restriction of the early days wasn’t lack of knowledge or thinking about the interaction of computers and people. It was technology itself.

The interface technology – adapted painfully from academic routes – was difficult to learn. And the skills needed to build the functioning parts of these applications were very hard to locate at all. At first, it all had to be written from scratch which few could do, and even if they could, even fewer could bridge the gap between the teams who would build the applications and the teams who could work out what they should do.

With hindsight, we’ve come a very long way in a very short space of time with web deisgn. The world is now full of very talented interface designers, user experience specialists, technologists, development platforms and reasonably coherent ways to get them all to work towards a common goal.

So, the challenge of developing in a fundamentally new paradigm for Surface is quite different. The technology itself is no longer the killer issue. The platforms (WPF, XNA and .net) are well understood, as are the architectures for information flow and manipulation.

Instead, in the user-experience space, we’re faced with a need to deeply rethink pretty much everything we thought we knew.

Here’s an app which allows multiple surface users to search through and display news and entertainment content from MSN: Pulling all the feeds together and building the app are relatively straight-forward but working out how the users best get to that information is a another matter.

In this new setting, we must avoid the non-direct metaphors which characterise GUI, so almost all the controls that we’ve spent the last 10 years standardising have to go. The concepts of buttons, windows and scroll bars should all be vanquished. Instead, we need to find direct metaphors about interacting with objects. We need to find ways to avoid having specific orientation. We need to think about how multiple users can work together with the content. And we need to do all this in way which is seductive and enticing.

Does the news reader do this? Well the feeds themselves may not be a problem, but they are still hardwired to sources that have been created for online applications. Are we avoiding orientation? The now somewhat ubiquitous use of a large circle seems simultaneously too obvious and problematic. Where is the natural metaphor for this? A bowl of sweets perhaps but inside our ‘bowl’ we have a swarm of what can only be described as ‘windows’. And what a rotatable device solves in terms of orientation, it screws up immediately in terms of multiple users, as it faces in one direction at a time.

It’s certainly not easy. And, it’s wrong to pick harshly at early attempts to solve these problems, but we’ll necessarily need to work a lot harder, take more risks and be more experimental if we’re to get more than a large, horizonal web styling.

Looking back to the web, we’re in a position now when so many elements have become engrained in our thinking. We have a great deal to rethink

  • Single source of consistent navigation
  • The concept of a home page
  • The concept of flow being within containing elements
  • The idea that the user opens and closes things (still very prevalent)
  • The concept of linear stand-alone ‘states’ (already eroded by ajax interfaces)
  • Ideas of what it means to be recoverable
  • Single user tasks (even if we only have one active user, we can have multiple concurrent task- or explore-elements alive concurrently)
  • The concept of private browsing – another user’s actions can (and on Surface normally do) interrupt and influence each other
  • The method of ‘what’s next’
  • The entire concept of intermediary metaphors
  • What we mean by scanability for copy and images
  • The role of content
  • The sorts of assets (copy, images and video) which make sense in the medium

The list goes on.

And owing to the nature of the device and its development, progress is likely to be slow. This is accentuated by the polarisation of companies who are developing apps right now – between extreme usability and user experience experts who look at these problems very academically, but also often in a great hurry,  and development companies who are keener to show technology ideas than think in too much depth about what goes on in the mind of the user.

Since Surface is a reasonably long way from being a consumer product, the audience for Surface applications is essentially the same groups – those excited about the hardware and those who’ve spent years learning how you do digital design for the web.

Few apps currently bring the promise of natural user interface, collaborative computing or object recognition. Before we can do that we still have a lot to learn and even more to unlearn.