Category Archives: Papers

Papers I have (co)authored recently. For full papers, best to see my website

Making Time

[This is a draft of a column I wrote for the ACM’s interactions magazine.  It will appear mid 2011].

One morning, as Gregor Samsa was waking up from anxious dreams, he discovered that in his bed he had been transformed into a monstrous verminous bug.

Thus begins one of my favourite novels, The Metamorphosis by Franz Kafka. What is most remarkable about Gregor’s awakening, and his discovery that he has metamorphised into a dung beetle is that, in the minutes that follow, his greatest concern is that he has missed his train.

Like Gregor, time and schedules have been much on my mind of late. Why? Well, firstly, I overslept the other day. My phone is my alarm clock. Sadly, my phone had died quietly during the night. Ergo–no alarm to awaken me. Although I did not wake up a dung beetle, I was nevertheless disoriented. Secondly, about a week ago, I missed a meeting. Well, strictly speaking, I didn’t miss it because I didn’t know I was supposed to be at it. All I can surmise is that there had been a breakdown in the complicated network of services, applications, devices and people that constitutes the sociotechnical practice of time management called “calendaring”. The meeting was clearly listed on my colleague’s calendar, but not on mine.

So, given my recent horological mishaps, I have been ruminating on the concept of time and it’s management through calendars and alerts.

Calendars reckon past and/or future time. The primary purpose of the calendar is the orientation of our bodies and minds–and those of others–in time and space. In contrast to the fluidity of experienced time, calendars create boundaries between activities. They prescribe the amount of time we should spend on something:  30 minutes with Jane talking about her project, an hour for the meeting on budget, 1 hour giving a lecture on HTML-5, thirty minutes on a mandated management course…..and of course, finally, a day of rest.

To be effective social coordinators, calendars require that we share an idea of how time is structured, how it breaks down quantitatively. My minute and yours should both be 60 seconds; thus we can pass time at the same rate quantitatively–even if, qualitatively, for me the hours have rushed by and for you they have felt like swimming in treacle. And, we should share an idea of exactly when 8pm is if we are going to meet for dinner at 8pm.

Calendars don’t just keep individuals synchronised. Calendars, so scholars like the sociologist Emile Durkheim tell us, are central to societal order. Calendars are the sentinels of ‘appropriate’ behavior. Minutes and days and hours often have activities associated with them–indications of when we should work, rest, pray and/or play. Different social values are placed on different hours of the day and on days of the week; in many calendars Saturdays and Sundays are by default given less space, reflecting social norms that separate workdays from (non-work) weekend days. Routine, calendared time is central to creating a social sense of belonging. In his 2006 article, Tim Edensor argues that structured time in the form of everyday rhythms–which he breaks down into institutionalized schedules, habitual routines, collective synchronicities and serialized time-spaces–are how a sense of national identity and belonging is sustained. One can see this play out in my neighbourhood, wherein many different immigrant cultures reside. What is considered an appropriate time for dinner differs by several hours: between 6pm and 7pm for some, between 9pm and 10pm for others.

I suspect most of us take for granted the idea that we have a shared concept of time. However, the carving up of time into seconds, minutes, hours, days, months and years is a convention, and the familiar structure of the predominant western calendar–the Gregorian calendar, which was only in introduced in 1582–differs from classical calendars like the Mayan, Aztec and Inca, and the more recent Julian calendar[1]. Notably, Russia and Greece only converted to the Gregorian calendar from the Julian calendar in the 20th century. Further, it has not always been the case that someone in Bangalore could so easily work out what exactly time it is for me in San Francisco. It was only in the 1880’s that a uniform time was imposed in Britain; until then, time in Britain varied according to location. This local time stood in contrast to ‘ London time ’ (i.e. Greenwich Mean Time (GMT)); Oxford was five minutes behind London, while Plymouth was twenty minutes behind London[2]. In his book The Culture of Time and Space 1880-1918 Stephen Kern writes of the railroads in the US, “Around 1870 if a traveler from Washington to San Francisco set his watch in every town he passed through, he would set it over 200 times”. The railroads instituted uniform time on November 18, 1883. In 1884 Greenwich was established to be the zero meridian and the 24 time zones one hour apart were established. Countries signed up to this structuring of time one by one: Japan in 1888, Belgium and Holland in 1892, Germany, Austro-Hungary and Italy in 1893. At the International Conference on Time in 1912 the telegraph was proposed to be the method of maintaining accurate time signals and transmitting them around the world; astronomical readings were to be taken and sent to the Eiffel Tower that would relay them to eight stations spaced over the globe. This process was inaugurated on July 1st 1913 at 10am. Global time was born, and the death knell rang for the quaint custom of local time. In an odd way, we can thus trace our globally shared, personal and corporate calendars back to the railroads for instigating the rationalization of time across the globe. It’s quite fitting, therefore, that missing the train is foremost in Gregor’s mind when he wakes up.

However, while synchronised global time connects us, it is all too easy sometimes to forget that there are in fact a number of calendars in operation in parallel today–Chinese, Hebrew and Islamic are just three examples.

As I turn back to my missed meeting, I note that calendars have ecclesiastical origins; the Book of Hours structured time into routines for work and worship for monks in the Benedictine order. However, in sharp contrast to the quiet, stable regularity of the liturgical life, my calendar is a chaotic beast in constant need of maintenance and management. Meetings pop on and off like jumping beans as the hoping-to-be-assembled try to find a time that works for all concerned. Vigilence is required lest one is triply booked, and priorities are always being calculated: Is this meeting more important than that one but if so and so is there then that is a good opportunity to get things moving forward…… Oh no, now they are not going to be there after all and yet I am committed to going, how do I shift this around…… and on and on.

The root of the problem lies in the multiples–multiple calendars and multiple people on one calendar. For the first point, I have too many calendars and the effective synchronization of my calendars is not a solved problem. Ghost (long departed/deleted) meetings haunt the calendar on my computer, while my mobile phone presents a suspiciously clean blank slate. Sometimes there is little correspondence between the two, despite their notionally being jacked in to the same server. For the second point, shared calendars (such a good idea in principle) are a gargantuan, social rogue elephant. Herein lie clashes in culture, herein lie power relationships and herein lie a network of complex dependencies. Routine issues arise for me in the following forms: blank space on the calendar, the curse of durational rigidity, the clash between sociotemporal and biotemporal time, and the problem of travel time. Lets briefly review each of these…..

Idle’ time People routinely look at my calendar to determine when I am free to meet; they plop meetings on my calendar based on what they see as ‘free’ time. This is based on a fallacious assumption–that if there is nothing recorded there, then I am free. This is a misreading of my practice of calendar use. Booked times on my calendar are not simply islands of colour in a collaborative paint-by-numbers schematic where the blanks are inviting others to fill them in–I saw a gap so I filled it.

Of course, idle time is anathema to the shared calendar in a culture where to be not actively doing could possibly be interpreted as shirking. In my view, day of back-to-back meetings means there is too little time for creative thought or for reflection. Research indicates that time when one is doing the least, as for example when meditating, is when the most creative moments can occur[3]. The jammed calendar, continual context-switching and mad dashes from one location to another are emotionally draining, mania inducing and counter to creativity.

So I sometimes put “meetings” onto my calendar to simply block some thinking time. I feel sheepish about this. I am reminded of a friend of mine, who, when we were teenagers, used to write things like “peas and carrots for tea” in her journal. Recording peas and carrots was not because of some dietary obsession, they stood in as code for ‘held hands’ and ‘kissed’, reporting on her teenage encounters with her boyfriend; the code was invented lest her mother should read her journal and be mortified by her teenage explorations. So, it is that I transform thinking, writing and reading into ‘Strategy’ and ‘Planning’, appropriate behaviours for a corporate context. Durkheim and followers are correct: how one manages one’s time is an issue of morality and social accountability, not just temporal coordination. It’s a tricky business.

Durational rigidity For the operationally minded, a meeting that is scheduled for an hour must last an hour even when nothing is being achieved. On the other side of that, sometimes one can be just warming up, just getting to the crux of a problem and the hour is up, the meeting has to end truncating the creative process.

Travel time Another problem, and one where a simple technical solution would help out, is travel time between locations. When one works in several different office buildings that are miles apart, it takes time to get from one to the other. It would be useful if I could hook my calendar up to these locations, and have travel time calculated and reflected automatically. So if a meeting is dropped onto my calendar, travel time is automatically blocked in–in fact, I could imagine a lot of background calculating that can be done by hooking my calendar up to location and to my social services and applications[4].

Biotemporal time Working across time zones can be really hard. The cheerful calendar flattens time, it sees all times as equal. Calendars are simply tabulated time in a grid, they do not reflect lived time. Odd times for calls can sneak in there, creating social and personal dilemmas–I want to be a good citizen but I know I am going to be less than my best at that time. Sociotemporal times (as in when it is appropriate to be working and when not) clashes here with biotemporal time. Being on a meeting conference call when your body and your entire environment tells you that you should be sleeping is simply hard. Time may be global but my body is not.

None of my observations are earth-shatteringly novel. There has been a wealth of research in the HCI community from the early 1980’s and continuing now today, on life scheduling and calendaring–in collocated and in distributed workgoups, in the home, in leisure groups, within families, between families, on paper, across paper and other devices, on personal computers, using mobiles, using location services and with visual front end experiences including3D representations. Just to name a few of the research directions. There are typologies of calender user type such as that offered by Carmen Neustaedter and colleagues who call out three different types of families—assigning them to the categories monocentric, pericentric, and polycentric according to the level of family involvement in the calendaring process. Monocentric families are those where the routine is centered on a primary scheduler, pericentric families have the calendar routine centered on the primary scheduler with infrequent involvement by secondary schedulers and polycentric families are those where the calendar routine is still centered on the primary schedulers, yet secondary schedulers are now frequently involved. BUT despite all this work, there’s still plenty we can do in the world of sociotechnical design to rethink the calendar. My calendar does not feel “_centric” in any way; it feels chaotic.

“We shape our dwellings and afterward our dwellings shape us” said Winston Churchill in 1943. We could apply this observation to time; we shaped the calendar and now the calendar shapes us, it dictates how we (should) live. True to Louis Sullivan’s adage form follows function, the digital calendar wears its assumptions and its intellectual heritage on its sleeve: computer science, psychology, information architecture and the ethical structure of the approved-of day. Perhaps we need a new tack.

In Branko Lukic’s and Barry Katz’s 2011 text, Nonobject, they explore product designs that sit at the interstices of philosophy and technology. They step back from simplistic notions of form and function to shake up how we think about products, to question what is ‘normal’ or taken for granted, and to question the values that are embedded within the typical form of everyday artifacts. In a section entitled Overclocked, they explore clocks and watches, our time-keepers. Katz writes, “as our measuring devices grow ever more accurate, we find ourselves perpetually “overclocked” to use a term familiar to every computer hacker who has ratcheted up a component to run at a higher clock speed than it was intended for in order to coax higher performance out of a system. We do the same to ourselves.” A number of designs are presented: the Tick-Tock Inner Clock that taps against the skin to let someone feel the passage of time and the Clock Book where time is laid out on pages we can turn–when we want to–push. Lukic’s watches and clocks invite us to rethink we conceptualize, represent and manage time. Somewhat less extreme but nevertheless taking a playful take on clock design, Alice Wang’s 2009 suggestion for the Tyrant alarm clock is brilliant. This alarm clock calls people from your address book on your mobile phone every three minutes if you don’t get up and turn it off; with this, Wang is betting that the anxiety of broadcasting your slothful habits to anyone in your address book will propel you to get up. Wang gleefully reports that it is the social guilt that will get people moving out of bed. Social anxiety has long been a driver for action; this is I think a nice example of it, and this is a step beyond thinking instrumentally about the clock’s utility/function in isolation from the rest of one’s life.

Let’s do the same thing with calendars. Let’s take a step back. Let’s follow Lukic and take our lead from Architectura Da Carta, the Italian tradition of articulating and illustrating the unlikely, the unbuilt and the unbuildable. Let’s use art, philosophy and technological creativity to envision a better aesthetic experience, to blast the calendar apart and rebuild it; let’s be better about enabling the plularity of private and public times that humans live in parallel; let’s automate the calculation of time in motion between location(s); let’s build in time for creativity and reflection as social and moral imperative; let’s make a calendar that adapts the schedule when it realizes you have woken up having metamorphised into a sentient dung beetle.

[1] See Anthony Aveni Empires of Time. Calendars, Clocks and Cultures, New York: Basic Books, 1953

[2] See Journal of Design History Vol. 22 No. 2 Designing Time: The Design and Use of Nineteenth-Century Transport Timetables by Mike Esbester

[3] See for example The neuropsychological connection between creativity and meditation published in ‘Creativity Research Journal’, 2009 by Roy Horan

[4] See Lovett and colleagues on this in their Ubicomp 2010 paper: The Calendar as a Sensor: Analysis and Improvement Using Data Fusion with Social Networks and Location


The golden age of newsprint collides with the gilt age of internet news

[This is an early draft of a column I wrote for ACM’s interactions magazine. It appeared here and the final version is available from here. It appeared in Volume 16 Issue 4, July + August 2009 of the magazine].


Sitting in the Economy Class seat on a United Airlines flight, I ducked for the third time as the gentleman next to me struggled to turn the page of his broadsheet newspaper.

While he was assimilating what was happening in the world, I was contemplating the unfortunate juxtaposition of two iconic forms – the over-sized broadsheet newspaper and the undersized airline seat – and the current state of two industries that find themselves in deep financial trouble.

News stories. Crosswords. Horoscopes. Book reviews. Political cartoons. Recipes. Print-dirtied fingers. Papier mache. Stuffing sodden shoes. Wrapping fish and chips. Ad hoc packing materials. Fire kindling. These are things that I think about when I think of newspapers. And despite the fact that I could never quite physically control a broadsheet without the aid of a table, I cannot believe that this everyday artifact may go away. But according to my friends here in the digiphilic environment of San Francisco it is inevitable – you can’t walk into a coffee shop, never mind turn on a TV or the radio without hearing someone opine about economic crisis that newspapers are facing and the likely disappearance of the daily rag. I am as shocked and mortified by this as I was by the 2003 news story that bananas may be extinct by 2013.

Newspapers have a long history. The first printed forerunners of the newspaper appeared in Germany in the late 1400’s in the form of news pamphlets or broadsides, often highly sensationalized in content. In Renaissance Europe handwritten newsletters circulated privately among merchants, passing along information about everything from wars and economic conditions to social customs and “human interest” features. In 1556 the Venetian government published Notizie scritte, for which readers paid a small coin, or “gazetta”. The earliest predecessors of the newspaper, the corantos, were small news pamphlets that were produced only when some event worthy of notice occurred. In the first half of the 17th century, newspapers began to appear as regular and frequent publications. The first modern newspapers were products of western European countries like Germany (publishing Relation in 1605), France (Gazette in 1631), Belgium (Nieuwe Tijdingen in 1616) and England (the London Gazette, founded in 1665, is still published as a court journal). These periodicals consisted mainly of news items from Europe, and occasionally included information from America or Asia. They rarely covered domestic issues; instead English papers reported on French military blunders while French papers covered the latest British royal scandal. Newspaper content began to shift toward more local issues in the latter half of the 17th century. Still, censorship was widespread and newspapers were rarely permitted to discuss events that might incite citizens to opposition. Sweden was the first country to pass a law protecting press freedom in 1766. Timeliness was always an issue; news could take months to reach audiences. The invention of the telegraph in 1844 transformed print media. Now information could be transferred within a matter of minutes, allowing for more timely, relevant reporting, and newspapers appeared societies around the world. This was truly a revolution.

The Internet is bringing about an even bigger revolution: timeliness, open rather than controlled information sharing and easy access. This shake-up is bigger than any other that has been faced in the last 100 years from the likes of radio and television. Broadcast radio in the 1920’s was low-cost with broad distribution, and content delivery was often more timely. The newspapers responded by adding content that was not so easily represented through audio waves, providing more in-depth and visually vivid coverage of key stories. As the 1940’s and 1950’s came around, television appeared as the main challenger. Newspapers again responded, taking from television the short, pithy story format. Newspapers like USA Today responded with graphics and colour imagery. More generally, news publications started diversifying their content, mixing human interest stories with puzzles, crosswords, book reviews, cartoons, cooking recipes and all the good stuff we have grown to love. Newspapers became about browsing, grazing, sharing, surfing, with content that satisfied immediate information needs and longer-term general interests.

Despite radio and television, newspapers managed to retain their position in the information value chain. Not so anymore. There are three interrelated causes for this shift in the information ecosphere: internet-related innovations in news production and news dissemination; the impact of new digital devices that are changing the ways in which content is consumed; and a no longer viable business model.

Lets quickly look at these in turn. It is obvious that the Internet has revolutionized news production and dissemination. Speedy transmission of information around the globe means news can reach us as events are unfolding – hot off the keyboard rather than the press with images and video for that “being there” feeling. “Citizen journalists” give us the lay-person’s perspective on events that journalists cannot or have not yet reached. Indeed, reports from various disasters from the fires in California to the shootings in Mumbai came for many people first from Twitter, the micoblogging service that is currently the darling of the media and blogs from Iraq told us much more that we could possibly find out from our daily newspapers. The efficiency and effectiveness of this interconnected internet world cannot be denied. Production and consumption of news has also been transformed by the explosion of lightweight, wireless, internet-enabled recording and reading devices, plus the proliferation of computers in the home and in offices. Finally, the old business model is failing. The newspaper industry in the US has been generating most of its revenue from advertising for decades. The global recession and the resulting decline in advertizing revenues has dealt a possibly fatal blow; the Newspaper Association of America reports that in 2008 the total advertising revenues declined 16.6 percent to $37.85 billion, representing a $7.5 billion reduction on numbers for 2007. Proposals on the table for saving the industry now include micropayment schemes plus bailout and/or government subsidies.

I don’t feel qualified to assess the likelihood of success for the various bailout schemes from micropayments to government bailouts, and for the purposes of this column, I will not go into the importance of ensuring we don’t lose good journalistic practice. But I am really worried about what is happening in the world of news, because I am screaming for a better news reading experience on my desktop and mobile devices. What the news industry at its best did really well is missing from the online reading experience: easy navigation of well filtered content plus effective selection and segmentation of content plus a clear voice/view of the publication. Can we take the best of what we had in newsprint and create a good digital news reading experience? Here are some basics I would like to work on:

(1) information quality- can we provide better tools for the collection and management of information gathered on the ground that would aid with quality and provide guidelines for the coupling of different media types (text, imagery, video) to avoid gratuitous visuals? Let’s be active in designing better technologies for production of the news by citizen and professional journalists and editors.

(2) information architecture design: can we design better relational models so we can surface relationships between stories that are actually meaningful instead of the ‘also see’ hyperlink that takes me to a story from 5 years ago that somehow got linked to the current one? Can we design better tools for following story developments, for enabling the creation of narrative by producers and consumers?

(3) can we improve the representation of information – graphics, fonts, layouts – so that it is possible to skim more effectively?

(4) can we design for reading the news – what is next after the Kindle? Is electronic paper or Xerox’s promised reprintable paper going to be a reality so I can have the large gesture, embodied experience of the broadsheet back and decent screen real estate for laying out content?

(5) can we design anything better than the crass, ugly and inconvenient model of url bookmarking to support different temporalities of information usefulness and different consumption paces, and for slow-burn stories to persist while fast-burn stories are updated with new content?

In sum, we should more deeply address the practices of news readership. We should design for convenience and skimming. We should design filters and surfacers of quirky items or items that for some reason search algorithms find unpalatable. We should develop better editorial tools than we currently have.

I am not alone in wanting some good design heads on these problems. Addressing people’s everyday news consumption practices, a 2008 Associated Press ethnographic study cited email and internet-based sources as a mainstay many young people’s experience of the news. However, these interviewees, plus interviewees in a study I am currently running in the Bay Area all talk about the “work” of reading the news online and that “news fatigue” is increasing. What this seems to boil down to is that there are plenty of places to find news on the internet, but in all this bacchanalian information glut the shallow story dominates, often it is hard to find the follow-up to a reported news item, and there is a lot of repetition. To the last point, the Project for Excellence in Journalism observed in their 2006 State of the News Media report that 14,000 unique stories were found on an internet news aggregator site in one 24 hour period, there were in fact only 24 discrete news events. There is vastly more content available of course, and things have improved somewhat since 2006, but that other content is relatively speaking hard to find. And online content usually does not offer the structured, well-designed experience that its printed counterpart does. Ethan Zuckerman of the Berkman Institute blogs about his experience of a national newspaper’s online presence “…counting possible links (using a search for anchor tags in the source HTML), there are 423 other webpages linked from the front page. A more careful count, ignoring ads, links to RSS feeds and links to account tools for online readers, gives 315 content links, possible stories or sections a reader could explore from the front page. While there are almost 14 times as many pages for a reader to explore, they’ve got much less information on what links to follow: while twelve stories have text hooks, the wordcount ranges between 10 and 26 words. While there’s a good chance one of those stories might convince you to click on it, you won’t start reading it on the front page, the way you might with the 200-400 word stories in the paper edition.”

I just replicated his analysis by looking at three online papers. He’s right.

But let’s not get ahead of ourselves or judge too early. We are only at the beginning. Right now we are at the stage the car was in the last 1800s and early 1900s – in shape and form reproducing the horse drawn carriage, not yet having found its own aesthetic reflective of its infrastructure and capability.

Newspaper companies are on board with enlisting others to aid in the design of the next generation of news forms. In early 2009, the New York Times Developer Network hosted its first API seminar so we can start designing and building new forms of content provision. The aim is to make the entire newspaper “programmable”. Programmers will be able to mash up the paper’s structured content — reviews, event listings, recipes, and so on. This is a great opportunity for those immersed in information and experience design.

Some are for example pushing on really nice “read later” and bookmarking facilities. This is a good start. But you still have to know how to search for content, and spend time doing that. In my opinion, today’s aggregators/algorithms and automatically updated webpages are in no way shape or form replacing the work done by a good editor and a good layout designer.

As I have been thinking about news, I conducted an informal review of a couple of local and national newspapers. Bearing in mind that I am only am interloper who is curious because reading the news is integral to my identity, I took a quick review of the differences between online and print versions. I could not discern any consistent re-representations between online and print. I wonder: Are there standard reformulations and standard channel “jumps” for different content types? Who is making those decisions and how? How is the “shelf-life” or temporal relevance affected by the channel or the medium? And…… what of the services that were once the purview of the local newspaper have not been reformulated or replicated? What has happened to the content that once constituted the local daily paper? How has it morphed and reformed? Finally, and of course, I note that I have not seen many people argue for the fact that the very form of the newspaper, it’s affordances (size not withstanding), may be missed. No indeed, the print form of the newspaper still has affordances that cannot be matched by the digital medium. These are, to me at least, complementary forms, but perhaps I am in the minority; I still print things out rather than reading them on a screen.

And, there is another affordance about paper. It gets left lying around, apparently discarded but ripe for re-reading. That is not the case with contemporary digital devices, although who knows where the future may take us as device manufacturing gets cheaper. Information left lying around for others to consume is important. It allows others who are idling to encounter than which they may not have otherwise; to be literal, if I sit on a subway train and out of curiosity read the newspaper that has been discarded on the seat next to me, I am encountering something I did not choose, that was not filtered for me. And just perhaps, I will learn something unexpected.

Addressing issues in the creation and dissemination of news is important; it is not enough to say that late adopters or those who not actively seek the news as opposed to having it literally pushed through the letter box should catch up with us digerati and get those phone applications downloaded. Making it harder to get access to information affects civic engagement. Following the closure of the Cincinnati Post in late 2007, Princeton University economists Sam Schulhofer-Wohl and Miguel Garrido showed a decline in people voting in elections, fewer candidates running in opposition to the incumbents, and less knowledge of and debate around issues and policies supported by the incumbents. If democracy in some sense depends on an informed electorate, then making it harder for people to easily find digestible but detailed and well-balanced arguments is a serious problem. Even if you don’t agree with everything you read in a newspaper, encountering things that you have not actively selected broadens your outlook, the flip-side of how filtering and narrowing can save time. Specifically, filtering saves time, but also shuts down challenges to assumptions, and it is those challenges that help us grow and create the debate of a functioning, democratic society. Newspapers can of course do the opposite; they can function to bring a community together in a shared narrative (whether they agree with the narrative or not will drive whether then accept or fight it). Moving to purely digital forms, some say could increase ‘discovery’ problems for the non-digerati, and thus the number of ‘news dropouts’.

So, I confess that personally, I love the materiality of a good broadsheet newspaper and of the magazines that I read. And it annoys me just a little that, thanks to my beloved Kindle electronic reading device, I don’t have newspaper lying around the house to stuff my rain sodden shoes. But I am also looking forward to a world with better designed digital news formats. What we need is some technical savvy, a design sensibility and a deeper human-centered understanding the Gestalt of news consumption in practice between and across representational forms. We need something more that the current state of the art, which offers us only the most superficial, easy to implement, of technical convergences. We need more than the horseless carriage of digital news.

Keep your hair on: Designed and emergent interactions for graphical virtual worlds

[This is a draft I wrote for my column, P’s&Q’s in ACMs interactions magazine. For it, I interviewed Bob Moore who was at the time with now defunct Multiverse. It came out in Volume 15 Issue 3, May + June 2008. The final version can be found here.]


Chatting with virtual world researchers Jeffrey Bardzell and Shaowen Bardzell, I found out that a seriously desired artifact in Second Life, not unlike in First Life, is hair. Swishy, shiny hair. But there is one problem with this real-dreamy hair: Such hair is computationally costly compared to render in comparison to your average avatar body. And so, sadly, your avatar body arrives before your hair. For a matter of moments, no matter how fashionable the outfits, everyone is bald as coots.

And it’s worse than that. Often from other players’ perspectives, your hair has failed to “rez,” but not from your perspective. So you think you look hot-REALLY hot–until, that is, some newbie says, “Why are you bald?”

There are now apparently a lot of people routinely spending time hanging out in virtual worlds. MMO Crunch reported 36 million regularly active MMORPG players in August 2007. And they are getting more popular.

“Virtual worlds are fundamentally a medium for social interaction. One that takes the face-to-face conversation as its metaphor. As such it leverages users’ common sense knowledge: I see a humanoid avatar and I know that if I want to talk to that player, I should approach his or her avatar with my own” says Bob Moore interaction researcher and designer at Multiverse. As with all metaphors, however, there are fractures in the metaphor. Which matter and which don’t? When do we happily immerse and feel like we are really there and when do we get amusingly or annoyingly jolted by something that doesn’t work? What are key things designers are thinking about right now? Bob and I chatted for a while, reflecting on the work that has been previously done on the research areas of graphical worlds, collaborative virtual environments (CVEs) and online gaming environments. Given my own background in researching text-based virtual worlds and animated interface characters, I was intrigued to her Bob’s perspective. Here are some of the main points we covered in our conversation:

(1) For virtual worlds, user interface design is also social interaction design

Bob argues that one thing UI designers for virtual worlds have not yet fully grasped is that when users are interacting with the system, they are often at the same time interacting with other users. When standing avatar-to-avatar, if you ask me, “Did you get the sword?” and I promptly open my inventory, that UI action is a relevant part of the interactional context. Or if I say, “Let’s go” and you promptly open your map, that’s a relevant next action that I should know about. In almost all current virtual worlds, opening your inventory or your map triggers no public cues, only private cues for the individual user. So you not only need to give the individual user feedback about what the system is doing, also need to give the other players feedback about what the individual user is doing. In other words, users’ interactions with the system should be made public.

(2) Where avatar bodies are not like physical bodies

Unlike real life, most people tend to play virtual worlds with their camera view zoomed back so they can see their avatars rather than in true first-person view where you can only occasionally see your hands or legs. There are a couple of good reasons for this. Computer screens don’t allow for peripheral vision. But pulling back the camera can help mitigate this limitation by widening your field of view. Similarly, as Bob points out, avatars don’t allow for proprioception, or our awareness of the positions of our body. Zooming back the camera also helps players deal with this fact. I may think my avatar should be waving like the Queen of England or winking flirtatiously at someone else because I typed /wave or /wink, but it’s hard to be sure if I can’t see my avatar. Maybe I mistyped the commands or maybe the animation associated with the command actually looks more like a New Yorker hailing a taxi than Her Royal Highness.

(3) When the conversation lags behind the avatars

Despite a lot of the interesting audio experiments that were being conducted over 15 years ago–see Benford, S. D. and Fahlén, L. E., Awareness, Focus, Nimbus and Aura – A Spatial Model of Interaction in Virtual Worlds, Proc. HCI International ’93, Orlando, Florida, 1993 as a good example–most virtual world conversations take place through chat. The avatars may be wandering about gesturing and wiggling their hips, but chat does not come out in audio from the mouth of the avatar. One problem this causes is the discontinuity, the lack of congruence, between action and uttered words. Bob points out that typing a chat message is another kind of action that other players should know about, and he recounts a case in which a team of players is about to attack a group of “mobs,” or computer-controlled opponents. While one player is composing a question about how the team might change its tactics, a fellow player initiates combat. The tactical question then publicly appears too late. The pseudo-synchronous chat lags behind the synchronous avatars. Bob, who was trained in Conversation Analysis, explains that a key feature of real-life conversation is that you can hear a turn unfolding in real-time. This enables you to do things like determine who should speak next, anticipate precisely when the turn will end so you can start your next turn with minimal gap and overlap, and even preempt the completion of the current speaker’s turn if you don’t like the direction its going. In other words, the ability to monitor other people’s turns-in-progress is a requirement for tight coordination in conversation. Most virtual worlds (with the exception of There) use IRC- or IM-style chat systems, and therefore, do not allow players to achieve this tight coordination among their turns-at-chat and avatar actions. The result is an interactional experience that feels very unnatural (at first) and which motivates players to invent workarounds to the system.

(4) Perennials of place

One of the amazing things about virtual worlds is how quickly we get a sense of being co-present in a place with other people, even though it may be an image on a screen, a world into which we are kind of peering. And, as in the real world, ambience is created by building and room size and scale in relation to crowd size. In my research on MUDs/MOOs with Jeni Tennison and then later work with Sara Bly, we found that even very simple text exchanges in textually described “rooms” can make dyads and groups feel co-present, imersively in the virtual world together. Humans tend to get engaged with each other as long as there is some consistent chain of action-reaction. Indeed some analysts would argue that turn-taking in conversation is the fundamental unit of human communication and connectedness.

As we explored these concepts, Bob described a comparative ethnographic study he’d done of bars and dance clubs in multiple virtual worlds. He hung out in these social public spaces and analyzed features of the design that impacted the success of the space as a social environment. One key feature of club design is size. Bob discovered that, while construction in these worlds is cheap compared to real life, it is more difficult to fill these spaces with people than it is to fill the real-life urban centers that researcher William H. Whyte examined. As a result, the dance club in City of Heroes and the majority of player-built clubs in Second Life are simply too large. They feel like an airport terminal or concert hall rather than a corner pub.

So in order to achieve the kind of social density necessary for a vibrant social space, or “third place” as academic Ray Oldenburg would call it, designers should make virtual bars and clubs much smaller than they currently do. The most successful virtual third place that Bob discovered was a Second Life bar that was intentionally tiny. In order to get into the place, you had to “rub elbows” with other patrons. The place felt “busy” with only five players and “hoppin’” with twenty. And everyone was within everyone else’s chat radius, which facilitated the public conversation. In other words, lessons from real-life urban design appear to apply in several ways to the design of virtual public places.

(5) A range of skill sets and a modifiable world.

There are challenges in designing a world where “newbies” or newcomers can learn the ropes. If you want to learn about interacting in one, you simply have to get off (or rather for most people onto) the sofa and get in there. Go in-world. It is much easier to learn in-game than to learn out of game, just like learning to play golf requires you take up a golf club and try it. You just can’t learn by watching someone else. Bob says, “I’d recommend getting into a pick-up group and go and doing some adventuring”. There are plenty of folks in-world who are willing to help out, to show off their knowledge. Bob agrees that not everybody likes to help, and admits that whether you are more of less likely to be helped depends on some other factors……you guessed it-having an attractive female avatar means you are more likely to get help.

There are other things to learn aside from interactions and activities. Many virtual worlds allow people to buy, build and exchange things. Second Life is perhaps the primo example, it is a sandbox with a constructive geometry, which enables them to stream your data and create your objects on the fly. But that also means people arrive and build stuff and leave it behind. There is a certain  “I just learned to build today” look that Bob identifies as one of the main scars on the aesthetics of the virtual world. Despite the parallels that some people make between the real live music/art/dance festival known as the Burning Man Project and playful interactions and explorations in Second Life, there is no motto inviting us to “leave no trace” in Second Life. Of course the ecological consequences are somewhat different but the visual aesthetic of clutter and detritis is experienced as the same for Second Life afficionados. So, there is a tension between giving people freedom to build anything they want and making sure the world doesn’t end up looking like the aftermath of an afternoon in a Montessori School for Gremlins. Beyond clean-up though, there is another point. At this juncture the building tools are much better than they were but they still leave a lot to be desired. I say: If we want Gaudi not Brutalism, we need to provide better tools to scaffold the building endeavours of the folks in-world.

To get an understanding of issues like these above and push on understanding how people really experience these virtual/online places, Bob advocates a close and detailed analysis of what is actually going on as it unfolds in real time, looking at patterns of action and interaction, how those patterns develop and are understood, learned and evolved, and identifying patterns that are persistent and prevalent. Too many people have theories about what is going on that are based on something completely external to the situation. “You need to get close to the phenomenon/experience” says Bob. And we need to “be concerned both with the in-world simulation of face-to-face interaction AND the usability of the interface for puppeteering the avatars and interacting with the system.”

By looking at the challenges in interaction that people routinely encounter and work around, it is possible to ask how important – or disruptive to interaction in-world – those challenges are, and propose ways to address them through interaction, interface and system (re)design. I could not agree with him more.

And on that note, it’s time for me to get my hair on and go build a shack.

Reference: Bardzell, S., & Bardzell, J. (2007). Docile avatars: Aesthetics, experience, and sexual interaction in Second Life. Proceedings of British HCI 2007. Lancaster, UK