Tales in the City: Adaptive Information in the Physical City

Matthew Chalmers
31 January 2001

Equator is an Interdisciplinary Research Centre, centred on research in computer science and informatics, but strongly interlinked with research in psychology, sociology and design. Its focus is on blending and bridging between digital and physical media, on interweaving electronic devices and information with physical objects and spaces. One project within Equator is Adaptive Information in the Physical City, also known as Tales in the City, which focuses on information structured as tours, histories and narratives, and ‘embedded’ into the physical objects, buildings and streets of the city.

Practical, technological and theoretical interests underlie this project. In practical or political terms, this project is intended as a means to let a subset of the Equator members and disciplines work together, and to cross-fertilise with other Equator projects. It is also a means for us to work with associates and affiliates in the computing and telecommunications industries, and in fields such as museums, galleries, architecture, and urban design. Technologically, it explores new devices and information systems that support mobility, adaptation to context, and access to a mixture of information types. Theoretically, it is a vehicle for an approach to information and computation that takes fuller account of subjectivity and context than currently dominant approaches.

This document is not intended to be the script for future work, but rather a resource for discussion, for planning that work, and for clarifying the interests and concerns that motivate it. While we do wish to formulate project plans for action over the next few months, it seems appropriate to begin with looser discussion, presenting current conceptions and aims while exploring and inviting new ideas. People looking to achieve similar goals in the past have often found that ‘envisionments’ or scenarios can helpfully ground discussion, and so the following section follows their example.

Vee

Earlier today, while on the train to Glasgow, Vee used her mobile phone/computer to take a look at tourist information about the city. She read that the physical interior of the home of Charles Rennie Mackintosh, including a good deal of the designer and architect’s furniture, has been reassembled in the Hunterian Gallery. She has just walked into this gallery, and is heading for the reconstructed house.

A large map-like display stands in the foyer, showing the layout of the house and some introductory images and descriptions of Mackintosh’s work. Vee takes out her mobile, which preloaded the gallery’s web page when she walked into the foyer. Using a stylus and the mobile’s display, she controls the larger display to better see images of paintings and drawings in the house. On the basis of her browsing, Vee has two of the gallery’s guided tours recommended to her. She picks out one of them, agrees to pay the fee, and then puts on her headphones as she steps into the house. In doing so, her mobile is detected and identified by a sensor. This triggers a hidden series of computations and communications: the identification of the mobile’s owner, the fee payment, the download of the audio from the gallery’s computers to her mobile, the logging of another visitor to the gallery. Vee, however, is not distracted. She just hears a description of the first room, tailored to her interests, position and pace.

The rooms, furniture and other artefacts that Vee sees are complemented by the images, text and audio that form one curator’s description of a sequence of paintings and watercolours within the house. A number of pieces are currently in store because of space constraints and restoration work, but Vee is still able to see images and read descriptions of them. She makes a note to come back when the gallery’s public inventory shows that a particular gesso panel is on view. While en route to an upper floor, Vee’s eye is caught by an unusual high-backed chair. The chair was not highlighted in her tour but, since Vee has been moving around it for a while, the guide automatically offers more information about it. It seems that similar furniture was made for Miss Cranston’s Tea Rooms in the city centre. Vee adds this name to a notepad, and tells the tour guide to adjust accordingly.

Along with a record of her progression through the tour and through the building, such actions build up a ‘path’ that records different places, artefacts and documents of interest to Vee. Her path overlaps with earlier visitors‘ paths and with other tours, and can be used to offer recommendations of other places to go and other information to read. As Vee returns to the foyer, she looks at the list of recommendations. It includes the Tea Rooms on Sauchiehall Street, the Mackintosh exhibition in the Lighthouse Centre, a book on the architect, a page on the gesso panel within a web site devoted to Art Nouveau cities across Europe, and a city map. The map shows details of the Tea Rooms and the Lighthouse, as well as public transport options and a walking route into the centre. Choosing the latter option, Vee walks through Kelvingrove Park, occasionally checking her position and route on the map. At a junction, a signpost triggered by the mobile in Vee’s pocket briefly flicks up a version of her map and a message: "across the bridge, up the opposite slope and then along Park Circus".

Approaching the city centre, her mobile’s map highlights the School of Art and the nearby architectural bookshop. The shop appears to have her previously recommended book in stock, and is offering a reduced price for those on guided tours, but Vee asks for more information about the School. Responding to this, the guide offers a new recommendation: a number of other visitors have formed an informal group and have been looking at Mackintosh buildings. The group hasn’t made their location or members’ identities known in any detail, but they have posted a note saying that are open to new members, have arranged to meet a professional guide later on at the Lighthouse, and wouldn’t mind spreading the cost of the guide. Vee sends a message asking them to get in touch, and a few minutes later they call her back. After a brief chat they invite her to meet them at Miss Cranston’s. Time for tea.

Weaving Information into the City

Hopefully the scenario above does not appear as wildly futuristic, and yet it in technological terms it offers an example of interaction, interconnectivity and information not currently available. It involved a currently non-existing degree of communication and co-operation between institutions and organisations such as the museums and galleries of Glasgow, the public transport services, a bookshop, the bank managing Vee’s credit card, and the supplier of a wireless network service spanning the city. Vee used tools such as mobiles, large displays and headphones, and also an infrastructure of networks, computers and information systems, but acted through them to focus on her task or activity rather than on the tools and systems themselves. Similarly, Vee’s movement through the gallery, through the city and through information did not involve a great deal of direct engagement with the institutions and organisations. They let Vee act through them, instead of demanding her attention for themselves. Vee could concentrate on what was important to her at that time: the tour rather than the mobile, the city rather than the map, the activity rather than the technology.

Our project aims to weave digital information into the physical streets, buildings and artefacts that people use, and to do this in meaningful ways i.e. ways that fit, show and support their activity. We tend to focus on the obvious differences between physical and digital media, and treat each one independently. Here, a broader viewpoint takes account of their similarities and interdependencies. For example, ‘the city’ means more to us than bricks and mortar. Our information, understanding and expectations of life in the city influence our activity as much as physical structure. As with information, a city‘s meaning is its use in the language and culture of people. A city space is perceived, as with each building, web page, door handle and scrollbar, through the way we perceive possibilities for activity, and our everyday activity moves between the physical and the informational with little recognition of any boundary between the two media.

Although we continually explore new combinations of media, many combinations are now insignificant, mundane, and everyday. For example, if I read an email and then speak to a colleague across the room about the message, neither of us would comment on the bridge between electronic and face to face communication. If I look at a sculpture then glance at its caption, all the while listening to an audioguide, the correspondence between the three media is unlikely to strike me as remarkable in itself. It is not that there is no difference between communication via email and talking to someone in the same room, or between sculpture, text and speech, but we are familiar enough with the constituent objects, tools and media to act through them and instead focus on our activity. They are so interwoven with everyday life that they are no longer worth noticing as special, novel, or even distinct.

Many electronic and digital media are already familiar and integrated enough in this way, so that activity is no longer exotic, foreign or ‘virtual’. Do we imagine that when the telephone was invented, its use was not just as novel and disjointed as that of ‘virtual worlds’ today? And why, for example, don’t people say that they are ‘entering cyberspace’ when they talk on the phone, play a CD, or read a book? Only a few years ago, wide-eyed Wired readers often used ‘cyberspace’ when referring to email, newsgroups and the Internet. Nowadays this term seems slightly embarrassing and gauche, and ‘virtual worlds’ and ‘virtual reality’ are heading the same way. Virtual worlds may, for the moment, strike us as strange and separate but they are part of the same reality and the same world—of work, leisure and society—as other media. Their novelty will pass, as happened a long time ago for the book, cinema, television and radio, and is happening for email and the Web.

As we use novel media and combinations of media, we weave them together, appropriate them for our own ends, and make them part of our everyday lives. When we recognise that many digital and electronic media are already part of our everyday lives, an aim to ‘bridge the physical and digital’ might seem rather antiquated or odd. Since we can now see that these two are already aspects of the same world, the boundary line of Equator is not between physical and digital media as such, but in that area where appropriation of mixed media is happening and where we make appropriation happen. Equator should be seen not as a static line between physical and digital, but as a shifting and shadowy grey area between familiar and unfamiliar combinations of these media, between everyday and novel, between appropriated and experimental.

The combinations of technologies and media that Equator explores will initially be novel or unfamiliar enough that the combination is itself the focus of attention. Our work will then be directed towards making the media workable as much as towards working with it. At first this will mean workable by us, the technologists, but we must aim to widen our view and loosen our control so that others can experience, explore and express themselves through these media. New mechanisms, devices and systems afford new forms of work, leisure and society. Our work should be based on an understanding of how they might be appropriated into everyday life as well as some idea as to what aspects of life they may create or destroy.

Therefore, we should show a balance between being technologically driven and critically aware. We should be selfish and creative while also being socially responsible and responsive. This may at first seem contradictory, but it is a necessary reaction to the fact that, as with anything we create, we cannot fully predict how new technology will be used and appropriated. Make something new demands difference, creativity and individuality. It is technologists’ awareness of the use and effects of their work that are often criticised, rather than their creativity. Artists’ and designers’ work is very similar, as they express new possibilities for use and interpretation in their work too. Both groups simultaneously create individually and intervene in others’ lives. The ‘users’ of one group are the audience of the other, and the community, market or habitat of both. Working with groups like RCA CRD will require our understanding and integration of contrasting and (hopefully) contradictory uses and interpretations. Opening up our work to public view lets more of the people whose lives will be affected by technology and design influence their development. If we claim that our work will ultimately help, enrich or inform them, they can help, enrich and inform us by offering interpretations and uses that change and intervene in our activity.

A Second (Short) Scenario

The professional guide had finished leading Vee and her companions through the Lighthouse. Vee planned to come back some time soon and so, as she sat in the café, she skimmed back through the route her mobile had tracked over the day. She left a few notes here and there, to remind her of sections worth revisiting, and cut out a few others. She knew some friends would be in the city soon, so she marked a few notes as public.

On the mobile’s screen she could see a dense pattern of paths braiding around the building and out into the city. Some were tours as she’d followed before, some had been left by other visitors, and some had been laid down by local artists and writers. These last art pieces could be rather hit and miss, she felt, but she recognised some of the names and checked for reviews. ‘The List’ magazine gave one tale a good review, so she clicked on it to set it up. Flicking on through the web pages of the magazine took her to the Cinema section, with times and locations. After booking a ticket, she went on to the Music section and chose some music to listen to. As the audio started up in her headphones, she finished her coffee and then headed out of the building in to the lane.

High above her, the tower of the Lighthouse loomed. The time gazing up at the tower was long enough for the tale to be triggered. Mixed in to her music was an old man’s voice, describing one of those dark and stormy nights when murders often seem to happen. Vee turned and saw an image of the old man, projected onto the wall of the building just beside her. Vee decided to follow the story, as it moved off down the lane.

Monomedia

Activity stems from previous understanding, but also feeds back into understanding by creating or reinforcing associations between individual objects, individual spaces and individual people. A person’s movement through data, through the city and through society adds to his or her understanding of information, places and people. One aim of this project is to support this interpretive process, improving information and information systems by representing and adapting with real use, making manifest more of the information and understanding that turn city spaces into ‘places’, and offering people useful and interesting ways to interact with each other. Vee used her mobile and other devices to get information based on her current location and the route she had recently taken, on the information she had read and written, the artefacts she had showed an interest in—and how this activity related to the activity of earlier readers, authors and visitors. The past routes and paths of curators, designers, authors and visitors were combined with her current context to suggest recommendations for the future. This interweaving of physical and digital media, using activity to associate and design text, building, video, museum display, web page, room location, audio track and so on has been called monomedia, in contrast to the rather wan term ‘multimedia’ that too often involves synchronised use of just two media, audio and video.

Computers and communications are used in the flow of the visitor’s activity, both in terms of collections of computers operating by means of wireless networks, and in terms of location and context being used to make systems do useful things given the time and place. We expect to start off with a relatively small, controlled environment such as one exhibition room or gallery, where visitors could use a mixture of mobile computers and static displays to get rich, tailored information as part of their visit. This would combine the artefacts and information they have shown an interest in, and how other people have interpreted them. This might start with guided tours and factual information that ‘official’ authors or curators would write, but we would also expect to weave in visitors’ own paths through locations, physical artefacts and information objects. We might then aim to extend the work to a pair of galleries or museums, to streets between the two, to more people, to a wider range of information, to a larger city area, to different cities... and so on, as far as our interest took us.

Our work will involve combinations of static and mobile devices: small portable devices communicating with each other, with large, static displays, and with server machines across the network. Bluetooth, IR and related technologies will be important here, as will wireless networks, GPS and other larger-scale communications media. Servers will store large volumes of information, of historical and cultural information as well as people’s paths, tours and explorations. Large static displays can offer the resolution and space to show information too detailed and large for small mobiles to handle. Mobiles are ‘ready to hand’ tools that afford not just portability but also individual control of shared technology. For example, a mobile can be used as an input device to control a large display, and of course as a personal source of audio and graphics. It does not just allow a person to carry with them his or her own ongoing information. It can serve that person as a key that represents a role or capability that he or she has, as an identifier to say who he or she is, or as a locator to say where he or she is.

We see resources and issues familiar from computer supported co-operative work (CSCW), such as representation, activity, awareness and privacy. Technology is seen as a medium of communication between people as much as a tool for individual use. Individual action in its social context binds technology into our everyday work, leisure, language and culture. The form of the systems and artefacts we will design in Equator will arise from the ways we perceive the subtle shades between individual and social, between focal and contextual, between local and remote, and between past, present and future.

Vee Again

Soho’s granite walls echo the sky as evening approaches. Vee doesn’t mind the approaching rain, however, as she’s going to the cinema to see the latest sweat-flick. Realising she needs cash to buy her refreshments-wireless transfers are fine for tickets and a cola, but the illicit drug trade still depends on paper-she stops by an ATM.

"Frank has made contact with the pilot who lives in Curzon street," the ATM screen reads as she waits for her cash.

Ah hah, she thinks, that means Frank will be able to check whether the pilot really did smuggle Mara in to England as Victor claimed. Thoughts of the movie drop away as she speculates on this development. She’s tempted to use the ATM again to see whether it will provide more information. But she knows the Tales Consortium curtailed this after the rash of queue-rage attacks when the system was first introduced. She’ll have to find another way.

Boarding a number 19 bus, she starts heading towards Holburn. She scans her pda-phone as she passes the corner to Curzon street. There it is! A short message has appeared, just as she passed. "Victor," it reads, "the consortium knows. Flee at once." The consortium? Who are they? And what’s the pilot doing communicating with Victor in the first place?

The thought of donning a sensor suit in an auditorium full of strangers has lost its appeal. She must know more about the latest developments of the Tale, even if she has to travel through all of London. She jumps off the 19 and hurries across the road to the wait for the 52. A number of tourists are waiting, and she hopes this is a sign that the next bus will arrive soon.

The bus shelter’s destination board is displaying a strange message when Vee happens to glance at it: "a member of the Consortium is here at this stop." As the last letters disappear from the screen, the regular display innocently reappears. She looks around hastily, half her mind expecting to see a shadowy figure, the other half wondering if anybody else is following the Tale-sometimes its useful to exchange notes. But if anybody has noticed the fleeting message, they give no sign.

Vee decides that she’ll have to go to her Portal, the one computer in London on which she can reach the Tale website for clues about the developing situation. As its all the way across town from where she lives, she seldom makes the journey to the internet café where she can find the one computer that has the right hardware IP address to let her on the site. But when the story gets exciting enough, its worth it. Besides, she’s discovered some nice shops and restaurants in the area-it’s nice to get new parts of town, she thinks.

City and Film... City seen through Film... Film over the City

There are places that have been used and reused in films, interviews and documentaries. Sometimes these are well-known places, such as Trafalgar Square in London and George Square in Glasgow, and sometimes the locations are more obscure or less well known. We could open up the history of these places to show clips and fragments of video, inserted, interwoven, or overlaid on the space. This might simply involve letting the clip pop up when in that location, so that you find bump into the fact that the pub you are in was used in Trainspotting (in which case you might want to quickly leave).

We might find clips that show contrasting histories, uses and interpretations of the same place or area. We might see Julia Roberts and Hugh Grant being fluffy and romantic in Notting Hill, set against the street riots that happened nearby. George Square once saw army tanks roll in to quash a rebellious crowd of Clydesiders, which contrasts well with New Year parties and dance music of recent years. We would have to link up with people like the British Film Institute and the Scottish Film Archive. The RCA recently received an archive of London interviews from the BBC—perhaps we could find the locations of these interviews and ‘plant’ them across the city?

Paths with Personality

Rather than tourists or tour guides of the expected ‘curator’ sort, we could follow the paths put down by a greater variety of people. An anti-capitalist revolutionary in central London (Winston Churchill and his mohican turf hair extension, flowers planted in the streets, sheep grazing on the Champs Elysées) contrasted with urbane party-goers glorying in the network of clubs, concerts and galleries that only exists in the city. Understanding the way that a middle-aged German tourist sees the same city centre used by a gang of hairdressers out on a hen night.

Another way of shifting this is to have ‘celebrity paths’ whereby famous people have their past paths laid down, so that people could go to the places in London that John Travolta visited last week (or that his PR company said that he visited last week). There are comparisons to be made with Brenda Laurel’s Guides, but this may be less overtly educational, less narrative in style, more fragmentary and mixing more media?

Be a Dot for a Day

We have occasionally discussed tracking and monitoring people, and this could be extended to making this more explicit. We could offer the ability for people to Be A Dot for a Day, tracked, monitored and displayed as a dot or tag on a map, shown to the rest of the world but without enough detail to be minimally invasive. Alternatively we could see detail about someone real who is far away or from a day ago, or alternatively we could see fictional information laid on to people close by: the person across the square using a phone is talking to the Pope, arranging a gun shipment...

This could also be shifted in different ways, to make the ’dot’ more physical e.g. having lights and images appear around a person as they walk through the city, making a ’glowball’ of streetlights turning on as someone walks across a bridge. Maybe we could take a share of the images of the security cameras that are popping up all over the city, whose images are currently only used or seen by the police. Perhaps those cameras should be more public property anyway, streaming on to Web?

Adverts and Graffiti

Folks at the RCA have been discussing using and reusing adverts on public surfaces, copying and shifting them around, and adding to sequence that’s there, for example taking a copy of a poster that I like and spreading it across town from billboard to billboard. One way this might be easily (but partially) done would be to let people use a public screen to see Web pages selected via their phone or PDA. We might imagine a ground floor window of the Lighthouse filled with a large display that shows a sequence of images in rotation, as built up over time by the selections of different passers-by from a pool moderated by a curator.

Each passer-by could phone up to select a new image to add on to the sequence, or to change the sequence. A more complicated arrangement would let people send in images captured by their phones and cameras, to add to the pool. Some advertisers would deny use on the basis of copyright, while others might encourage it. This reuse and commentary on the imagery of street adverts and signs would let the people who are the passive audience of adverts become more active, talking back in the same language as the advertisers.