I was surprised to hear recently from a few friends (who should know better) that Augmented Reality is already finished?! – killed off by marketing superficiality over the past 6 years?! Of course I beg to differ and actually think we have not even started down a ‘real’ AR road. Augmented Reality is still emerging out of the woods from a technological, editorial and awareness perspective and already rising from a ‘hype-haze’ are the first of some really synergistic applications – ‘historical & futuristic’ factual stories ‘experienced’ in contextual location – Situated Documentary.

Londinium - Street Museum Enhanced AR App

From a present day perspective, the world is becoming saturated by millions of our location stamped ‘social’ stories inside services such as Google Earth, Maps, TagWhat, HistoryPin (more below), Facebook Places, CheckIn+, Foursquare etc: As these stories recede into past events we will start to see some very interesting social and anthropological forms popping up – ARDs (augmented reality documentaries) will be aggregates of the best of those stories and I can see simple parallels – SocialAR extending Reality TV, DramaticAR drawing on Cinema and HistoricalAR evolving alongside Documentary, which is the focus of this post.

There are a range of new services that are various flavours of documentary being delivered in relevant location, as experiential POV events. The ability to recreate historical events, see the world as it was and live temporarily in the past is now becoming a major force in AR falling somewhere between my Intertainment and Experiential Education models from my 16 Business Models post of 2009.

Creating dramatic, emotional and experiential factual stories in this way is the core of transmedia, as carefully placed stories across time, place and platform. I won’t be going into the technologies that deliver these (like Layar, FlashAR or Junaio etc:) or the devices (from smartphones, bespoke handheld devices, glasses, large shared touch-screens all combined with other platforms) but focus on some of the first round implementations made by companies using these relatively crude tools and imagine a very near term future.

Preamble – A State Apart, Teaching Northern Ireland Troubles

As a producer it is always a challenge when on location to bring the essence of ‘that place’ and what it means through your transmedia offering. I produced a pilot while at the BBC  (with Mickey McGowan who made the educational documentary series A State Apart )  a connected, interactive journey through the Troubles.  It’s transmedia pedigree was as a social connected website between schools and the Broadband connected CD Rom elements allowed extended personal story-lines (a small sample is still up) which were newly sourced to act as extended narrative threads from the 6 part documentary.

As part of my asset collection work I travelled to various places such as Belfast, border towns and Derry, met and interviewed people on both sides and went to key locations. One particularly that stuck with me was the Bloody Sunday environments in Derry – the infamous march route in Bogside and the Cemetery where the graves of the 14 shot dead by the British Government in 1972 still remind us of that event nearly 40 years ago. Although I was taking hundreds of images and video, the feeling of standing there, almost hearing the marchers shouting, gunshots, screams was overwhelming.

Ten plus years ago the only tools I had at my disposal was sound based Quick Time panorama, bringing place as exploration out of its environment to peoples screens – one key thing was missing – being there! Today I consider, when creating experiences (drama or doco), the possibility of allowing those who are at the place to immerse themselves via AR, using it as a metaphorical time machine, they can replace the new shops, roads, pristine sidewalks with perspective accurate views, video and using 3D sound headphones become deeply embedded in a past, many pasts – what does this mean for documentary producers?

So going beyond helpful ‘guide apps’ such as the American Natural History Museum app covered by Read Write Web  here are a few other key examples around the world of these early days as we place factual story around us, for generations to build on and explore our important and not so important pasts as we move through the present. Non-exhaustive examples that show an evolving continuum from photos through video through role play to fully immersive education.

Philly History

Recreating past story worlds. There are several organisations around the world with massive archives of photographs and video of their cities or towns and Phildelphia’s Dept of Records are one of them – with nearly 93 000 images already available already on PhillyHistory.org. Andrew Thompson of Azavea contacted me and told me in 2010 they & the Dept. of Records were awarded a grant to create prototype iPhone and Android apps to allow users to see Philly as it was through the ages. It was meant to be a prototype but Azavea (A Geo Mapping Company) described how once they built the engine with 500 images the whole city was far from ‘covered’ so:

“However, even with 500 photos, the team found areas of the city with large gaps between photos.  To ensure individuals across the city could use the app, DOR and Azavea decided to incorporate all 90,000 geocoded images from the PhillyHistory.org collection in the AR application”

Creating a historical record of your locale using this method is really the first step for many, although still tricky, mapping images in a perspectively correct way to modern buildings and places allows an immediate connection for the user to ‘change’, how the world around them is connected to the past. Seeing people walking past a brand new (back then) building a hundred years ago directly in front of you, and imagining what they were thinking, what there lives were like, the stories they left behind is a humbling experience.

“In Spring 2010, the City of Philadelphia Department of Records received an NEH Digital Humanities Start-Up grant to investigate mobile augmented reality technology for use in displaying overlays of historic photographs on the current urban landscape. The project utilized the resources of PhillyHistory.org, a collaborative online database of historic photographs and maps from five Philadelphia area institutions. This paper will examine the state of mobile augmented reality technology and its current and possible applications for use within cultural institutions. Through an exploration and evaluation of the successes and challenges of the Department of Records’ augmented reality project, we will investigate how this technology can be applied to other archives and museums.”

You can read about how they achieved this first stage effort in their online white paper here and the prototype Augmented Reality by PhillyHistory.org application will remain available for download until July 2011. More at PhillyHistory.org

History Pin



HistoryPin is a great social example of how we as a global community can leave a ‘place based’ legacy of our stories for generations to come. I liken this Webby Award winner to leaving residual ghostly stories, thoughts, images to be discovered perhaps 300 years from now by a child growing up where we once stood or someone looking for material for an historical drama or an historian filling in gaps in cultural research. I becomes part of our responsibility and vanity to do this perhaps?

“What’s nice about HistoryPin is it combines that archival collections with user generated content, so we’ve had a year of testing and in that year, we got about 50,000 pieces of content, around half of which are from archives — about 100 different partners, New York Public Library, Boston Museum,” says Stanhope. Via the map, you can explore street by street, you can let the location-aware tours follow you around, take you to the closest pictures, tell you how far away you are from them,” says Nick Stanhope of HistoryPin.com. “Then, when you get to one, you can find the full screen picture, find the stories attached to it, then you can overlay on to your camera view get a really nice comparison between then and now, fade your camera view and the old picture.”

Who Do You Think You Are? Natural History Museum

The famous Natural History Museum in London wanted to take ‘factual’ AR to the next step by incorporating over-layered moving, alpha video and contacted BBC R&D (old cohorts of mine!) to see if TV style ‘Walking with Dinosaur’ type graphics could be incoporated into the new David Attenborough Studio lecture theatre – specifically for an experiential film about evolution. Various partners, complex camera tracking and bespoke handheld displays later the results give an insight into what will be on every smartphone in a few years – not in a controlled indoor environment like this, but all around and anywhere there is a good GPS/WiFi/3G mix!

There is more detail on the project at the BBC R&D blog and a couple of nice images

Street Museum of London & latest Londinium

As reported by many sites (eg: PSFK, Petapixel) last year the Museum of London were one of the first to dip their toes into AR photo mapping and created Street Museum, which like Philly History above maps all those dusty, hardly viewed photos of a cities past, into a smartphone AR app. As it says on the still available iTunes free app

Streetmuseum gives you a unique perspective of old and new London whether you’re discovering the capital for the first time or revisiting favourite haunts. Hundreds of images from the Museum of London’s extensive collections showcase both everyday and momentous occasions in London’s history, from the Great Fire of 1666 to the swinging sixties.

A significant part of this and other more advanced ‘doco smart AR apps’ is the ability to create your own historical tours, community created ghost tours anyone? “Once you’ve got the hang of it, use Streetmuseum to create your own trails around London” –

More significant is the new extended Street Museum module taking us waay back into the past, 2000 years to be precise, Londinium. (see image at top of post for idea of the app ) A collaboration with the History Channel the app will be available in two weeks (26 Jul 2011) and features most notably (in true Situated Doco Space), augmented reality video, layered into the streets – you will see gladiators fighting, builders creating and listen to romans talking as well as some fun ‘interactive’ bits and pieces – from the History Channel site:

HISTORY™ has joined forces with the Museum of London to develop a free new app for your iPhone or iPad. Streetmuseum Londinium gives you the opportunity to see Roman London as it might have been centuries ago. Key Roman sites in London will be brought to life through augmented reality video which will overlay scenes of Roman London onto the view today as seen through your camera. Have a go at digitally excavating Roman artefacts by blowing or using your finger to dig and reveal objects including a leather bikini, a foot-shaped lamp, and a hoard of gold coins. An ancient map helps you to explore the city and uncover what stands there now – did you know The Roman Temple of Mithras stood just moments away from where Bank Tube station is now situated? Or that the Amphitheatre – used for gladiatorial games – was located near St. Paul’s Cathedral?


Augmented Cinema

Related to the AR locatinima element of the previous example  context of this post but I have been having lots of requests over the past year to help filmmakers segment their videos up into locative chunks and make them playable in location. I have also been presenting & blogging how this might look over the past two years and indeed am currently scripting two live ‘locatimima’ original films which really take into account a ‘voyeurs vs viewers’ journey as they take linear and non-linear routes physically through a story.

Gary's Matrix at Sydney Fountain from 2 years ago

So it was also great to see this “See a bit of the film where it was made” app go viral in the past few weeks, suggesting that a lot of people now get this for basic ‘film’ in situ. OK for documentary it is not the most experiential and apart from wandering to various locations, it could still be very passive. In fact there have been several audio based AR apps (that shall remain nameless) that do exactly that, you walk to a spot, listen to a bit of a story, then go to next spot – repeat until bored. The key with all of this is to, like any good multi platform, interactive property, make it personal resonant with a good dose of social! But as I said good to see they have awakened some global interest though!

Dow Day – Vietnam War

In contrast with the previous examples, putting you in the thick of the action and Seriously Gamifying the Situated Documentary (that is turning into a mouthful) we have a few offerings from ArisGames that give you roles and scenarios to play within. Being a journalist, a peace maker etc: in the riots in the US during the Vietnam War anyone – in the location those riots took place!

Dow Day virtually places you in 1967, on the campus of the University of Wisconsin – Madison, when the students began a protest against the Dow chemical corporation for making napalm for the war. You take the role of a news reporter and investigate the different interests and perspectives of students, police and Dow employees.

For more information and insight into these role playing, early days Scenario based Situated Documentaries here is a great paper covering  Dow Day as well as Greenbush, Mad City Mystery and Hip Hop Tycoon – has some good charts & pictures. This role playing and getting inside the documentary by living through characters leads onto the role within formal curriculum…

History & Documentary Education Future through Gamified & Place Based AR

Karen Schrier wrote a great paper / thesis back in Sept 2005 called Revolutionizing History Education: Using Augmented Reality Games to Teach Histories ( online PDF here ). In it she talks about injecting some life into historical and documentary by taking students to the location, and augmenting it through visuals/audio and role-playing. Her paper has some good quotes from participants in trials. From those aware of poor experiences, old school teaching (Using Augmented Reality Games to Teach Histories • Karen Schrier • ©2005).:

“I re-learned U.S. History One, which is what I took sophomore year of high school, and it was a total waste of my time. And I just re-learned it in three hours, which is kinda scary. The course I took in sophomore year and the teacher didn’t really know what she was doing, I think it was her first time teaching it. But what she said or what she taught was really that clear. But this recapped it and I relearned it and now I know more about history. … the pictures, and the items [helped make it clearer].”

Through to those who obviously got something out of the AR experience

“It didn’t feel like a game, it felt like a tool, like an experience, a tour kind of thing. It was an interactive experience, … but I don’t think of it as a video game.” Another felt like it was different than a video game because it was more active, “I think video games are more passive. If this was just a virtual Lexington, and you were sitting at a computer screen typing things in, this was much more active, you were walking around, talking to people, and picking up clues.”

Another article called Learning History with AR in Edublogs concludes with:

Augmented reality immersive interfaces possess the potential to improve student learning and move students beyond predetermined textbook notions of history. History is something students should do, not something they should consume and augmented reality games and simulations can scaffold student learning to the historical inquiry process.

There is a great deal of interest in these ‘enhanced’ school field trips as it both appeals to students as an experience but also has true (Purposeful ‘Edgar Dale’ Play – slide 3) and Pennsylvania teachers have developed a Civil War Augmented Reality Project that uses Layar to map history on notable sites such as the Battle of Franklin which are about to be enhanced with true AR elements.

GHSI – Georgia Historical Scene Investigation (an 8th grade project for Georgia history students) combines elements of all of the above examples with a fun ‘TV-style amazing race format’, social media and game. It is not clear if this has run yet but the video has all the right intentions! This 2MB PDF explains a little background and Camaal Moten, it’s designer, has a great blog on Augmented Reality Education here.

Tourism becomes DocoAR – Tour Wrist, dARsein

To finish I wonder how much of the above will become commonplace so that people arriving in a new location may be tempted to take part in a much more involved Transmedia Situated Augmented Reality Documentary? I think we are already seeing the morphing of the tour guide book into an electronic form for many – the heavy Lonely Planet guide is now a downloadable app. From there we can browse that far and distant world before we go, perhaps taking mini remote tours so we have insights, understand culture and stories. Tour Wrist covered here by FastCoDesign allows you to stand in ANY location and view a completely new world through tablets – and allows unlimited zoom too, you are virtually transported to that place.

But once there the electronic guide starts to evolve. These basic tour offerings from the Fraunhofer Studios (the dARsein tour through the Architectural House of Olbrich and below that the historical tour of Berlin) pave the way for all of the above – locked into using AR enabled smart devices to guide you by the hand will mean all travellers (whether visitors to countries on the other side of the world, or those just walking to their local town) will be tempted to go that little bit deeper, to view the past, to immerse themselves in it, to become someone in that Situated Location Based Documentary.