Jan 012012

Originally published Oct 2011 in Wired Magazine ‘Change Accelerators‘ by Gary Hayes 4 of 5

Image by Gary Hayes

When planning your next holiday to London with the fam, don’t forget to sync up your iGlasses and load up the London experience packs. On arrival, slip on your augmented reality sunglasses and take a look around: Roman-era London appears before your eyes. Slaves and gladiators walk through the streets and chariots rush past. You can add your own comments leaving virtual “We Were Here” graffiti for all time. The experience is part documentary, part user-generated narrative, and entirely pervasive. In other words, augmented reality meets living history.

In our everyday lives, we engage with stories in many ways, whether it’s eye-to-eye contact with a stranger that sparks an instant connection or a well-crafted movie or TV show. But what if we started experiencing those stories in the outernet’s layers?

While online networks are evolving traditional entertainment, such as TV and web series, we are also witnessing the rise of a new form of media called “augmented reality storytelling.” I’ve dubbed this new form of diversion ’ntertainment, as a shorthand for immersive augmented reality entertainment.

At its broadest level, augmented reality is about enhancing the physical world through digital elements, such as images, sound, and information. Now technology is enabling us to further situate and layer our digital stories in places where other narratives can’t reach. Right now, we see this happening when someone holds up a camera on an iPhone or tablet and shares objects or stories from the real world.

The opening Roman London example is based on an existing service called Londinium, which is a collaboration between the History Channel and the Museum of London using augmented reality video layered over real-world streets to re-create an alternate history. Coincidentally, London is also used as a site in the globe-spanning Ghost Tours 2.0. Haunted London encourages visitors to explore the city’s eerie side using locative AR (augmented reality). Likewise, another situated project is Witness, which draws participants into the dramatic and seedy underbelly of criminal Berlin. In this case, players are the hero: They watch graphic video scenes at different city locations and are then sent detective challenges to uncover the truth. But here’s the twist: The story might just bite you back! Augmented reality games and stories can even get physical, like the recent example of Chelsea FC playing the world’s largest Space Invaders game in a stadium using projection AR.

Gaming is leading the way. New consoles, like Vita, allow users to literally take game characters orreality fighters into the streets. Other gaming advances like AR games on Nintendo’s 3DS start to recognize place markers placed around a player’s city, transforming screen-based MMORPG(massively multiplayer online role-playing games) into an LMMOG (location-based massively multiplayer online games).

Augmented reality storytelling is starting to appear across our smart GPS mobile devices. Several marketing campaigns are taking the initiative by spearheading real-time AR challenges, such as Vodafone’s Buffer Monsters, which challenged German smartphone users to download a mobile app to capture virtual creatures and win a lifetime plan. This is only one example, other AR advergames encourage users to competitively run around cities on scavenger hunts for real-world prizes, such as the Droid Bionic AR Game. Similarly, this October, Gundam, the Japanese anime giant, release an iPhone/iPad app called Gundam Area Wars. The game uses the devices’ camera and gyroscopic sensors to show life-size 3D models situated in the player’s real-world landscape.

Given these above examples, I return to my earlier travel scenario and I wonder how commonplace it will become for people arriving in a new location to start experiencing it through augmented reality storytelling and gameplay? The traditional guidebook has already morphed into digital form. The Lonely Planet is already a downloadable app. Is it a big jump to imagine AR and location-based storytelling won’t soon allow travelers to engage history on a whole new level? One might even argue a deeper and more meaningful one than just the 2D sightseeing experience of looking at crumbling ruins. So many guidebooks have been written on the principle of making history come to life—AR actually makes it possible.

One could even take this one step further and question, why do we need to travel at all when we have our own personal Holodecks at our fingertips? Fast Company recently reported on Tour Wrist, a virtual tour that lets iPad users move around a global location with unlimited zoom and freedom. “Travelers” are virtually transported to that place and able to immerse themselves in it becoming the hero in a remotely situated, digital storyworld.

Finally, in the near future, we might all have the capability to create duplicates of our surroundings in 3D for others. This Microsoft R&D initiative to map the world uses the fastest selling piece of tech on the planet, the Xbox Kinect. This would allow everyday people to create unlimited user-generated 3D AR—foreseeable as easily as snapping a digital picture. In addition to this, there is a saturation of location-stamped social stories inside services, such as Google Earth, TagWhat, HistoryPin, Facebook Places, CheckIn+, Foursquare, and Gowalla, among others. What will result from all these stories becoming interconnected and navigable using AR devices?

From that point on, we will be co-creating an augmented entertainment eternity. Together. Will you be a part of it?

Dec 312011

Originally published Oct 2011 in Wired Magazine ‘Change Accelerators‘ by Gary Hayes

Image by Gary hayes

Grab some nibbles, pour yourself a drink, and sit down. You’re now ready to immerse yourself in a TV show. And then you notice that CSI Miami is placing Facebook photos of your aunt, uncle, and cousins onto the desk of a perp. Lean forward, and keep your eyes peeled, updates from your Facebook page are about to be incorporated into your favorite TV show’s narrative. (Your best friend becomes the suspect!) As you immerse yourself in the story, the story immerses itself in your social world. In this context, online meets offline and your family and friends will never look the same again!

While this opening scenario sounds like pure fantasy, it’s not. It’s actually based on Warner Brother’s Aim High, an upcoming web series that will integrate pictures, music, and information from a viewer’s Facebook page into the video. One might call it the ultimate transmedia vanity blockbuster, where viewers are watching and playing with their own distributed, but connected story fragments. If this trend continues, soon we’ll be interacting with TV and games mashed up with our own social networks on big and small screens everywhere.

Since 2003, tools that allow people to easily create, upload, and share personal content are now commonplace. With so many people sharing their lives through networks, there is a social story revolution unfolding. There are more photos taken every two minutes today than during the entire 1800s and, as my Social Media Counter shows, most of this new content is created by individuals who used to be called the audience.

Since the late ’90s, the vision of interactive TV has been to meld this viewer-generated content into shows, particularly live TV. Today, we are taking the greatest evolutionary steps in broadcasting since the advent of live TV. In my presentation, “The Gamification of Social TV,” I examine the ways audiences are becoming more and more integrated into media, such as shows, films, games, and live events; first, there is the social level, then the participative, and finally, the inclusive.

Real-time conversation about what’s happening on TV has pretty much been with us since the beginning of mass TV in the ’50s. However, nowadays we have advanced well beyond the cord-tethered telethons of yesteryear. Today, advanced technology, like text voting, allows shows to measure the sentiment of the crowd sitting at home, not just the studio audience, in real time.

Andy Warhol famously predicted, “In the future, everyone will be world-famous for 15 minutes.” That future has arrived: We can all be stars in our own lifetimes, even if it’s just among our social network. We have become our own entertainment hubs, around which our friends and media circulate. Broadcasters and service providers have caught on to this trend, and now, entertainment-based social networking websites, like GetGlue, allow viewers to check in to movies, TV, and music. These personalized hubs fueled by recommendation and loyalty are allowing viewers to lock their worlds to TV space.

Not surprisingly, marketers are also taking notice of the advantages presented by social storytelling. A recent example is the Rommy Gulla Facebook video campaign run by Panasonic Australia. To demonstrate a new Blu-ray recorder’s ability to store 28 full days of HD content, the company developed a Truman Show-esque, promotional Facebook campaign encouraging input and social sharing.

Other online services, like Hulu, are also allowing users to bring media directly to their networks and take root inside Facebook itself, creating a forum for friends to share video content seamlessly. At the other end of the spectrum, there are options like Beckinfield Mass Participation TV, which takes social media to the nth degree by inviting users to film themselves as the stars and extend the web show format. This concept borrows from the realm of social alternate reality games, such as a World Without OilTruth About Marika, or Conspiracy for Good, which have been allowing users to write themselves into scripts and become the activist hero for quite some time now. Social media storytelling has deep roots in multiplayer role-playing gaming; in fact, the fastest growing game on Facebook at the moment is Sims, the $4 billion franchise game, where players inhabit and merge with social experiences in an alternate character-driven world.

Is alternate character acting the future? The film industry is not far behind in embracing social films. Earlier this year, Toshiba, Intel, and their ad agency Pereira & O’Dell took a gamble onInside, an interactive film experiment starring Emmy Rossum directed by D.J. Caruso. Now some people are speculating about whether or not social films are the next big thing in Hollywood. Will we see a social film revolution where plot dilemmas are handed over to the audience to experience and solve?

When it is done well, traditional storytelling married to social media is very powerful: It takes those people who want to go beyond a behind-the-scenes DVD extra into the story. While we watch to see if integrated social media entertainment will really take off, there are still some issues to consider, such as, is it invasive for characters from shows to enter an individual’s social networks? and Can a TV blockbuster become too personal?

That said, for now, I’m off to watch an episode of House, where I’m the patient!

Dec 302011

Originally published Oct 2011 in Wired Magazine ‘Change Accelerators‘ by Gary Hayes 2 of 5

Image by Gary Hayes

The alarm rings. Get out of bed, have a shower, dress, breakfast, grab your briefcase, and bus ticket and out the door. Now start RUNNING! Zombies are chasing you. Or is it pirates? Or the Mafia? Games and stories have grown legs and Facebook MafiaWars and Playstation console games have jumped into the physical world. Welcome to the most exciting and entertaining commute of your life.

For the first time in technological history portable devices that are able to send and receive game/video media, are now location aware. This means big changes in where we experience our stories. They will be always on. Around the next corner. We will be living inside the story world.

To get a glimpse of future services in the locative, pervasive space we simply need to look at the now and extrapolate.

As kids we all loved playground games: Hide and seek, capture the flag, cowboys and Indians captivated our imaginations and it was an important part of our social development. Today adults use technology to recreate those experiences for real using our own imaginary, place-based Holodecks. Fulfilling some of our adult needs in this space, The Go Game for example advertises itself as “the future of corporate play,” team-building, and self discovery.

Beyond mapping, check-in utility, or GPS functionality, what kinds of story game and locative services will get millions off their behinds and out into the real world? It takes a lot to commit to wandering around, scavenger hunting for morsels of story. How in the future may locative gamesevolve into being mass media? Or are they really just for niche groups?

This is and has moved way out of niche already. Early locative entertainment, such as orienteering and geocaching (user-created capsules hunt) are still played regularly by five million globally. More story-driven, first-generation Alternate Reality Games with locative elements such as Gentrificationor I Love Bees, attracted several million players. LBS, or locative-based services, have radicallyaltered the social gaming landscape already in Asia for hundreds of millions. A recent Pew mobile report noted that more than half of all smartphones are used for social and utility locative services and by 2015 it will be a $21 billion industry.

What makes and will make these so compelling, to so many?


The stories and games contextually fit their surroundings. The origins of this genre have often been based on architecture, audio tours, and urban map layouts, using early wearable computers or earphones. For example, back in 2004 PacManhattan re-created the 1980s game around Central Park’s grid-like streets and had hundreds of players running around collecting virtual dots. The creators wanted to:

“explore what happens when games are removed from their ‘little world’ of tabletops, televisions and computers and placed in the larger ‘real world’ of street corners, and cities.”


This element can put you and your friends at the center of the action. For example, “Spy Am I” a massive multiplayer locative game, lets players be the secret agent,  or live the life of a criminal. There are many story based pervasive game apps being launched on GPS enabled smartphones that let you explore your darker side such as Crime is Life or Crime Plays.

For those who want to make their real world a little bit more terrifying than it already is we have social locative games such as Zombie Apocalypse, where you have to really run, to outrun the attacking man eaters.

Speaking of running, marketers are not far behind in this race with viral examples such as Nike Grid or Mini’s Getaway (“this game has united us, it is so social”) that combined the best formula of story, game, and exercise with lots of social locative strategy.

Our locative experience tools are powerful social organizers. Everything in our future lives will be geo-tagged.


Set in the physical world means the same tools available to storytellers and marketers are available to everybody, including activists. So the recent Occupy Wall Street meet-ups themselves begin to look like locative fiction. Hundreds of thousands of geo-tagged narratives create probably the world’s most connected locative story yet. In a similar story-world, but oddly similar in focus, Tourality and Shadow Cities get you and your friends to battle, find treasure, and lay claim to areas of the city.

Flash mobs are a form of locative storytelling, a sort of organized social Holodeck. The social unification that comes from the best of these choreographed events, beginning with Frozen Grand Central, spread around the world. They become memes copied and extended through global imitation.

The story future of this specialized entertainment format is likely to come from expert traditional cinematic and marketing storytellers. They will be experimenting and extending fictional narratives into geo space. Pandemic 1.0 by Lance Weiler & Seize the Media is an infected world extending plot lines and challenges from cinematic games three years earlier. A present day example of future locative story, this played at the Sundance Film Festival and involved the best aspects of team play, discovery, and clever use of multi-device and place.

The popularity of locative stories and games will be proportional to marketers and writers making it worth our while to ask for our stories-to-go. As the technology matures and location lock-on is instantaneous, it will truly take off. The technology already fits in with our existing out of home routines, particularly when it comes to commenting, working, eating, and entertainment.

But back home, watching a forensic cop show on the TV. At the end, you’re told to go out of the house and help track down the killer on the street, mobile locative device in hand!

What are the security implications of this? In one evening, 200,000 viewers turn into 200,000 private eyes on the streets of London?! Would you take part in these types of services?

But think, at the end of the day, as the sun sets in a dark backstreet in your hometown, your heart starts pounding again. Will you make it back home alive?

Consider yourself, entertained.

Jul 022011

I presented at the end of the inaugural GameTech conference last week before a panel looking beyond console, revenue streams & individual game formats and looking at games breaking out into real space and becoming 24/7 – my talk was entitled

“Pervasive Entertainment – Games, Film, Physical, Print & TV merged with social networks”.

“Pervasive entertainment – entertainment that is all around you, 24 hours a day, persistent – probably location based – possibly merged with real world – driven by devices that are mobile, always on & location aware?” G Hayes

It was great to see industry heads gathered at the beginning of the conference such as this State of Industry panel twitpic I took featuring the Australasian heads of Ubisoft, EAGames, Sony and Microsoft.

Panel photolp, Ubisoft, EAGames, Sony, Microsoft heads at Gam... on Twitpic

As well as a government endorsement introduction from Brendan O’Connor, the Australian Minister for Home Affairs & Digital Culture who talked briefly about games as portable, ubiquitous & networked – yay!  He also talked about the R rating for Australia on the way which is a big relief for games distributors!

But my talk later was a broad brushstrokes whirlwind tour at the exiting period we are entering where the promise of ‘technology based’ pervasive entertainment for the last decade or two is getting very close. Another perfect storm as locative play intersperses with augmented reality, where socially produced media becomes embedded into real time broadcast networks and where game is truly dispersed across multiple platforms.

Here is the basic structure of the prez:

  1. What is Pervasive Entertainment / Gaming
  2. What is Multi Platform / Transmedia in a Gaming Context
  3. Games spilling into the real world Evolution of Experiential AR
  4. Business Models of Pervasive AR Entertainment
  5. Futures and Takeaways

The presentation is embedded below but before I launched into the definitions & case studies I asked the game industry audience –

“Who is the games industry? As all aspects of our lives become ‘gamified’ such as shopping, travel, social life, locations & TV/Film, has the games industry lost the initiative by allowing marketeers, AR & transmedia companies, ad agencies, film & TV producers to create & monetize these new pervasive forms of entertainment?” Gary Hayes – GameTech 2011 Sydney

It was too late in the conference for this to be tackled or even mean anything to those locked into AAA console title production line or part of an incumbent traditional media machine. Earlier in the conference there was a sense that if the game is not commoditized (delivered in a nice box on the shelf of the local games store) then it is outside the industry boundaries and therefore let those companies involved in more distributed, transmedia games fight over the scraps. Full slide show follows

Continue reading »

3 pages