I promised quite a few folk to provide a walk-through of my short 35 minute presentation at the Augmented Reality Event in California last week. The intention of the presentation was to take my AR Scenario & Business Model thinking to the next level, to go beyond marketing eye candy, clunky ‘questionable’ games and really dig down and think hard about the value proposition for users. In creating the presentation I had to look at a deeper level at the nature of experience, as in that we can start to really find true value in Augmenting our Reality. To begin though a little compilation video I threw together for this post and some future talks looking specifically at a range of locative augmented and alternate reality services (entertainment, promotion and advertorial) to set the landscape.

Music track is called Zemith from my ‘Calm After the Storm’ album in progress – subscribe free

The only way the Augmented Reality industry is going to emerge from its current commercial birthing period is for the brands, corporates & creatives to make sure that AR is delivering a unique, immersive experience and to start to consider the value of experiential (a marketing definition here). This nature of experience, which I believe is inextricably linked to the future of AR, and the value users place on immersive services also leads at the end into a ‘experiential’ panel I am leading at Creative Sydney at the Opera House this week and I cover some of my thoughts in that space first.

The Building Blocks of Experiential Media ©© GaryPHayes

I couldn’t find any real classification of experiential, or ‘levels’ of engagement, work across the web to help frame my thinking. I am sure there are many psychological & educational studies around but nothing really from an entertainment, marketing or life utility perspective? The following five areas I therefore put forward as a layered, building block model for fully immersive, ‘around us’ experiences. So we start with physical and ‘add’ or build the others on top to create a solid experiential service where each level also extends the potential time in that experience and the likely ‘value’ (money etc:) for the user.

Level zero, btw, is any services that are viewed through a window (TV, PC screen, tablet) often at a fixed location. These are in this classification separate from our physical world as your imagination provides the ‘surround immersion’ – even though we use ‘window’ interfaces to have some of the more surround experiences.

So moving beyond that…

The Building Blocks of Experiential Media

The diagram exhibits a range of complex concepts. The Y axis is the likely time spent in each experience from short theme park rides through to long gamified life applications and will often indicate likely volume of revenues, over time. The X axis explores the nature of engagement from shallow ‘drinks of coffee’ through to deep ‘personal development’. Yes I know there is some subjectivity creeping in here as you could say a ‘certain’ theme park ride is extremely intense whereas ‘dating’ in the emotional sector may not be – but here I use engagement as something that encompasses body, mind and spirit and over longer periods. I also added to the diagram steering examples. The darker bullets are media forms and the lighter ones are life activities…the diagram will be refined over the next days. Here is a description of the levels.


  1. Physical Experiential – The entry point. We are stimulated by an altered ‘real’ environment around us (think: surround 3D or 4D cinema) or our bodies are impacted by external forces (think: 5G theme park rides) or we find something very pleasurable (think: warm drink in a nice cafe or insert your own!). Short bursts of activity over minutes
  2. Mental Experiential – Our brains become inter-active in the ‘surround’ experience. We have problems to solve that requires us to make decisions, we have to assess characters who may be trying to deceive, complex puzzles abound, historical and forensic examination, moving around & touching the environment around us and so on. May engage for hours.
  3. Social Experiential – Where experiences start to include your personal tribes, family, friends and you are engaged in competition, team or peer activities. From the early days of group treasure hunts through to present day Augmented Reality team scavenger hunts you are much more driven in your personal involvement than if it was just for you and is a much more profound immersion than any individual pursuit. The shared physical, mental and now, social experience becomes something much more powerful and may last days at a time.
  4. Emotional Experiential – Where we personally connect at a much deeper level either through bespoke elements or story. Well written stories and characters draw us in further, we empathise or even take parts ourselves in a dramatic narrative. We are also given true agency at this level and our personality and creativity is reflected. The experience is extended further to potentially months if the story ‘experience’ world is well constructed.
  5. Spiritual Experiential – A sublime and transcendental final level. This is where the experience becomes almost religious in nature and the users start to change their own belief systems. They start to believe in the artificially created world around them and evolve – change their lives & values. From a pure experiential perspective, short term examples of this include deeply involved as a fan at a large sporting arena or physically at a breathtaking global music event. There are not many ‘created’ experiential media events that have ‘altered’ lives, yet, but there will be soon. This element can last years if the experience is sustained by the users own belief system.

So at a very practical level to create compelling experiential services we need to first design the physicality, where it is, how much movement is required by the user, how encompassing the environment (enhanced real or immersive surround) and then add on the next four levels – interaction as mental tasks, social components followed by emotional connection and finishing if possible with something much more spiritual. I can name a handful of pervasive games and stories that have come close to achieving the bottom end of emotional and spiritual and most that fall below the social levels. Another post on the design process of that later.

BTW – Laurel Papworth has a great post and podcast on the Experiential Economy from a Social Media Business Perspective and worth a look with a vodcast attached to that post.

Truly immersive experiences that combine digital content from the cloud with the real world around us will only work when the connection, the glue, is seamless and has no barriers or enforced windowing. The interface to these experiences has to be transparent to be of real value – this is the segue into…

The Value of Experiential – New Augmented Reality Business Models Gary Hayes AR Event 2011 Santa Clara

Back to more practical things now and my presentation at the AR Event. After a short intro, I began by briefly looking at my infamous 16 basic business ‘scenarios’ around Augmented Reality from an article I did a couple of years ago.

16 AR Value Propositions & Usage Scenarios – 2009

  1. IN SITU: Aiding sale by seeing projects & products placed in the environment before completion.
  2. UTILITY: Selling life enhancing AR applications perceived as useful.
  3. TRAINING: Hands-on with complex equipment and work scenarios.
  4. SOCIAL GAMING: Both connotations of the word, pay-per-play mixed reality games in physical space.
  5. LOCATION LAYERS: Blended guides to new places, tourism, enhanced travelling or themed space.
  6. VIRTUAL DEMO: Display to promote sale, of product in pre-release or remotely via catalogue etc:
  7. EXPERIENTIAL EDUCATION: Pay-per-visit educational services to museums, ancient sites etc:
  8. ENHANCED CLASSIFIEDS: An AR directory that promotes local 3rd parties product & services overlaid at the location.
  9. 3D VIRALS: Branded company or personal promotion & ads using ʻcoolʼ 3D toys.
  10. PERSONALIZED SHOPPING: Walking around stores made relevant, opt in personalization and targeting.
  11. COOPERATION: Service industry for augmented virtual meetings.
  12. BLENDED BRANDING: The equivalent of hoardings, virtual poster ads.
  13. AUGMENTED EVENTS: Pay-per-use of enhanced sport or pop concerts.
  14. INTERTAINMENT: New form experiential TV and films. Blu-Ray seem so 18th century.
  15. UNDERSTANDING SYSTEMS: Creating AR for internal or exploded views of complex objects.
  16. RECOGNITION & TARGETING: Pushing ʻrelevanceʼ to outdoor consumers – facial recognition linked to online data.

Many of the above are in the b2b or marketing areas and probably only 4, 5, 7, 12 and 14 are truly user centric – meaning it has ‘value’ for them. Since this list there has been lots of papers, trend predictions and talk about AR becoming a multi billion dollar industry. At least $3 billion within 4 years is oft quoted. But I questioned have we really defined what industry we are talking about, what is ‘it’ exactly? AR as a technical definition certainly has valid industrial applications in health & manufacturing, AR as ‘eye candy’ marketing is already reaching 100 000s of jaw dropped users, and we have some early ‘creaky’ games (and I include 3DS in that!) and AR stories and films being trialed. But as a easily identified ‘cohesive’ space we are still someway off and perhaps we are actually talking about several industries here?

Can we define future Augmented Reality markets if we don’t know what it is?

If you ask most users and non-users what AR is you are going to get some pretty fluid answers –  ʻthat webcam, tilt pattern thingyʼ? ʻdots & text panes floating in the skyʼ? ‘silly glasses’, ‘weird type of kids game console’? What’s the value for them in these kind of answers ? We, as a fledgling, ill defined, industry need to better educate them and agree between ourselves about the ʻvast potentialʼ and scope of AR and be aware of the perception gap.

The perception gap is the mismatch between value proposition and value perception

Many in the know see AR as just three simple technical propositions on which they base their service and business modelling. (Note: here the content is ‘anything’ locally stored or from the cloud)

  1. Intelligent surfaces – projection mapping, tables, walls, kiosks provide tactile, localised content
  2. Locative – device is place (gps) aware and overlays content over reality – Location Based Services
  3. Recognition – marker and image aware devices overlay content over objects

I present to you a more, value orientated definition of Augmented Reality

AR is a new INTERFACE that better enables

Insight & CONNECTION into Products, Places & People

then a ʻsense-of-place EXPERIENCE & STORYʼ

by MERGING engaging digital content with our real world

OK not the most precise definition but it is of more scenario development use than those technical definitions. Also my definition got me thinking about the likely era’s of AR coming up and I presented them thus.

There are 3 eraʼs

  1. INTERFACE – mostly technically driven, start-up, eye candy 1980-2013
  2. CONNECTION – truly useable & valuable melding of digital with real, the internet of things 2013-2023
  3. EXPERIENTIAL – where we can loose ourselves emotionally and consciously inside merged real & virtual, story worlds 2023 –

I then went onto to discuss how we can gauge the industry in terms of stakeholders (on slide 9 of the embed below). The Interface era draws in early entrants of software developers, advertisers/promoters and service providers. The Connection era adds to that portal owners, network operators, equipment manufacturers and consumers. Finally when we get fluid, mature and into the Experiential era the true creatives come into play and the content creators, owners and distributors come into play. By the way in this case, ERA = Every Reality Augmented 🙂

I continue after the slide show embed here (some slides may differ from this article as some of the presentation elements were draft) –

So the rest of the talk I intended to approach two key questions

  1. How to make AR valuable to users, glue & connection
  2. The Value of Experiential

I talked a little about my personal context here and after a little bio stuff drew attention to the parallels with this early, over hype’d AR landscape with the Second Life brand gold rush between 2006-8. On a slide titled  “The old un-welcome brand parallels with Second Life” I asked if history is repeating itself? Will we see the equivalent of empty ‘cloned’ stores in an AR context, brands jumping on the band-wagon and souring a rich future for the rest of us? I talked about my own company MUVEDesign and how I really studied Second Life as a social space and created two sims (amongst) others for Telstra BigPond and ABC that were actually sims that drew the highest ratings using Second Lifes main dwell metric. (Independent summary of brand stats here). The reason I mentioned the parallel is that there is the same rush in blindfolded mentality happening again versus more considered user value. I talked briefly about six simple reasons I think the sims I did differed from the others Why were these two sims (ABC, Telstra) so popular even though this had branding & advertising?

  • In-built community components
  • Exhibited an immersion
  • The places felt homely/organic
  • Events – regular social and entertaining get-togethers
  • Variation as well as depth, refreshed content
  • Authenticity – didn’t feel too manufactured

Making AR Value-able The Connection Era

I then talked about the era we are entering. The Connection Era 2013-2023 means truly gluing digital with real vs todays crudity. This is all about providing In-Sight (enhanced vision) & CONNECTION into Products, Places & People . The shift to truly valuable services that engage and enrich and glue the cloud to the world around us. I mentioned that there are more ‘engaging’ AR concepts already in regular use such as simple things like the extended eye on the new A380 airbus sky cams or connected wifi cameras or the infamous (already) AR Drone. These fall into the AR industry’s space but I feel some companies are introducing these kinds of services out in front of the big wave about to follow. I asked some more questions about the users perception and then gave some simplistic examples of value – everyday uses, particularly of ‘thing and places’ search that will put AR on consumers ‘useful’ roadmap. What the heck will happen when the cloud really descends?! Will it be seen as gimmick vs the real potential The AR industry is not an industry until it provides value AR is cheapened if it is perceived just as a:

  1. A ʻfiddlyʼ link to other media (marker to video, image to iTunes store or a building (image) to a website)
  2. Tiltʼilation models on their webcam

Of course I made another reference to looking at the three eras mapped to Maslow’s Hierarchy on slide 24 and then questioned the video of Myles Peyton of Total Immersion talking about ‘value’ in this video. Are some of these examples really, truly valuable?

In response I quickly introduced a few very simple micro level physical search as a kind of next step of the more macro GPS, place search. Finding product based on criteria on magazine or book (what are they again?) racks eg: Johnny Depp Articles. Finding products in stores eg:  Bickford’s Lime Juice. Finding product based on type and criteria such as Low Fat, Low Sodium. Finding Recipe ingredients and so on.  Also simple next step simple AR intelligence that thinks for you “a 5 car pile-up, estimated 45mins to clear this up based on social conversation & police report – for now take this exit (points over road) to I15 etc etc:’ The point about the last one is it provides HUD or overlay accuracy combined with simple intelligence. Onto intertainment.

The Value of AR Experience Design

The Experiential Era 2013 until the end of time was the last part of my talk really focusing on new forms of entertainment and experiential services rooted in social, emotion and spirit. I have covered these in at least two previous posts New Playgrounds Augmented Reality Story Worlds and Future of Location Based AR Story Worlds both looking at how Augmented Reality Story is really about experience design and is more closely aligned to transmedia than the tech we currently see.

Locatinima – Locative Cinema, Films that are fragmented and story elements played at specific locations, linked by a connecting narrative over time and place.

I played with the word machinima (meaning machine [games] and cinema) and then thought we are about to see the dawn of Locat-inima (Locative Cinema). Examples include a couple I am scripting at the moment and I showed  ʻWitnessʼ from 13th Street & Universal, new form pervasive entertainment example of a typical cop show, crime film played out in the streets of Berlin. I talked of a few more I and others are working on and made the point that all higher level experiential scenarios can develop revenue through the usual means – promotional/advertorial, sponsorship, subscription, peer-to-peer, pay- per-play, expansion packs and most other key digital models

The Matrix - AR Version at Sydney Fountain

I talked about other scenarios of experiential AR – Hybrid experiences, Social Community, social/emotional, meet-ups leaving rich media in location, flash groups but also competitive (how I did, now you come here and try or do better, gamify rich media sharing in location. Spiritual Experiences, buying ʻvirtual-locative’ goods to enhance elements of personal dev. Seeing through the eyes of thought leaders, improving your future or that of others… life based on your current location Gifting AR Experiences placing objects for others, Learning by doing, accredited courses virtually.

Finally, finally some simple next basic steps. What needs to happen for us to move into the CONNECTION or EXPERIENTIAL eras?

  • There needs to be a jump in the ease of use of current AR for USERS not just the developers.
  • Away from clunky browsers, poor locative tech etc:
  • Smoother animation and video
  • Standards are required for mass use and interoperability – we canʼt expect each new service to require users to download a new client
  • The education of the consumer & content creator base is critical to be clear on what AR really is and the different types
  • The brave new world we are evangelising requires sustainability as a foundation and critical to that is finding real value that consumers will happily pay for

QUESTIONS – or rather comments in this context…


Are you Experiential?

Lastly a quick overview of my session this week at Vivid Creative Sydney, at the Opera House where I will be talking & moderating to four great speakers across the gamut of experience design and what makes it so relevant at this transition in media.Beyond digital, beyond social media, as we move our stories, products, lives and embed and experience them ‘in the round’ so to speak.


We’re living in media-saturated times, and artists, content makers, marketers know just how hard it is to cut through and create a meaningful connection. We meet people creating immersive experiences in cinema, gaming, advertising and visual arts, and discuss how to stage encounters that matter. Participants include:

    • Warren Armstrong – new media artist and curator of the (Un)seen Sculptures augmented reality exhibition



        • James Theophane – Digital Creative Director, Clemenger BBDO


          • Dr Tim Barker – iCinema Research Centre


            • Session moderated by Gary Hayes, CEO MUVEDesign & Head of Transmedia at Metro Screen



            Some issues we will be addressing – in no particular order and perhaps you may want to comment on in true blog style 🙂

              1. How is the immersive, experiential landscape changing in art, marketing, education and entertainment in an experiential context – overview of their and others works


                1. Defining experiential, basic industry definitions to film/art etc and from the five traditonal areas physical, social, emotional, mental, spiritual


                  1. The Experiential economy driving change – how experience is premium value vs replicable content – is there money to be made?


                    1. What are some key social aspects of experiential – particularly new forms such as LARPs, Locative Games, Social and other pervasive Games


                      1. Are there parallels between life experience and life gamification (turning traditional media into playful experience with game mechanic)


                        1. A nation of passive or static users? From TV watching, PC using people turning into active participants in physical spaces


                          1. Are we returning to an old paradigm? Theatre in the round, mobile musician concerts, parlour games, treasure hunts etc


                            1. The pros and cons of living inside stories – where do Alternate Reality Games or surrounded by scripted characters become overpowering and overtly real?


                              1. Participatory nature of involvement – how the being a role player or creator locks you into something that in itself ‘is’ the experience – the journey more important than the destination


                                1. more to come…


                                Over to you!