I am busy consulting, commercial projects and presentations (eg: SBTUG this eve) on Augmented Reality and the near future benefits for storytellers, a range of businesses and service industries. Key questions often crop up as to the relevance of AR to their ‘bottom line’ as well as usuability issues – beyond showing a range of videos from Layar, Junaio and the like it is difficult demonstrating how ‘consumer/users’ of AR devices may be interacting with them in the near, 1 to 2 year out, future. So based on a certain large tablet released recently I created a device that gave a much ‘bigger’ experience than current AR enabled mobiles, had a cleaner interface (as in less clutter POI dots on the screen – most of the time) and fitted into a series of AR future videos ‘AUGMENTED WORLDS’. Please find below Part One looking specifically at the Business Opportunities around type 3 of my AR types – Recognition (Outline) and more on how it was made and a description of the sketch interface below…
AUGMENTED WORLDS Pt.1 – Business Opportunities: RECOGNITION.
An ‘iPad 4.0 – like prototype’ stroll around an augmented Sydney. Video, music devised & created by GARY HAYES of MUVEDesign.com
Music – BTW if you like the music please subscribe via iTunes or other fav podcast tool to download over 250 of my euphoric/progressive trance, organic real harp/guitars or cinematic soundtracks via garyphayes.com or direct feed here.
…and a little end of 2008 Virtual Worlds, State of Play…
Just back from a short break in the lovely town of Broome in NW Australia (my pics). It was interesting being disconnected from ‘the cloud’ but in the process having a few ‘virtual experiential’ moments. One of these was watching the controversial film ‘Australia’ in the worlds oldest picture gardens, Sun Pictures (pictured below). Several parts of the film are set in an open air cinema in the 40s and it was so odd to actually be ‘in’ more or less the same scene of deck chairs, insects buzzing around – as the real sun set, the wind blew off the Northern Territories outback while the film panned around those environments, and lizards crawled around on the screen, bats flew overhead, propeller planes took off from the nearby Broome airport and in the audience several from the Broome aboriginal community. A kind of forget 3D lets get to 4D film experiences.
In other parts of Broome I talked to a few people about some of my work, y’know, the web, cross-media, film and virtual worlds (and just like those low hanging fruit journalists who are constantly predicting the end of 3D worlds) even out here in the styzx a couple of folk suggested that games & social virtual worlds especially will really suffer in this economic downturn and may not survive. Which leads to the point of this post to put things in a little perspective.
IS THERE REAL INVESTMENT?
First lets look at investor confidence in them. From Virtual Worlds Management Reports there was $1 billion US invested in 35 virtual world companies between Oct 06-07 – and since Oct 07 to the present day there has already been $918 million trusted to the success of this particular industry. This breaks down roughly as:
Q3 08 – $148.5 million invested in 12 VW companies
Q2 08 – $161 million in 16 VW companies
Q1 08 – $184 million in 23 VW companies
Q4 07 – $425 million in 15 VW companies
As a topical reference, and to put things into heritage media perspective the total spend on all film and tv drama in Australia in 07-08 was $420 million US (at current exchanges). Now the majority of these worlds invested in are youth based but many specialised ones aimed at the Gen Y hole (see kzero.co.uk charts for more info) that are focusing on key niches. These start to fill in the gaps that ‘generic’, jack-of-all-trades, social virtual worlds such as Second Life cannot truly cut the mustard as sub-builds inside the service. So we have recently had in the last week the to user launches of a dedicated real life buy with real cash Virtual eShopping just in time for XMas and what will be a real winner in my view (having just tried it finally) the social sports virtual world, Football Superstars which combines EA-like footy with there.com-like social activity and even has a bit of WoW-like quest giving challenges.
The social aspect of virtual worlds are not lost on the big consoles either with the Launch of XBox and PS3 virtual worlds that I covered in a recent post and also the Inquirer’s article Sony, Microsoft begin battle of Virtual Worlds. I was going to talk a lot about how during hard economic times people turn to escapist activities. In the past it used to be film or TV, but now there are many more choices and as we haven’t seen a global economic downturn of this scale since the 2nd world war – the escapism of choice is now immersive interactive media. This will not be lost on advertisers who also need to optimise their spend across the many variants of shared social worlds.
BUSINESS WAKES UP
Savvy businesses have now moved beyond the hype bubble of Second Life’s superficiality and realise the power of social collective collaboration. As well as education and science virtual worlds as ‘tools’ are developing into major economic government initiatives. The Athena Alliance have released a report called “Virtual Worlds and the Transformation of Business” with some optimistic summary lines.
“The rise of the collaborative enterprise that is likely to result from the successful deployment of Virtual World technologies will usher in a new era of business. It will change the way firms compete with one another for customers in both goods and services industries. It is our firm belief that if our nation accelerates the development and maturation of Virtual Worlds, it will encourage a more collaborative and enterprising form of business. This will lead to greater innovation, sustained productivity, and competitive growth in the world economy.. the companies and workers can use the tools of Virtual Worlds to transform the United States into a collaborative enterprise-driven economy.”
The use of virtual worlds for simulation is not lost on the military either. This goes way beyond using first person shooter games to train late teens for an army life using well, first person shooter game technology in war zones. Last week the largest global simulation conference ever was held “The Interservice/Industry Training, Simulation and Education Conference (I/ITSEC)” focused on the use of more social virtual worlds for training and education for military and scientific use. It was keynoted by General Wallace, the Commander, United States Army Training and Doctrine Command who talked with other big government players about the likely hundreds of billions of dollars that will be invested in virtual simulation technology. As we know most media developments have come about from love and death, porn and war. So this growth as always will resonate in the commercial entertainment industry. An example of how military and education are mixing here is The University of Florida recently announced too that it will be spending $1.25 million on building a Second China for the US Foreign Service and Military to understand the culture without the need to go there and fail-forward.
“The goal of the federally funded research project: To educate and prepare foreign service or other government professionals to arrive in the country prepared and ready to work.”
SHOW ME THE VIRTUAL MONEY
On the money side there is a great deal of research now going into how virtual world economic models and currencies will evolve from a range of closed systems to a state that may become viable alternatives to ‘real world’ currencies. The Virtual Economy Research Network just had an interesting article on the VW freemium model – free-to-play but encourages the adoption of the inworld currency rapidly, for example.
Forester and MillionsOfUs have just published a report looking at how traditional corporate business will begin to flourish in these spaces and to quote their executive four point summary:
It grants unprecedented depth of engagement with consumers. Second only to inperson
consumer meetings, virtual worlds allow marketers to get up close and personal
with individual consumers. Using these interactions to allow for feedback, creative tasks,
and just plain fun creates brand and product advocates in the user base who go far beyond
It taps into an audience that is difficult to reach via other channels. Today’s virtual
world users are seen as a minority vanguard for future usage, but they are also difficult to
reach via other channels. This is especially true of youth groups and deeply creative
communities supported by various virtual worlds.
Newer worlds offer better opportunities for cross-channel tracking and more
targeted audiences. Early virtual worlds, while technically groundbreaking and providing
the necessary foundation for future worlds, often lacked audience-tracking tools and were
open playgrounds without a specific purpose. New, recently launched worlds or those just
around the corner will offer better tools for customer tracking and tend to target gamers,
youth, conversation, or other specific tasks, rather than just being open. This allows better
brand alignment and campaign integration.
Virtual merchandizing resonates with youth – and can be very cost-effective. Virtual
items and other digital assets resonate with Gen Y consumers far more than with older
(physical-media-loving) consumers. They appreciate novel, unique items and accept brand
involvement in these items and their distribution – provided it has been thought through.
Needless to say, the creation, storage, and distribution of virtual items can be very costeffective
compared with traditional merchandise like t-shirts and caps.
There is no decline happening. So journos, nay sayers, please look at your own industries please. To reiterate the above examples are social or simulation virtual worlds and there are around 78 currently being used by 360 million people. I haven’t touched on online game worlds or offline games which starts to turn the whole affair into a $40-50 billion industry overtaking movies (including home entertainment elements too). All suggestions are that VWs and Games will be the dominant entertainment form and a widely used tool for business and education and revenues will start to match that of the $300 billion TV industry within five years time. A big issue for me is the lack or real courses in higher education in this space too. Most training is on how to use software to make fps console-type games, there needs to be a paradigm shift otherwise media education will be irrelevant as the heritage media linear form falls into the background.
Now tell me again that these wacky 3D worlds are about to disappear?
To finish I will be adding a presentation I gave at the Online Distribution and Business Collaboration Conference two weeks ago as it contains many references to the above post…hold your breath…
What is Mixed Reality, why is it relevant and has TV become it’s background medium of choice for larger numbers of media consumers who around the world are spending more time in online pursuits than glued to the content breaks in-between the advertising slots of commercial TV. How will TV survive in a world where social and gaming worlds are far more compelling?
Latest video above available as a download (with better audio & creative commons) 50MB MP4 click here
I am up at the SPAA conference in the Gold Coast producing at and speaking on one of only two panels, for TV and Filmmakers, that are really focused on the ‘creative’ vs biz only possibilities of games and virtual worlds and (that oft misused term in these circles) New Media (that internet thang). The topic of our panel is “Where Virtual Worlds Meet TV, Where Films Meet Games” and features SilkCharm aka Laurel Papworth, Keren Flavell aka SL’s Starr Sonic (SLCN.tv)Â and truna a great games evangelist and IGDA leader hailing from Brisbane.
This topic is driven partly by a mass fragmentation and a paradoxical reversal of fortunes at the moment – where TV itself is the actual snacking medium and where the longer form immersion is in online communities, virtual worlds and games. But can the mediums truly cross-over?
I have been talking for a couple of years now about the fantastic potential of the live and by implication shared TV experience to be enhanced by extending the world into online games. Where are we and where might we be in a few years once the ‘broadcasters’ realise that keeping an audience involved in the ‘IP’/programme in-between airings is a good thing. Good for the story creators, the latent creative audience and of course advertisers who need eyeballs/hands/ears/minds. Some older Personalize posts on this topic.
Anthony Zuiker creatorof the CSI franchise was at one point going to join my panel and his perspective is critical in this:
“The advertising model for TV is completely broken top to bottom. I’ve lost 20% of my viewership in a year and a half… where are they? They’re on the web and other platforms.” Despite a “revolution” in television that now delivers some of the highest quality programming in the history of the medium…Zuiker believes that the “technological boom” happening concurrently is negatively impacting the popularity of the medium.
A part of the SPAA panel here on the Gold Coast, is looking at the issue of where TV and Virtual Space ‘will’ cross-over, not some wacky installation in a dark art gallery but a new emerging element of the entertainment industry where film and games are joined at the hip and TV and social virtual worlds work in glorious real time – live broadcast driving and linking to highly social environments, what Interactive TV has always tried to be. FYI here is the synopsis of my panel “Where Virtual Worlds Meet TV, Where Films Meet Games”.
“Which side of the wall are you on? Are you ready for the Mixed Reality, Entertainment Perfect Storm? TV and Film OR Games and Virtual Worlds? That wall is about to crumble. This is a wake up call to all entertainment producers and consumers to prepare for an almighty collision. Audiences are already spending up to four times as much of their entertainment time in virtual spaces than they are watching TV. Gary Hayes says “I think we are really approaching a perfect storm, a mixed reality perfect storm, because we are seeing several things happening. The first one is a long history of games based on TV and films. Another ‘wind’ is virtual worlds, particularly the exponential growth of customisable social spaces and the important ability to integrate external media into them. The third force is audience behaviour, who involved in far more simultaneous activity particularly between broadband web and TV. The fourth element to this perfect storm is actually what is happening to TV and film, such as live reality TV becoming more game like and now over 100 feature films in production based on game worlds. All of these forces together are creating a really potent mix that all producers need to be aware of”
What is Mixed Reality?
Mixed Reality is a term that needs a more focused definition, as I believe it is also misused. So I see Mixed (or Cross) Reality as
“a creative or social activity that takes place simultaneously in real and virtual worlds and where the interaction in each are resonant and dependent on each other”
So typical practical examples – a conference that has active audiences in the virtual and real world and both communicate seamlessly with each other or where a kids TV show is broadcast while children collaborate in a virtual world and appear to help the progression of the story or a film on a DVD with clues that as you watch it allow you to progress with other players in an online game. But there is also another key aspect of cross or mixed-reality, that of virtual “˜things’ appearing around us in real physical environments. This is often called “˜augmented reality’ but I believe it sits under the broader “˜mixed reality’ umbrella.
Some folk also call this form ‘blended entertainment’ – but to me that doesn’t specifically suggest virtual space and is more related to the already out-dated ‘cross-media’ RIP (hello – Social Media Entertainment) or Extended Entertainment, which is more in my Level 1 or 2 cross-media definition on Wikipedia.
About the Mixed Reality Compilation Video Above
I think the potential of this two-way flow is significant for education and business but also for entertainment and advertising/marketing. So I put together the short film above that hopefully captures a sense of where we are with this circa 2008. I wrote a rather rushed voice over for the short film, a kind of pretrospective (writing back from the future) looking at a “Dawning of Mixed Reality“. As well as my music and voice over (apologies, it was done at 1am a few days ago)Â it contains many choice clips trawled off YouTube but also quite a few that I have worked on at LAMP, the Format Factory, BBC and various other hats.
Is this reality? Is this fantasy? In 2008 the great transition truly began. Whereas a decade earlier we used to “˜log in’ to cyberspace, now we began to semi-permanently inhabit virtual space and the digital world started to leak out into everyday life. It began to infiltrate our shopping malls and became integrated with our everyday lives. Humanity started the relentless journey towards the natural future where virtuality became a greater reality,Â and where digital fantasy and organic reality were inextricably intertwined.
Through the early 2000s pervasive wearable computing started to break down the walls that used to separate our virtual existence from our physical one. This same technology also allowed us to embed ourselves seamlessly into virtual worlds which is where we could truly experiment with future forms of entertainment, art, education and business.
In these new places we became the real time, living celebrity. Many of us became avatar stars, pixel gods – exalted as those linear film and TV icons. This became the only place for many. By 2007 hardcore gamers were already spending fifty five hours per week “˜inside’ these shared worlds and learned to forgot about the real world. This became the place where their most meaningful hours were spent. Everyone began to blend the two domains, combining them in new ways, driven by an invisible urge. Thus started the global meshing, of today’s mixed reality.
But in 2009 questions were asked. As in the many films that hinted at our transhumanist future, would the physical embodied self ever become redundant? Would we truly evolve as self contained, digital entities? May we as well be slumped in a chair with a cable plugged into our heads? Regardless, as the real world became inhospitable we all started to crave for the immersion and began moving our physical selves into these spaces. Even more than that we wanted our peers our un-virtual audiences to be able to share our experiences there. It became a contagious as the most poignant moments and memories were the ones in virtual space.
But we needed to look beyond these screens and into our own souls. Find the truth written along the “˜thin white line’ that separates analog from digital, person from avatar. This was the beginning. It always was. As far as the meshing or blending of reality and virtuality, in 2009 they were the equivalent of grainy black and white movies seventy years earlier. Our experiences were simultaneously part real, part virtual but it didn’t matter which was which anymore. Both were valid and they depended on each other. Those early parallel existences of twenty years ago started to move humanity forward, released our imaginations and we coexisted with our precious and ultimately fragile, physical selves. The great crossing had begun.
I was particularly interested in several new developments that are allowing “˜minority report’esque’ eye-catching. holographic human figures into real space. You will spot these in the film, the live Telstra conference or the models trying on clothes in store windows and closed displays. I tried to list most of the items below and have copy paste some of the text that describes the videos. If there are others you think are significant or that I have missed, please comment and as I do with other compilation films I will be updating regularly, particularly on the download versions.
The various clips in the film are detailed below:
InLimbo Blended Reality – A.M. Architect sits In Limbo to converse and showcase their electronic music. Simultaneously to the broadcast, the show was recreated in the KRTU-Second Life complex, the audio streaming and photos of the studio dynamically updating
Gizmondo Augmented Reality Game – Catapult. Please not this was never released it was in development when Gizmondo went under. There is no sound on this clip.
Motion Capture & Augmented Reality. Virtual Dancers Part 1- A good sign the tracking is okay is that all this has been obtained on the first test and with no feedback for me when I held the camera. In brief, as long as it is a bit smooth, the real-time tracking is okay.
Tim Johnson’s presentation was held at the Centre for Digital Media at Great Northern Way Campus – home of the Masters of Digital Media program. The event was simultaneously broadcast in Second Life at the school’s virtual world campus located at University Project.
A re-invisioning of a 1916 Italian Futurist film using Second Life Augmented Reality Technologies, being developed at Georgia Tech. The short film is a hybrid live action and machinima captured in real time. There are no use of Green Screens. Pos-production is not used to composite the Second Life image and live image.
Aussie talkshow host Rove McManus made history yesterday at Auckland’s Vector Arena as the first person to have his hologram beamed across the Tasman. Multiple cameras beamed his three-dimensional image from a studio in Melbourne to the paper-thin Auckland screen, accompanied by sound effects from Star Trek’s transporter beam, enabling McManus to have a live discussion with Telstra-Clear chief executive Allan Freeth.
2004 Video, computer, pneumatics. A stack of books sits in a chair and, on them, a monitor abuts a small table. The surface of the table holds water, forming a small rectangular pool in front of the monitor. The monitor depicts a woman’s face, her chin resting on her folded hands.
au.youtube.com/watch?v=Jd3-eiid-Uw – Using the infrared camera in the Wii remote and a head mounted sensor bar (two IR LEDs), you can accurately track the location of your head and render view dependent images on the screen. By Johnny Chung Lee, Carnegie Mellon University. For more information and software visit johnnylee.net
Larngear Technology’s story in the research and development, designing, and marketing of Mixed Reality technology is well describe and illustrated in the TV program Clickzone, supported by SIPA. The show is hosted by Tai, Chutima, from the movie Season Change.
More to come…
There are several companies around the world developing Cross-Reality forms. One that I am heavily involved with, The Format Factory, are pioneering formats that bridge the space between compelling participatory TV and online game worlds. This is a kind of teaser video I created that metaphorically demonstrates some of the ’embedded’ world-within-worlds. Their sister company The Project Factory have been doing some basic but interesting cross-overs in Second Life too.
I also talk in more detail about Mixed Reality potential in this early 2006 seminar, here are the slides from that one containing many cross-branded vs cross-reality examples.
I will leave the final word again to Mr ‘CSI’ Zuiker who instinctively points out that particpant audiences expect to be involved in the story, to be the hero, the villain and to feel they are partly in control of their own personalised route through it. Customisable and game worlds with existing culture are the best tool at film and TV makers disposal.
“In the gaming area, you want to give people tasks, to shoot things and upload pictures… You’re doing this because you want these people to be creating their own story and it will be part of the crime on the broadcast… Even if it’s not the actual thing I shot, I was part of that experience, that community, that narrative.”
PS: BTW I think the term ‘participatory audience’ used in the title of this post/article, is a bit of an oxymoron. Audience as a term suggests passivity not participation…but I am sure you will let me off for that one?! Please.
Interesting times ahead – the Console space finally collides with the Social Virtual World space as Sony and Microsoft race to be first to offer non-closed beta, ‘social (read: commercial) virtual world’ front ends to their ‘trojan horse’ consoles. Will they start to reap the benefits of a very large installed user base as both are likely to launch this side of Christmas in several international territories, and will they fly?
Both the Social World front end and the DIY games components (LittleBigPlanet and Buko) of these consoles have tremendous impact potential due to the massive installed base. As at the end of 2008 we are looking at PS3, XBox360 and Wii have a potential ‘Console Social Virtual World’ user base of 100 million! Compare that to the 1.5 million Second Lifers or even the 12 million WoW addicts…
The report from International Tribune about the two new ‘social worlds-in-your-console’ rivals, XBox New Experience (launching Nov 19) and Sony’s PS3 Home, suggests that Sony has cried wolf too many times. Sony have over-delayed the launch and are probably are trying to start out too big (vs the lower rez, cartoony avatars we see in the Wii [Miis]) and now the XBox equivalent, image above.
…Hirokazu Hamamura, a game expert and head of Japanese publisher Enterbrain Inc., who was at the Sony booth, said he needs to see more to assess “Home.” “You still can’t tell what it’s all about,” he told The Associated Press, adding that “Home” may be coming a little late compared to rivals. “There are so many more possibilities for a virtualÂ community.”
The NewXBoxExperience (NXE as it shall be now known) on the other hand has much more accessible friendly ‘toons’ which are very simple characters representing you in the basic XBoxLive interface. As I mentioned the interaction is likely to be similar to that on the Wii and Animal Crossing Wii also about to launch looks interesting too in this regards. But NXE will likely put off some of the hardcore gamers who don’t want to be represented by Simpson’s like avatars with minimal options to customise/personalize and make them their own? But is this just a half baked attempt at encouraging more group/tribal ‘mall’ type interaction to get folk to watch more of those ‘netflix’ (one of XBox’s live partners) videos or peer pressure to play/purchase online games they wouldn’t normally play? One think I do like the idea of is layering groups of avatars over full screen movies, so they can ‘play/chat/critique’ but I suspect the studios will put pressure on Microsoft to not allow that. We shall see. Other key partners in the NXE include Netflix, USA Network, MGM, NBC Universal, Universal Studios Home Entertainment and the SCI FI Channel.
The other big question is how to ‘really’ commercialise these spaces vs just incrementally increasing sales of existing product, like videos, within the portal. It is one thing getting your massive online user base to create an avatar and hang out with their friends in an abstract ‘exhibition’ hall while clicking (with a TV remote) on buy-me items like videos and other online games as in NXE but another to draw them in to having their own ‘pad’ as the case in PS3Home.
Having a persistent place to call your own produces, like Second Life, a big increase in user hours (for those who stick with it!) approaching 50 hours per week in the social space. This also brings with it the desire to purchase ‘virtual life’ enhancements (show-off pixel products) and the whole thing turns into aspirational lifestyle marketing on the Home side vs a 3D ‘TV-catalogue’ world on the NXE side. The next question is advertising during your social console moments. Both Home and NXE worlds will have a spattering of environmental or portal advertising from the outset and it will be great to see some contextual ads in there vs generic billboard equivalents. I would hope that Sony or Microsoft don’t go it alone here (even with Massive’s involvement with MS) and that they do adopt the expertise of the worlds largest advertising corporation Google.
So with Google moving into the fray with adsense now being delivered into online games reported by Reuters if brands don’t reach the gamers in the social front end worlds then they have another chance in the games themselves. Google are keen to point out the benefits they offer marketeers here A selection:
Drive your brand: In-game ads have been shown to drive brand familiarity and consideration by significant percentages*
…and have the option for custom sponsorships and integrations: In addition to the media buy you make directly with Google, your Google sales representative can connect you directly with publishing partners for deeper integrations.
Reach the new generation of social gamer: The face of online gaming is changing to include users of all ages, backgrounds, and interests. Get your brand in front of users on the largest social networks, including MySpace, Facebook, and sites across the web.
OK they are starting with simple flash games and SocioNet widgets but they have their sights on traditional online games as it says in the Reuters report they are working with Konami and Sony and a few other key partners are already listed on their Adsense in Games site. (It is interesting to note also that Google are kicking advertisers into action with other initiatives including text ads in Google Maps/Earth and YouTube click-to-buy buttons – both reported by TechCrunch).
But back to the ingame advertising which if done right and using dynamic behavioural and personalised targeted techniques will indeed be a significant step forward for marketeers who are just getting their heads around basic social media. One big hurdle to come though is the old walled garden product/service vs open field product/service – you have lots of great ingame ads pointing to limited content in say NXE or Home Walled Gardens vs the potential of wandering around Halo 3 or Far Cry 2 and being able to purchase the ‘book, music, film’ inserted with 1-click to buy from Amazon, while STILL ingame or at least a quick hop out to the ‘social virtual world portal’.
This is all about clever product placement and relating it to the game your in (see my recent post on the renaissance of hundreds of films being made of games) is both the opportunity and the real challenge (being ‘sensitive’ to the story world and narrative of the game). So for example making sure the latest 2008 car is not being advertised in the 2020 story world of Crysis or subscriptions to Star Trek eps on demand embedded in Star Wars Galaxies. At least the social, vanilla spaces will allow contemporary advertising without too much jarring such as in a simple ‘gathering environment’ like Home below…
In summary I still think NXE is a half way house, a little too old school, cable/IPTV for my liking, and that Sony have the right model in the medium/longer term by persisting with a much more sticky, immersive and larger scale social ‘customisable’ environment – which as we have seen for the past 4 years in Second Life will drive much more inworld commerce. I hope that PS3Home allows some ‘theme’d’ areas too – based on loyal fans of certain games – to the extent that the social hangout becomes almost like a TV/film green room, a place to relax outside game world but feel your with like-minds…the 3D forum becomes a reality.