Mid 2010 draft catch-up post – What will it mean when we all use a handful or even just one device to consume ‘all’ our media? Will we also use it to share ‘all’ our content, pushing it to large, dumb screens around us? When we talk about transmedia we often mean, telling a complex story across many platforms used by many users, objects and screens, perhaps partly in a book, on a TV show, inside Facebook on the PC, in a console game or at the cinema – but what will happen if all our personal media is consumed only on one screen? A world where TV is not about home screens, where Facebook is not about desk or laptop PCs and the most used games are not on chunky, dedicated consoles?
This is article is not a resurrection of the dreaded, old school (circa late 90s) convergence debate but something much more akin to the Trojan Horse saga. We are palpably moving into a space where a certain medium size screen, portable device, connected, personal & social is slowly permeating our world. As powerful and practical as all the other gadgets & screens we have gotten used to the 7-10″ tablet is has hit a sweet spot. Already the fastest selling device of all time, the iPad has caused a storm, the dam holding the waters back has leaks and other similar devices are starting to trickle out, but the dam is about to burst and we will be flooded in the next year as these tactile hybrids of smartphones and laptops seep into our daily lives – once again 🙂
But will we converge towards this swiss army media device? Does it fulfil all our video, game, communication, work & social needs? More specifically, just as we are starting to master the ‘Art of transmedia Storytelling’ are we now looking at a mono device future? Will the art of transmedia storytelling turn into telling our stories across services and channels on a ‘single’ device rather than across multiple devices and platforms?
Almost half a decade ago I did a post called Media Journeys Part 2 that explored a simple evolution of media technology from cinema at the start of the last century through to the portable revolution of the mid noughties. That post implied a device that would be a screen, with a quality good enough to view films on, portable, tactile, connected, communicative and powerful enough to play networked & graphically rich games on. This post completes that train of thought and asks a key question – are online tablets the end point of a 100 years of platform evolution and more significantly can we actually expect to see a decline in the number of ‘discrete’ platforms available to transmedia producers?
The Evolution Timeframe
Firstly the timeframe. As explained in my earlier post the most useful timeframe for this ‘postulation’ is the last 110 years – from the dawn of mass media communication and non text based story-telling (film). There has been a compression of the evolution in the last twenty years, so the curved template below reflects that year-wise. The reason the chart is curved is to allow my five key trends to converge visually.
Evolution of the Human Interface
One thing I didn’t cover in the post from five years ago was the evolution of interface which reflects how the technology has become sufficiently powerful enough for us to need to do less ‘unnatural fiddling’ at the ‘control’ end and use our bodies more naturally – less of a slave to qwerty or cross, square, circle, triangle (PS reference!)…a continuum (each number corresponds with the icon sequence, left to right, on the chart)
The remote or keyboard – Alongside the TV in the 1950s the button based infrared remote control was born and a decade plus later early QWERTY keyboards were used (using strange alien languages) to communicate with computers. The remote is still with us today but as we know a revolution is about to take place there.
The mouse – The PC’s popularity spread quickly when the Mac was born in the early 1980s and the computer mouse became the norm for how we interact with complex lean forward screens vs rather clunky text entry using QWERTY keyboards.
The controller – When game consoles entered the living room in the mid 80s more complex controllers were required
Voice – although still not universal, voice controlled PCs became usable for dictation and basic control in the late 90s
Touch – Touchscreens were suddenly on every device from 2005 onwards and today any portable device that is not touch feels very antiquated
Body – at the end of 2010 XBox Kinect led the way for popular use of the whole body to interact with games, of course Sony and others had launched similar interfaces many years earlier, but the 3D sensing of kinect raised the bar significantly
Mind – (future only) having played with controllers such as Emotiv we can certainly look to a time where using parts of our body will seem so old fashioned, but that is another evolution diagram
Items 4 to 7 are of course sensory, based on natural human movement & communication.
So we need a device that responds to my touch, I can wave it around so it gets a good sense of the GPS environment it is in, as well as controlling games or measuring my physicality and without a mouse or remote in sight.
col·labo·rate (kə lab′ə rāt′)
1. to work together, esp. in some literary, artistic, or scientific undertaking
Almost 13 years ago I recall evangelising to various groups of top BBC exec producers, writers & creatives that the internet is an important new medium. “Lets be clear” I would say “audiences will have unlimited choice not only to watch what they want, when they want but to create content themselves for each other. You will need to change your relationship with them and find ‘time’ for this new relationship, to make any kind of impact in their lives in the future”. Time is the operative word. Is there time to win audiences back? Do you have time to spend collaborating with them and creating meaningful vs flash-in the-pan viral, transmedia content? Is there time before you become mostly irrelevant? I am still saying the same things to today’s generation of producers.
Traditional providers – it’s very simple. You no longer control distribution or dictate what gets consumed. Apart from a few mass produced, live, appointment to view episodics, reality shows & hyped box office the rest is in competition with people-produced, personally relevant media & conversation.
There are a multitude of sites out there asking the ‘audience’ (grrr) to submit films, music, scripts, stories, bits of their personal life and anything the brand or property feel will draw them into their branded world. Many go about it in a really poor way, providing virtually no incentive, a pretty small audience (as regards the actual community that will likely see their work) and often give little or no encouragement to learn and improve on their original submission. Many even resort to seeding the ‘competition’ with faux community created videos (made by pros deliberately shaking the camera!) – to pretend to kick-start it – they have no idea how easy it is for the community to sniff that out.
An easier and more enpowering way in for the co-creative community is to give them ‘great’ assets to create great content – it gives them a big kick-start. Giving them the same stuff that the pros get generates the real big incentive – it lets them show how ‘they’ are as good, if not better, than the pros! It also shows a willingness on the part of the usually out of reach ‘creative production’ to expose some of the real behind the scenes’ness and draw communities into the brand firstly from a ‘trust’ perspective. Secondly as the ‘creative audience’ members themselves will be spending many hours with the assets and this creates very strong brand/story loyalty. Often these creative types are pretty active influencers on the web anyhow and will draw their own communities into the brand. The list of benefits goes on.
Below are relatively recent examples of professional film and music folk throwing top draw fragments of content (rushes, isolated tracks etc) over the wall for the wannabee’s to work their own magic.
I have written about this subject a few times in the past but Bronwen Clune on Twitter pointed out her brainchild initiative, a Disney / AFTRS Film “Two Fists One Heart” making available some rushes for the community to re-edit. Cutscene site has all the information on the project and a mention should go out to my colleague at AFTRS, Bill Russo who has the enviable task (yes enviable as it is a joy to see community wisdom like this) of viewing potentially hundreds (see examples below) of alternately edited scenes.
We are giving you the chance to download and edit rushes from the international film TWO FISTS ONE HEART. This is the footage that editors work with and it is free for you to use under creative commons as long as you acknowledge the source and link back to the official movie page. We€™ve tried to give you a mix of scenes so that you can even download all of them and put together a short film of your own. When we told the folks at AFTRS and Disney what we were doing they were so exited they wanted a way to recognise some of the best videos that are created. SO €¦
The best 5 scenes will be posted on Disney€™s promotional TWO FISTS ONE HEART site. This is a great opportunity for some exposure to high-profile people in the film industry. The 5 best scene cuts will be selected by Bill Russo head of Editing at the AFTRS and the creative team from TWO FISTS ONE HEART. TWO FISTS ONE HEART Director Shawn Seet, Editor Milena Romanin and Cinematogropher Hugh Miller are all graduates from AFTRS, Australia€™s premiere Film and Television school.
The WINNERS will be personally contacted by Bill Russo who will give them editing advice and help with their editing careers. All you have to do to enter is post your video on YouTube, link to the Two Fists One Heart site for the movie in the info section and tag it TFOH, then email the link of your entry to firstname.lastname@example.org
Of course this is not a new idea and I recall at least five major projects at the BBC I was involved in from 96-03 that did a call out based on downloading and then re-editing, ‘professional’ footage (and at least 20 other ‘mash-ups’ using web interfaces). A recent non-BBC, feature example (Jan 08) that springs to mind is Tracey Fragments – a sort of timecode’esque Canadian film that gave away the ‘whole’ film to re-edit and again used YouTube to show the re-fragmentation. These are still available on the site – click refragmented. Here’s one example of a re-edit that have around 10 000 views each on YouTube.
Tracey: Re-Fragmented made available all the footage from the shoot of the film for users to download and re-edit their own replated projects including music videos, new trailers or to re-redit the entire movie themselves. A contest for best use of the footage has just closed at the end of January and judging will commence soon. The re-reditng initiative also has a competititve elements with Bruce McDonald and his editors selecting the best from the pojects from Canadaa to win an Apple Final Cut Pro prize pack. The winning project will also appear in the bonus features on the DVD release of The Tracey Fragments.
As mentioned the BBC has a long track record of opening up its rushes to the public (well it would do because the ‘internal’ BBC av assets are effectively owned by the taxpayer!) as well as providing community filming skills to a vast audience with 15 year old initiative such as Video Nation. Another recent example (2006), was when the BBC Commercial Archive opened up some of the rushes from its natural history section and asked the audience to re-cut or rather creatively make a brand new trailer for Planet Earth. Here is one example entry to the BBC Planet Earth video editing competition
To highlight the creative potential of the Open Earth Archive the BBC is also running a competition to make a short ‘advert’ for Planet Earth. Novice editors can enter the competition through the Easy Edit Suite, an exclusive application available free on the Open Earth Archive site allowing users to create a short video with a sound track using some of the best bits of the archive. The competition closes on 30 April 2006 and winners will spend time in an edit suite with experts seeing how the professionals edit for BBC Television. The Open Earth Archive is made freely available for the UK public to use under the terms of the Creative Archive Licence. The Creative Archive Licence allows people within the UK to watch, download and edit material released for non-commercial purposes, using it to inspire and create their own creative endeavours.
MAKE IT A GAME TOO
At LAMP there have been at least 20 projects based on the call for creative contribution. This includes the iEmmy nominee Scorched as well as a Gruen Transfer’esque ‘Sold in 60 Seconds‘ and most of our ARG variants. One really cool ‘video fragment’ example was on our very first residential workshop lab in late 2005, Insect Men. Insect Men was a video fragment scavenger hunt. The mind of a character (of course represented by a linear film sequence) was shattered in a freak lab accident and his mind spread all over the web, outdoors and on linear channels (these things happen!). The audience had to find, collect them and put it back together in a meaningful way. This predated Eternal Sunshine of the Spotless Mind and other similar recent play’s on fragmented memory but this is less about story and more about bringing a game-play element into the re-cut too. The team were clear that there was no right or wrong way to put this unfortunate characters mind back together but rather courted community compare contrast – which is the best way forward.
BE PREPARED FOR A TIDAL WAVE
What to do when all this great stuff comes back?! Looking at exactly the same methodology of – here are some professionally created assets by your heros for you to rebuild in ‘your’ way – Imogen Heap (who I keep going on about?!) in the past month gave away a song in the form of lead and backing vocals for the co-creative fans to provide their own music. This takes it a step further as the backing to the vocal tracks truly represents the genre and style of the contributors vs the more subtle ‘persona’ that comes through a pure film edit. Imogen and team got back nearly 400 completely rebuilt unique 3 minute songs and you can listen to 219 of them here on the Twestival site. They were so overawed by them they are going to release a special EP with the best ones on. Shame that wasn’t part of the original incentive – but it probably didn’t matter in the case of a loyal fan base already!
OK there are folk out there who use ProTools, Final Cut, Logic, Cubase etc etc: There are even more who are now familiar with iMovie, MovieMaker, Garageband etc: But although these will produce much better and original results it requires a significant amount of extra effort than a browser based tool. I was a little derogatory about web based editing or mash-up tools earlier but there have been a few good examples over the past 12 years on the web we know today – yes folks they have been around for that long! Even I did some in 1995 when I put the first audio and video on the main BBC sites in the UK – simple quicktime based mixers which worked even in the days of 28k modems!
A recent one that works because it is so integrated with the TV component is ABC Australia’s Gruen Transfer. The thing that really works for me (above and beyond the AdMixer interface and usuabilty – yes they are getting better) is the fact the call out from the show gives a very specific task – this week/series for example the presenter said – go and create using a bunch of pre-built clips/audio and text, a promotional advert to sell Australia. Simple, a bit limited tool and clip wise, but really easy for an audience to quickly produce something meaningful with very little effort.
THERE’S A LOT OF IT ABOUT
As regards remixing, mashable content there are many others worth investigating if your into the subject and a quick look around will reveal the likes of
and the list goes on especially if we include fan driven or film school based competitions!
But a last point I shall leave you with is make it accessible. If only 1-2 % of the audience will get off their a$&#s and make something and send it in then make it easier for them – grow that to 10% or more. I covered the sort of splits of the co-creative audience on my post Web 2.0 and the Myth of Non-Participation. Allow them to search and embed other peoples work, allow them to just vote or rate (obvious and used a lot of course) or give them some really good material so they can create something ‘they’ will be proud of – like most of the examples above here.
OK there are lots more examples of this including community driven film sites like my fav triggerstreet and I have probably missed the ‘big’ ones (so tell me in comments!) but the last message – Throw good stuff at your collaborative, co-creative audience and they will reward you many times over!
What is Mixed Reality, why is it relevant and has TV become it’s background medium of choice for larger numbers of media consumers who around the world are spending more time in online pursuits than glued to the content breaks in-between the advertising slots of commercial TV. How will TV survive in a world where social and gaming worlds are far more compelling?
Latest video above available as a download (with better audio & creative commons) 50MB MP4 click here
I am up at the SPAA conference in the Gold Coast producing at and speaking on one of only two panels, for TV and Filmmakers, that are really focused on the ‘creative’ vs biz only possibilities of games and virtual worlds and (that oft misused term in these circles) New Media (that internet thang). The topic of our panel is “Where Virtual Worlds Meet TV, Where Films Meet Games” and features SilkCharm aka Laurel Papworth, Keren Flavell aka SL’s Starr Sonic (SLCN.tv)Â and truna a great games evangelist and IGDA leader hailing from Brisbane.
This topic is driven partly by a mass fragmentation and a paradoxical reversal of fortunes at the moment – where TV itself is the actual snacking medium and where the longer form immersion is in online communities, virtual worlds and games. But can the mediums truly cross-over?
I have been talking for a couple of years now about the fantastic potential of the live and by implication shared TV experience to be enhanced by extending the world into online games. Where are we and where might we be in a few years once the ‘broadcasters’ realise that keeping an audience involved in the ‘IP’/programme in-between airings is a good thing. Good for the story creators, the latent creative audience and of course advertisers who need eyeballs/hands/ears/minds. Some older Personalize posts on this topic.
Anthony Zuiker creatorof the CSI franchise was at one point going to join my panel and his perspective is critical in this:
“The advertising model for TV is completely broken top to bottom. I’ve lost 20% of my viewership in a year and a half… where are they? They’re on the web and other platforms.” Despite a “revolution” in television that now delivers some of the highest quality programming in the history of the medium…Zuiker believes that the “technological boom” happening concurrently is negatively impacting the popularity of the medium.
A part of the SPAA panel here on the Gold Coast, is looking at the issue of where TV and Virtual Space ‘will’ cross-over, not some wacky installation in a dark art gallery but a new emerging element of the entertainment industry where film and games are joined at the hip and TV and social virtual worlds work in glorious real time – live broadcast driving and linking to highly social environments, what Interactive TV has always tried to be. FYI here is the synopsis of my panel “Where Virtual Worlds Meet TV, Where Films Meet Games”.
“Which side of the wall are you on? Are you ready for the Mixed Reality, Entertainment Perfect Storm? TV and Film OR Games and Virtual Worlds? That wall is about to crumble. This is a wake up call to all entertainment producers and consumers to prepare for an almighty collision. Audiences are already spending up to four times as much of their entertainment time in virtual spaces than they are watching TV. Gary Hayes says “I think we are really approaching a perfect storm, a mixed reality perfect storm, because we are seeing several things happening. The first one is a long history of games based on TV and films. Another ‘wind’ is virtual worlds, particularly the exponential growth of customisable social spaces and the important ability to integrate external media into them. The third force is audience behaviour, who involved in far more simultaneous activity particularly between broadband web and TV. The fourth element to this perfect storm is actually what is happening to TV and film, such as live reality TV becoming more game like and now over 100 feature films in production based on game worlds. All of these forces together are creating a really potent mix that all producers need to be aware of”
What is Mixed Reality?
Mixed Reality is a term that needs a more focused definition, as I believe it is also misused. So I see Mixed (or Cross) Reality as
“a creative or social activity that takes place simultaneously in real and virtual worlds and where the interaction in each are resonant and dependent on each other”
So typical practical examples – a conference that has active audiences in the virtual and real world and both communicate seamlessly with each other or where a kids TV show is broadcast while children collaborate in a virtual world and appear to help the progression of the story or a film on a DVD with clues that as you watch it allow you to progress with other players in an online game. But there is also another key aspect of cross or mixed-reality, that of virtual “˜things’ appearing around us in real physical environments. This is often called “˜augmented reality’ but I believe it sits under the broader “˜mixed reality’ umbrella.
Some folk also call this form ‘blended entertainment’ – but to me that doesn’t specifically suggest virtual space and is more related to the already out-dated ‘cross-media’ RIP (hello – Social Media Entertainment) or Extended Entertainment, which is more in my Level 1 or 2 cross-media definition on Wikipedia.
About the Mixed Reality Compilation Video Above
I think the potential of this two-way flow is significant for education and business but also for entertainment and advertising/marketing. So I put together the short film above that hopefully captures a sense of where we are with this circa 2008. I wrote a rather rushed voice over for the short film, a kind of pretrospective (writing back from the future) looking at a “Dawning of Mixed Reality“. As well as my music and voice over (apologies, it was done at 1am a few days ago)Â it contains many choice clips trawled off YouTube but also quite a few that I have worked on at LAMP, the Format Factory, BBC and various other hats.
Is this reality? Is this fantasy? In 2008 the great transition truly began. Whereas a decade earlier we used to “˜log in’ to cyberspace, now we began to semi-permanently inhabit virtual space and the digital world started to leak out into everyday life. It began to infiltrate our shopping malls and became integrated with our everyday lives. Humanity started the relentless journey towards the natural future where virtuality became a greater reality,Â and where digital fantasy and organic reality were inextricably intertwined.
Through the early 2000s pervasive wearable computing started to break down the walls that used to separate our virtual existence from our physical one. This same technology also allowed us to embed ourselves seamlessly into virtual worlds which is where we could truly experiment with future forms of entertainment, art, education and business.
In these new places we became the real time, living celebrity. Many of us became avatar stars, pixel gods – exalted as those linear film and TV icons. This became the only place for many. By 2007 hardcore gamers were already spending fifty five hours per week “˜inside’ these shared worlds and learned to forgot about the real world. This became the place where their most meaningful hours were spent. Everyone began to blend the two domains, combining them in new ways, driven by an invisible urge. Thus started the global meshing, of today’s mixed reality.
But in 2009 questions were asked. As in the many films that hinted at our transhumanist future, would the physical embodied self ever become redundant? Would we truly evolve as self contained, digital entities? May we as well be slumped in a chair with a cable plugged into our heads? Regardless, as the real world became inhospitable we all started to crave for the immersion and began moving our physical selves into these spaces. Even more than that we wanted our peers our un-virtual audiences to be able to share our experiences there. It became a contagious as the most poignant moments and memories were the ones in virtual space.
But we needed to look beyond these screens and into our own souls. Find the truth written along the “˜thin white line’ that separates analog from digital, person from avatar. This was the beginning. It always was. As far as the meshing or blending of reality and virtuality, in 2009 they were the equivalent of grainy black and white movies seventy years earlier. Our experiences were simultaneously part real, part virtual but it didn’t matter which was which anymore. Both were valid and they depended on each other. Those early parallel existences of twenty years ago started to move humanity forward, released our imaginations and we coexisted with our precious and ultimately fragile, physical selves. The great crossing had begun.
I was particularly interested in several new developments that are allowing “˜minority report’esque’ eye-catching. holographic human figures into real space. You will spot these in the film, the live Telstra conference or the models trying on clothes in store windows and closed displays. I tried to list most of the items below and have copy paste some of the text that describes the videos. If there are others you think are significant or that I have missed, please comment and as I do with other compilation films I will be updating regularly, particularly on the download versions.
The various clips in the film are detailed below:
InLimbo Blended Reality – A.M. Architect sits In Limbo to converse and showcase their electronic music. Simultaneously to the broadcast, the show was recreated in the KRTU-Second Life complex, the audio streaming and photos of the studio dynamically updating
Gizmondo Augmented Reality Game – Catapult. Please not this was never released it was in development when Gizmondo went under. There is no sound on this clip.
Motion Capture & Augmented Reality. Virtual Dancers Part 1- A good sign the tracking is okay is that all this has been obtained on the first test and with no feedback for me when I held the camera. In brief, as long as it is a bit smooth, the real-time tracking is okay.
Tim Johnson’s presentation was held at the Centre for Digital Media at Great Northern Way Campus – home of the Masters of Digital Media program. The event was simultaneously broadcast in Second Life at the school’s virtual world campus located at University Project.
A re-invisioning of a 1916 Italian Futurist film using Second Life Augmented Reality Technologies, being developed at Georgia Tech. The short film is a hybrid live action and machinima captured in real time. There are no use of Green Screens. Pos-production is not used to composite the Second Life image and live image.
Aussie talkshow host Rove McManus made history yesterday at Auckland’s Vector Arena as the first person to have his hologram beamed across the Tasman. Multiple cameras beamed his three-dimensional image from a studio in Melbourne to the paper-thin Auckland screen, accompanied by sound effects from Star Trek’s transporter beam, enabling McManus to have a live discussion with Telstra-Clear chief executive Allan Freeth.
2004 Video, computer, pneumatics. A stack of books sits in a chair and, on them, a monitor abuts a small table. The surface of the table holds water, forming a small rectangular pool in front of the monitor. The monitor depicts a woman’s face, her chin resting on her folded hands.
au.youtube.com/watch?v=Jd3-eiid-Uw – Using the infrared camera in the Wii remote and a head mounted sensor bar (two IR LEDs), you can accurately track the location of your head and render view dependent images on the screen. By Johnny Chung Lee, Carnegie Mellon University. For more information and software visit johnnylee.net
Larngear Technology’s story in the research and development, designing, and marketing of Mixed Reality technology is well describe and illustrated in the TV program Clickzone, supported by SIPA. The show is hosted by Tai, Chutima, from the movie Season Change.
More to come…
There are several companies around the world developing Cross-Reality forms. One that I am heavily involved with, The Format Factory, are pioneering formats that bridge the space between compelling participatory TV and online game worlds. This is a kind of teaser video I created that metaphorically demonstrates some of the ’embedded’ world-within-worlds. Their sister company The Project Factory have been doing some basic but interesting cross-overs in Second Life too.
I also talk in more detail about Mixed Reality potential in this early 2006 seminar, here are the slides from that one containing many cross-branded vs cross-reality examples.
I will leave the final word again to Mr ‘CSI’ Zuiker who instinctively points out that particpant audiences expect to be involved in the story, to be the hero, the villain and to feel they are partly in control of their own personalised route through it. Customisable and game worlds with existing culture are the best tool at film and TV makers disposal.
“In the gaming area, you want to give people tasks, to shoot things and upload pictures… You’re doing this because you want these people to be creating their own story and it will be part of the crime on the broadcast… Even if it’s not the actual thing I shot, I was part of that experience, that community, that narrative.”
PS: BTW I think the term ‘participatory audience’ used in the title of this post/article, is a bit of an oxymoron. Audience as a term suggests passivity not participation…but I am sure you will let me off for that one?! Please.