Joshua Monolith ©Gary Hayes 2005Anyone who has used the web since 2004 is aware that we are quickly moving out of a basic text/graphics global broadband network into one dominated by video, audio and deep interaction…

“Just as humans eventually were unable to tame the complexity and scale of the physical universe so it will be with our media universe. The only course of action will be to send personalised intelligent agents, reconnaissance drones, deep into the content cosmos to capture relevance. The personalized future will be a world where rich audio visual and game media orbits around the digital you – occasionally being sucked into your ‘realm’ like stars pulled by forces unknown into black holes”
Gary Hayes, 2004

OK, enough ‘space cadet’ metaphors. Back to reality…

I have been emailed by several folk (ok – two) about one of the most important personalize media futures – how do we annotate video, audio or gaming type content with ‘rich and deep’ metadata so ‘agents’ can do their wonderful stuff. We are not talking about the first level (described in my ‘nuts and bolts of personal tv‘ page) searching for titles of film, music or tv, but more about time-stamped-vision-waveform-analysed almost quantum, nano level metadata. Lots of detailed text stuff, temporally sitting alongside the media which can be be crawled, crunched, searched on and personalized. Our friends at Google are getting the ball rolling with basic search, but they also allow video that comes complete with annotations or caption to provide the next level.

“Google Video lets you search a growing archive of televised content — everything from sports events to dinosaur documentaries to news programs. In addition to televised content, we’ll also host video from anyone who wants to upload content to us.”

Once video and audio is richly tagged the available range of interactive video based applications are immense. I will save that long list for a post in the very near future but services based on non-linear narrative, micro-content, personalized cross-media and personal av/game-type journeys become possible. Let’s look at the enabling tools rather the current distribution mechanisms (p2p, bit torrent, iptv and so on) or the p2p clients that use simplistic high level labelling – y’know the old, download this “title”, get me this “keyword”.

“In a near term broadband future where video is truly on demand the ability to hyperlink around video and audio, across platforms, becomes pervasive especially when it is personalized. I also think one of the most competitive software battlegrounds in the near future will be between companies building the tools to enable content creators (especially you and me) to deeply tag audio and video content – to personalize media” Me again.

So what kind of tools will enable the ‘how-meaningful-rich-media-content-will-find-you’ thing? There have been quite a few infamous failures in the past but the environment, I believe is ready for a proliferation of specialized software in the personalize media arena.

Some of the tool types already exist having had a long tradition in the TV production and broadcast space. So subtitling, closed captioning, on-screen graphics, content management systems and so on from companies like Aston, Chryon, Sysmedia, Virage and many others … many of the products from these companies can and are being modified for broadband metadata futures. In the US the FCC has just ruled that by 2006 all broadcast TV will be closed captioned – which is good news for this bright new future as at least that can be used as a foundation of synch’d metadata. What is synch’d MD I hear you ask – well as you watch a film or tv show engines look at the text streams and match those to existing media sources (using your own profiling layer) and as the video or audio progresses, real-time personally relevant other stuff is presented. Some people and companies working in either general targeting, synch or adding metadata for cross-media link applications are:

Autonomy – One of the more active umbrella companies in this area at the moment, just announced Virage’s VideoLogger 6 – speech analysis, segmentation, indexing etc: Some of their technology is based on a company called Dremedia that I knew well in my BBC days.

IBM MPEG-7 Annotation Tool

Berkeley’s Query-Free News Search

Zetools – Enterprise Media Company. Note their prototype: The Scripps Living Project as part of the AFI labs last year has synch content using Windows Media Metapanes, one early application of good synch’d video metadata

Ricoh MovieTool

Great Video Indexing paper from Brisbane Uni and DSTC – PDF link

Videto – in German

Visible World – Targeted advertising and storymaker

and from the Google video upload site

Automatic Sync Technologies – Automated video transcriptions

Talking Type – Descriptions of what’s happening on the screen or in the audio

Anymore companies you know of in this field you know who to call 😉

Posted by Gary Hayes ©2005