“The Sixth Sense” – Wearable Tech

This demo — from Pattie Maes’ lab at MIT, spearheaded by Pranav Mistry — was the buzz of TED. It’s a wearable device with a projector that paves the way for profound interaction with our environment. Imagine “Minority Report” and then some!

Think Microsoft Surface without the table! This is amazing. The basic idea is that this device can augment our reality by providing information about objects we interact with in the real world. This immediate access to information at all times reminds me of Cory Doctorow’s book “Down and out in the Magic Kingdom”.

Minority Report like gesture control for PC’s

Remember Tom Cruise in Minority Report? Remember when he magically moved photos, videos, etc. from one place to another with just a simple flick of his wrist? Well, that power can be yours too. This is not science fiction – this is Mgestyk gesture-based control from Mgestyk Technologies.

Wow! This looks awesome.

Visual Interfaces to the Social and the Semantic Web (VISSW2009)

I’ve recently been invited (and accepted) to join the Program Committee for the Visual Interfaces to the Social and the Semantic Web workshop which will be held as a part of the 2009 International Conference on Intelligent User Interfaces which takes place in February in Florida. Here’s a brief description of the workshop:

This workshop aims to bring together researchers and practitioners from different fields, such as Human-Computer Interaction, Information Visualization, Semantic Web, and Personal Information Management, to discuss latest research results and challenges in designing, implementing, and evaluating intelligent interfaces supporting access, navigation and publishing of different types of contents on the Social and Semantic Web. In addition, the workshop also serves as an opportunity for researchers to gain feedback on their work as well as to identify potential collaborations with their peers

I’m quite excited about this workshop, I’ve been spending some of 10% time at work on playing with ideas around how to visualise data on the Semantic Web just over eighteen months ago I was already using and extending Moritz Stefaner‘s excellent Relation Browser to visualise semantically structured data, and was effectively walking around entire graphs of data:

However there were problems with this approach, performance over large sets of data was one, and I spent a considerable amount of time extending the tool. However it occurred to me that with Relation Browser and other similar tools you are forced to take a very node centric view, when sometimes whats more useful is to be able to take a graph or named graph centric view – its not only important to be able to center on one node and see it’s direct relationships it’s often just as important to understand where that node sits with a much wider context. To that end I’ve been experimenting with visualisations that allow you to not only center on a particular node and ‘follow your nose’ but to also pan out take a more hollistic view of that graph of data – I’ve also been thinking about ways to visualise the provenance of data.

I wish I had known about this workshop sooner, it might have been the kick I needed to actually finish demonstrators of these ideas, I think I might still have time though …

With reference to the workshop itself though here’s some important dates and other bits of information if you’re interested in submitting a paper:

  • Paper submission deadline: 14th November, 2008
  • Notication of acceptance: 7th December, 2008
  • Camera-ready paper submission deadline: 14th December, 2008
  • Full papers which should be between 6 and 10 pages.
  • Short papers and position papers which should be up to 5 pages.
  • Demo papers – 2 page description with a screenshot of the working prototype or preferrably a link to an online demo.
  • Submissions must be in PDF format and prepared according to the IUI format.

It promises to be an excellent conference, and one that I will definitely be attending.

OLPC the new XO-2 model

This video provides a quick look at the second generation OLPC, XO-2. The new system has two touch-sensitive displays. The XO-2 will be much smaller than the original machine , almost half the size! and will have a foldable e-book form factor. “The next generation laptop should be a book” according Nicholas Negropante.

The XO-2 will also have dual indoor-and-sunlight displays, which was pioneered by former OLPC CTO Mary Lou Jepsen. This new design will provide:

  • a right and left page in vertical format
  • a hinged laptop in horizontal format,
  • and a flat, two-screen continuous surface for use in tablet mode.

Younger children will be able to use simple keyboards to get going, and older children will be able to switch between keyboards customized for applications as well as for multiple languages

The have also stated that the new model will reduce power consumption down to 1 watt!

Find out more here.

Touché – An open source multi-touch framework

Touché is an open-source multi-touch tracking environment for Leopard. It has been designed and written specifically for MacOS X Leopard and uses many of its core technologies, such as QuickTime, Core Animation, Core Image and the Accelerate framework.

The Touché environment consists of two parts: A standalone tracking application written in Cocoa, that comes with lots of configuration options as well as calibration and test tools, and a Cocoa framework that can be embedded into custom applications in order to receive tracking data from the tracking application. This way, you can easily experiment with MacOS X technologies such as Core Animation or Quartz Composer on your FTIR multitouch table.


Touché Multitouch Framework – Simple Demo Apps from Georg Kaindl on Vimeo.


Touché Multitouch Framework – Introduction from Georg Kaindl on Vimeo.

It’s pretty impressive stuff, but is dependent on a FTIR screen which it looks like you have to build yourself unless you can find someone to do it for you. However this might be an excellent 10% project at work for me, some of the modes of interaction I’ve been experimenting with for navigating large graphs of data would be more intuitive with a multi-touch interface … wow … *me has a cunning plan*.

HCI2007 … Day One Summary

 The last couple of weeks have been extremely busy and consequently I haven’t had much of a chance to post up about the conference. I thought I’d write a summary of some of the presentations that stood out the most for me.

For anyone interested you can view all the papers that were presented during the conference which are available from the BCS website on this page.

Day One

View the schedule for the day.

The day was divided into three sessions. During each session there were three concurrent tracks. This was essentially the format for all three days of the conference. I decided to stick with the Creative and Aesthetic experiences track for most of the day.  During the first session the presentation that captured my imagination the most was Dharani Perera’s short paper on Investigating paralinguistic voice as a mode of interaction to creating visual art.1.   In the paper Dharani and her colleagues reported on their research into how it is possible for people to use the volume of their voice to control cursor movement to create drawings. The research is especially hopeful for artists with upper limb disabilities who  show remarkable endurance, patience and determination to create art with whatever means are available to them. Listening to her presentation and watching one of the video’s they had recorded of an artist using the system was quite inspirational.

Another interesting paper presented during the first session was by Jennifer Sheridan from www.bigdoginteractive.com on Encouraging Witting Participation and Performance in Digital Live Art 2. Jennifer and her colleagues worked on developing a framework for characterising people’s behaviour with Digitial Live Arts. They identified three key categories of behaviour with respect to a performance frame – these are defined as performing, participating and spectating. They used an iPoi to illustrate their framework. Imagine spinning or throwing a tiny computer round your body to create your own visual projections, music or light show. Poi is a traditional Maori art form. iPOI (pictured above) is a sensor packed upgrade of the original that can trigger visual and audio soundscapes in real time using wireless technology. The goal of this peer-to-peer, exertion interface is to draw people into the performance frame and support transitions from audience to participant onto performer. The presentation was really about describing how evaluating and measuring interaction in public performance is very differnt to the frameworks and measures currently employed to understand interaction in tradtional HCI. I think what Jennifer and her team showed was that traditional HCI, like it or not, has focused on understanding interactions in desktop computing, however as computing moves away from desktop into more ubiquitious, mobile devices we will see a shift to non task based uses of computing, and as such we need to new ways to understand this new kind of interaction.

The first session was followed by the first of the conference keynotes by Professor Stephen Payne on the subject of Deciding how to sped your time the keynote was an excellent presentation of Stephen’s current research area of Cognitive strategies and design heuristics for dealing with information overload on the web3. He began the keynote with the simple and yet insightful observation that …

time is natures way of preventing us doing everything at once

The talk was about the very real problem that many of us face … In a world where the internet and search engines can provide individuals with relevant texts on any given subect how can readers allocate time effectively across a set of relevant documents, and how can they be helped so to do. So when faced with more relevant texts than time to read them all what strategies do you use when you have a specific learning goal.

Stephen presented the results of an experiment where a number of test subjects were presented with four texts to read in 6 minutes, after which they would have to take a test. Whilst the four texts covered the same subject (the functioning of the human heart) the complexity of the texts varied. For example Text A was a primary school text on the topic, whilst Text D was a post graduate medical text. B and C were both somewhere in the middle.

Now, before seeing Stephen’s presentation, I would have assumed that people in this situation might have attempted to sample each of the texts, in other words review each on briefly. What Stephen’s research has begun to demonstrate is that individuals, when faced with a specific learning goal, don’t sample but rather Satisfice, in other words in their minds they set a threshold of acceptability and if it’s met by a particular text will settle on it. It’s sometimes referred to as Information Foraging, which is analagous to problems faced by animals foraging for food in nature, how long do they stay in one patch patch before moving on to another.

What I found most interesting in Stephen’s presentation was the idea that even when skimming documents we build a mental representation of the mapping between the physical structure of the text and the meaningful content of the text. Now if this is true then it would suggest that it might be possible to find ways to construct documents so that they take advantage of this structure map. It was an excellent keynote, and I must admit I wasn’t at all suprised to learn that Alan has worked with Stephen4 on this area since much of Stephen’s talk seemed to remind me of ideas and theories of the mind that Alan had related to me in the past.

After lunch the second session of talks began. Again I opted to start off in the Creative and Aesthetic Experiences track. The first paper presented was by Shaowen Bardzell, and had the intriguing title – Docile Avatars: Aesthetics, Experience, and Sexual Interaction in Second Life5. It’s no secret that I’m not a huge fan of Second Life, but listening to this talk made finally accept that even though I don’t understand why people in second life seem to take it so seriously the fact remains that many do and because they do it’s important to understand their needs as users and the communities that are emerging within Second Life. I did find this talk to be somewhat surreal the whole idea of sexual interaction and in particular BDSM as something thats possible in a virtual environment felt kind of bizarre. I did laugh about it with Alan at the bar later when I recounted to him a conversation I had with someone from the second life team who couldnt understand why i wasnt willing to spend more time in it and asked me what the real world had that second life didnt and I had replied “real women” (stop laughing Alan!). What did emerge from Shaowen’s talk was that it is possible to construct powerful aesthetic experiences and that were only now beginning to try to understand through the use of virtual ethnography and HCI theories of experience design to understand how and why this complex phenomenon emerged in Second Life.

The most interesting of the talks from this session, under the Communication and Sharing Experiences track, was entitled Thanks for the Memory6. This was a join project between Manchester Met University, The BBC and Microsoft. It was based around a piece of life-logging technology developed by Microsoft called The SenseCam,which they decribe as a piece of memory prosthesis. The SenseCam is a passive device that users wear, the device is designed to take photographs, at regular intervals of around 30 seconds, without user intervention whilst it is being worn. Unlike normal camera’s it doesnt have a view finder, it’s a simple camera fitted with a special fish eye lense that ensures that the field of view is maximised so that almost everything in the wearers field of view is captured. Whilst it sounds a bit strange the effect on the six test subjects seemed to be quite profound, the authors of the paper put it in these terms …

What we have seen is that the relationship between things-as-remembered-by-thesubjects-in-ordinary-ways and things-as-presented-by-the- SenseCams is complex. For one thing, SenseCam data captured things-that-might-have-been-remembered-but-not-intentionally and things-that-were-beyond-the-possibility-of-being-recalledby-the-user-but-which, -when-presented-to-the-same-user, -somehow-provoked-a-recollection … This awkward language alludes to the difficult and complex relationship between human memory and digital traces of action. We have seen that SenseCam data makes livedexperience, in various ways and in varying degrees, strange to the persons who had the relevant experiences in question. Strangeness here is not a negative thing, as we saw. Strangeness brings values of various kinds. The crux, it seems to us, is that in creating discongruent experiences to the one’s imagined or recollected, SenseCams brought to bear ways of seeing that were not obviously the subject’s own, but which were nevertheless empirically related to those experiences, though in complex ways.

For the final session of the afternoon I decided to stay with the Communication & sharing experiences track. There was a very interesting paper presented The Devil You Know Knows Best How Online Recommendations Can Benefit From Social Networking7 which I thought was quite relevant to some of the stuff were looking at here at Talis. The content of the paper seemed to be me be fairly obvious but nevertheless it was interesting to see some empiricial research done to prove some of the points. In short there thesis was that the defining characteristic of the internet is an abundance of information and this is problematic, how do we know which information to pay attention to and which we can just ignore? Recommender Systems were envisaged to try to solve this problem but have for the most part been unsuccessful. Research would suggest that this is due to lack of social context and inter-personal trust. What the researchers presenting this paper discovered was that participants overwhelmingly favoured recommendations made by people familiar to them, and if they recommender was not someone familiar then recommendations were favoured if the recommender could be identified as having the same interests. Consequently the conclusion offered, and this all felt terribly obvious t me, was that Recommender Systems should be integrated with Social Networking Systems.

The next paper from Day One that I want to mention was the presentation by my HCI tutor and aQtive Colleague Russell Beale Blogs, reflective practice and student-centered learning8. Russell argued that Blogging can be used to enhance education by encouraging reflective practise. Russell defines reflective practise as

an approach to learning that encourages thought about what has been experienced and seen, which can then drive new theories and investigations to test those theories, leading to new experiences that may, or may not, validate the original ideas. This leads to them being modified, extended, and refined, and the cycle continues.

Whilst the notion of reflective practise should be familiar to everyone, this is the first time I’ve come across any real research into using Blogging as a medium to encourage and enhance reflective practise in education. Russell was keen to point out that from a social and pedagogical perspective blogging can support a sense of community amongst students – they can interact with each other, post comments on each others blogs. But because of the semi-public nature of the content they are generating students can see the work done by other students and as such gain an insight into how much work they themselves need to do – since others can see the level of their activity. This creates a kind of peer-pressure that exerts influence over students to at least maintain some kind of acceptable level of activity. Russell has always been a good presenter, and I enjoyed his presentation – even though I do think I have some misgivings about the kind of peer-pressure this creates – but that doesnt mean its a bad idea.

The last paper of the day was also very interesting, the researchers sought to compare traditional and novel user interfaces for exploring a blog. The paper is entitled Contextualizing the Blogosphere: A Comparison of Traditional and Novel User Interfaces for the Web9. They investigate how contextual user interfaces affected blog reading experience and also how novel contexstual user interfaces can increase user performance and statisfaction. They compared a standard Blog Interface (WordPress), with StarTree, and the Focus Metaphor Interface. Star Tree uses a dynamic navigation tree that presents all nodes in the navigation concurrently. Each node correlated to a category on the blog. When a user selects a node it displays the associated article in a content pane. The presenters argued that by providing the entire structure of the information space concurrently this could result in superior orientation in the information space, making it much easier to find what your looking for. As such Star Tree out scored both the standard blog and the FMI interface in terms of task performance.

What this demonstrated to me was that when you have a large data set that you need to navigate or traverse in order to discover key bits of information tradition interface metaphors like facetted browsing, key word search etc. don’t provide enough context especially when the information space is large or complicated. What you need is a way to intuitively navigate to the bit that your interested in. It’s an idea some of us here at Talis have been playing around with and we think we have come up with some interesting results. 😉

At the end of the day most of the delegates attended the reception where we had some nice food, tonnes of drinks and I managed to have a lot of conversations with some very interesting people. After the reception many of us ended up in the bar where I camped out with Alan and Russell and spent the evening being introduced to tonnes of people spoke to them about their research and interests as well as what we do at Talis. The evening was a lot of fun and a great way to round off a very intense first day. All in all the first day seemed pretty fast paced, there was a lot to absorb ( some of which im still absorbing … ).

I’m going to start writing up the summary of day two ….

  1. D. Perera, J.Eales, K.Blashki [Voice Art: Investigating Paralinguistic Voice as a Mode of Interaction to Create Visual Art] [back]
  2. J.Sheridan, N. Bryan-Kinns, A.Bayliss, [Encrouaging Witting Participation and Performance in Digital Live Art] [back]
  3. Stephen Payne, IGR Report[back]
  4. A. Dix, A. Howes, S. Payne [Post-web cognition: evolving knowledge strategies for global information environments] [back]
  5. S.Bardzell, J.Bardzell [Docile Avatars: Aesthetics, Experience, and Sexual Interaction in Second Life] [back]
  6. R.Harper, D.Randall [Thanks for the Memory] [back]
  7. P. Bonhard, M. A. Sasse & C. Harries, [ The Devil You Know Knows Best: How Online Recommendations can Benefit from Social Networking ] [back]
  8. R.Beale [Blogs, reflective practice and student-centered learning] [back]
  9. S.Laqua, N.Ogbechie, A.Sasse [Contextualizing the Blogosphere: A Comparison of Traditional and Novel User Interfaces for the Web] [back]

Microsoft launches Tafiti

If you haven’t tried it yet check out Microsoft’s experimental new search front end Tafiti – http://www.tafiti.com.

It’s based on Microsoft’s new SilverLight technology, their competitor to Flash. It uses a cool desktop metaphor where you can spin through different kinds of results, drag and drop them into piles which you can label and share with your friends.

Although it might seem gimmicky and not particularly useful at first glance, it is actually quite innovative and and a lot of fun to use. It’s a great showcase for SilverLight but it’s interesting to contrast how Apple are doing a lot of work in bringing metaphors such as stacks to the desktop UI, and Microsoft seem to be focussing on Search and the Web.

Try it out for yourselves! I think it’s awesome.

Turning the pages of an e-book. The book metaphor

This is a fascinating talk about whether the use of the book metaphor is appropriate for electronic documents. There’s two schools of thought on this. On the one hand we have those who believe that the book metaphor is useful because humans are already experienced in handling physical books, its ingrained into our psyche. On the other hand many others argue that the metaphor is not useful, and by clinging on to it we are limiting the potential of electronic books – these people also argue that whatever new metaphor are presented they will gain traction and people will get used to them over time, but one could argue that about any technology that becomes pervasive over time. .

I dont want to sound critical of Veronica’s work or even her findings I think it is fascinating, and well worth watching the talk. I just cant help but feel that we are going to be stuck with the book metaphor for a long time to come. All I have to do is look at myself – I work with computers all the time, I download e-books, PDF’s, manual’s ranging from a few pages in length to hundreds of pages. I know that it’s less wasteful and possibly more efficient, and even faster to find the bits i’m interested in by searching the electronic document … I rarely ever bother doing that. I tend to print them out, and if you a funky printer like the one we have at work it’ll print out 4 sides per sheet of A4 then hey presto you have a small A5 size copy of the document/book.

I think there’s several reason’s for this, firstly screens are terrible for reading documents on, they strain the eyes, it’s hard to find the right level of contrast, most screens flicker and the more intently you stare at them the more your eyes hurt over time, if your reading on a desktop machine or a laptop then it’s also easy to be distracted. In contrast, I can happily spend a whole day reading a book cover to cover without ever feeling fatigued or distracted. I can also scribble in them (ok I use those funky peelable sticky labels these days:p as colored bookmarks), I can carry the book with me, read it on the train or the bus, or in bed.

I don’t believe that the book metaphor actually works when interacting with a computer using a keyboard or a mouse. Any time I’ve ever tried to read a book on computer where I had to drag one corner of the book in a pseudo page turning movement, I’ve been left wondering how it would be so much easier with a next page button – the page turning animation looks nice but after a while they become irritating. The metaphor might work on more tactile interface, like the multi-touch interfaces that are slowly being rolled out.

Now I know I’m not a luddite and I’m certainly not anti technology it’s just that I haven’t come across a way to read an electronic book that I’d prefer over having a real book … yet! We’re seeing advancements now in paper technologies where electronic books can be downloaded onto special types of paper that will display the content. Perhaps that’s the real way forward? Have a real book but you can change the contents of to whatever your currently reading … best of both worlds?

Microsoft Surface

Rob mentioned Microsoft Surface to me at lunch time, I had seen it on several blogs but   haven’t really had a chance to have a look at it. It’s very impressive technonology. as Rob points out its a combination of many different muli touch techniques that we’ve seen in other demonstrations over the last six or so months. As far as I’m aware this is the first commercial product to enter the market, and they’ve done a great marketing piece on it over at http://www.microsoft.com/surface

However if you want to see it an action, I found this product demo on YouTube. I was very impressed with how the surface interacts with real world objects like camera’s, for example, placing a digital camera on the table will automatically grab the photos off it.

Below is another demonstration of Surface, this time over at PopularMechanics

There more I see this in action the more I want one!