Motion

The web page for our new research project called Sensing Music-related Actions is now up and running. This is a joint research project of the departments of Musicology and Informatics, and has received external funding through the VERDIKT program of the The Research Council of Norway. The project runs from July 2008 until July 2011. The focus of the project will be on basic issues of sensing and analysing music-related actions, and creating various prototypes for testing the control possibilities of such actions in enactive devices....

Talking about large-scale motion tracking:

Wired reports that Apple has filed around 200 patent applications related to multitouch and gesture control: Yet it appears that the company is not trying to patent the entire multitouch concept, but rather trying to protect certain uses of it – specifically the methods to interpret gestures, and in some cases, the gestures themselves. It is interesting to see that they mention the interpretation of a gesture. This means that they separate between gesture and action, i....

The location of the EyesWeb Week is the facilities of the DIST group in the beautiful Casa Paganina, including a large auditorium next to the laboratories. This allows for an ecological setting for experiments, since performers can actually perform on a real stage with real audience. I wish we could have something like this in Oslo! Here a picture from an experimental setup where we are looking at the synchronisation between the musicians in a string trio....

On the plane down to Genova I had an interesting tactile experience. It turned out that the box that the lunch was served in had this 3D ornament of a walnut on the top cover (not so easy to represent in a picture, but you get the idea). Interesting how much this enhanced the experience! {width=“400” height=“300”}

The Peretz group has made available a set of musical excerpts with emotion ratings. Perhaps not the most exciting musical collection, but I think it is very important that the community starts building some data sets that can be used as reference for various type of analyses. We really need to create a set of music recordings including motion capture and video, but this first requires that we develop some common format that can be used for synchronisation and sharing....

I just heard Esteban Maestre from UPF present his project on creating a database of instrumental actions of bowed instruments, for use in the synthesis of score-based material. They have come up with a very interesting solution to the recording and synchronisation of audio with movement data: Building a VST plugin which implements recording of motion capture data from a Polhemus Liberty, together with bow sensing through an Arduino. This makes it possible to load the VST-plugin inside regular audio sequencing software and do the recording from there....

I just heard a presentation by a group of researchers from the Tril centre (Technology Research for Independent Living) in Dublin. They have developed Emobius (or EyesWeb Mobius), a set of blocks for various types of biomedical processing, as well as a graphical front-end to the forthcoming EyesWeb XMI. It is fascinating to see how the problems they are working on in applications for older persons are so similar to what we are dealing with in music research....

We had a programming session this morning, and Paolo Coletta implemented a block for creating motiongrams in EyesWeb. It will be available in the new EyesWeb XMI release which will happen in the end of this week. Great!

All the big companies are launching new devices with “gesture control” these days, and Microsoft is following along . This insider story presents some of the new features in Windows Mobile 7 which is supposed to take on the iPhone later this year. The most interesting part of the “leak”, I think, is the differentiation between touch gestures and motion gestures, where the former is related to what I would call manipulation, such as in this example of one and two finger strokes:...