class: center, middle # To Gesture or Not? An Analysis of Terminology in NIME Proceedings 2001–2013 *Alexander Refsum Jensenius* *fourMs group, Department of Musicology, University of Oslo* ??? C = two screens p = presentation mode cmd+f = fullscreen Good afternoon everyone, my name is ARJ... This will be a presentation on terminology, a meta-analysis, if you will --- class: center, middle # Gesture? ??? My starting point is the term "gesture", which I have been struggling with for some time. It is not that gesture in itself is such a bad term. Rather: it seems like we in the NIME community are using it somewhat differently from the rest of the world. That may not be a problem in itself, if we understand each other, BUT: 1. we may be confused within NIME 2. we may get into trouble with people outside our community Gesture can be used for many things... --- background-image: url(figures/gestures.jpg) --- background-image: url(figures/singlove.jpg) --- background-image: url(figures/10-songbook.pianist.jpg) --- background-image: url(figures/15847938_debf4d0022_o.jpg) --- background-image: url(figures/5662692724_ac5389a641_b.jpg) --- background-image: url(figures/Award-Show-Percussion-D53C1531_JPG.jpg) --- background-image: url(figures/4711402304_0d4da898d5_b.jpg) --- background-image: url(figures/NIME_6a.jpg) --- # ? -- 1. How much is ‘gesture’ used at NIME? -- 2. How much is ‘gesture’ used in related fields? -- 3. How is ‘gesture’ used, and with what meaning(s)? ??? Three main points --- class: center, middle # Gesture definitions --- # Oxford dictionary **Gesture:** *"a movement of part of the body,* *especially a hand or the head,* *to express an idea or meaning"* ??? Similar to MacMillan and Dictionary --- # Oxford dictionary **Gesture:** *"a **movement** of part of the body,* *especially a **hand** or the **head**,* *to **express** an **idea** or **meaning**"* ??? Similar to MacMillan and Dictionary --- # MacMillan dictionary **Gesture:** *"a movement that communicates a feeling or instruction [...]:* - *hand movement that you use to control something such as a smartphone or tablet [...]* - *the use of movement to communicate, especially in dance"* --- # MacMillan dictionary **Gesture:** *"a **movement** that communicates a **feeling** or **instruction** [...]:* - *hand movement that you use to **control something** such as a smartphone or tablet [...]* - *the use of **movement to communicate**, especially in dance"* --- # Research definitions of gesture -- *(Cadoz & Wanderley, 2000) – (Jensenius et al., 2010):* -- **Communication**: convey meaning in social interaction (linguistics, psychology) -- **Control**: interact with a computer-based system (HCI, computer music) -- **Metaphor**: project movement and sound (and other phenomena) to cultural topics (cognitive science, psychology, musicology) ??? All in all, in the sub-communities around NIME, gesture is used very differently. Now... --- class: center, middle # Gesture@NIME? ??? Using the NIME proceedings as source material --- # Method -- 1. Download PDFs from nime.org -- 2. Consistency check in Acrobat -- 3. Replace/remove PDF files -- 4. Define search terms: ‘music’, ‘gesture’, ‘action’, ‘motion’, ‘movement’, ‘emotion’, ‘expressive’, ‘motion capture’, ‘accelerometer’, ‘wii’, ‘kinect’, ‘leap motion’ -- 5. OSX shell script: ```mdfind -count``` -- 6. Manual check of some files -- 7. Calculate basic statistics ??? Result: 1 108 files! --- # NIME | % | Total | music | gest- ure | act- ion | mot- ion | move- ment | emot- ion | expre- ssive | motion capture | acceler- ometer | wii | kinect | leap motion | | :-:| -: | -: | -: | -: | -: | -: | -: | -: | -: | -: | -: | -: | -: | | 2001 | 14 | 100 | 64 | 57 | 50 | 64 | 14 | 57 | 29 | 29 | 0 | 0 | 0 | | 2002 | 48 | 100 | 65 | 52 | 58 | 65 | 17 | 60 | 23 | 15 | 0 | 0 | 0 | | 2003 | 48 | 100 | 71 | 35 | 40 | 48 | 19 | 50 | 15 | 17 | 0 | 0 | 0 | | 2004 | 54 | 100 | 56 | 37 | 39 | 54 | 22 | 43 | 22 | 20 | 0 | 0 | 0 | | 2005 | 75 | 100 | 63 | 48 | 45 | 56 | 23 | 48 | 23 | 24 | 0 | 0 | 0 | | 2006 | 81 | 100 | 64 | 41 | 36 | 52 | 7 | 41 | 23 | 16 | 0 | 0 | 0 | | 2007 | 103 | 100 | 55 | 38 | 40 | 57 | 20 | 50 | 18 | 17 | 4 | 0 | 0 | | 2008 | 87 | 100 | 60 | 52 | 52 | 59 | 14 | 45 | 25 | 22 | 16 | 0 | 0 | | 2009 | 110 | 90 | 54 | 36 | 37 | 51 | 12 | 35 | 13 | 26 | 12 | 0 | 0 | | 2010 | 111 | 100 | 66 | 50 | 44 | 55 | 23 | 45 | 26 | 26 | 14 | 0 | 0 | | 2011 | 130 | 100 | 67 | 60 | 45 | 59 | 16 | 51 | 25 | 26 | 13 | 5 | 0 | | 2012 | 129 | 99 | 63 | 61 | 44 | 57 | 16 | 53 | 26 | 33 | 10 | 12 | 0 | | 2013 | 118 | 99 | 65 | 51 | 49 | 64 | 25 | 56 | 30 | 26 | 11 | 22 | 1 | | ||||||||||||| | | **Mean** | 85 | **99** | **62** | 48 | 45 | 57 | 18 | 49 | 23 | 23 | 6 | 3 | 0 | | **Stdev** | 36 | 3 | 5 | 9 | 6 | 5 | 5 | 7 | 5 | 6 | 6 | 7 | 0 | ??? Music: almost 100% Gesture: 62% -- remarkably stable! Action, motion, movement: around 50% Emotion: low Expressive: high (removed expression) Mocap + accelerometer: 20% (fairly low) Technologies as they are introduced to the market --- # SMC | % | Total | music | gest- ure | act- ion | mot- ion | move- ment | emot- ion | expre- ssive | motion capture | acceler- ometer | wii | kinect | leap motion | | :-:| -: | -: | -: | -: | -: | -: | -: | -: | -: | -: | -: | -: | -: | | 2004 | 47 | 100 | 30 | 38 | 15 | 36 | 17 | 40 | 9 | 2 | 0 | 0 | 0 | | 2005 | 32 | 100 | 28 | 47 | 28 | 56 | 25 | 31 | 13 | 0 | 0 | 0 | 0 | | 2006 | 26 | 104 | 27 | 50 | 27 | 42 | 19 | 31 | 23 | 0 | 0 | 0 | 0 | | 2007 | 61 | 100 | 38 | 44 | 41 | 54 | 25 | 41 | 13 | 7 | 2 | 0 | 0 | | 2008 | 34 | 100 | 32 | 32 | 26 | 56 | 9 | 24 | 9 | 0 | 0 | 0 | 3 | | 2009 | 62 | 100 | 26 | 39 | 26 | 47 | 16 | 35 | 10 | 13 | 3 | 0 | 0 | | 2010 | 74 | 99 | 27 | 39 | 34 | 34 | 18 | 36 | 11 | 4 | 1 | 0 | 3 | | 2011 | 79 | 99 | 38 | 44 | 35 | 54 | 22 | 41 | 22 | 14 | 5 | 4 | 0 | | 2012 | 75 | 99 | 40 | 52 | 31 | 39 | 19 | 23 | 19 | 11 | 4 | 7 | 1 | | 2013 | 111 | 100 | 38 | 38 | 33 | 47 | 22 | 30 | 18 | 11 | 5 | 5 | 1 | | ||||||||||||| | | **Mean** | 60 | 100 | 32 | 42 | 30 | 47 | 19 | 33 | 14 | 6 | 2 | 2 | 1 | | **Stdev** | 26 | 1 | 6 | 6 | 7 | 9 | 5 | 7 | 5 | 6 | 2 | 3 | 1 | ??? Same analysis for SMC Gesture: 32% action, motion, movement: higher than gesture emotion, expressive, lower --- # Comparison | % | Total | music | gest- ure | act- ion | mot- ion | move- ment | emo- tion | expres- sive | motion capture | acceler- ometer | wii | kinect | | :- | -: | -: | -: | -: | -: | -: | -: | -: | -: | -: | -: | -: | -: | | NIME | 1,108 | 99 | 62 | 48 | 44 | 57 | 18 | 48 | 23 | 24 | 8 | 4 | | ICMC | 3,687 | 100 | 17 | 17 | 20 | 24 | 4 | 20 | 2 | 3 | 1 | 0.2 | | SMC | 601 | 100 | 34 | 42 | 31 | 46 | 19 | 33 | 15 | 8 | 3 | 2 | | ACM + Aff. | 399,664 | 4 | 3 | 22 | 8 | 10 | 2 | 4 | 3 | 1 | 0.4 | 0.4 | | ACM Guide | 2,193,894 | 2 | 1 | 11 | 5 | 5 | 1 | 2 | 1 | 0.4 | 0.1 | 0.1 | | J. bio- mechanics | 18,193 | 0.3 | 0.2 | 38 | 51 | 43 | 0.1 | 0.03 | 6 | 4 | 0.1 | 0.02 | ??? NIME + SMC = similar analysis The others: search in databases ICMC: small percentage, but similar absolute numbers as NIME --- # Gesture collocation @NIME -- - "gesture **recognition**" (R1) -- - "**sound** gesture" (L1) -- - "gesture and **sound**" (R2) --- # Collocation method -- 1. PDF -> txt (CasualText) -- 2. Clean up (TextMate) -- 3. Collocation analysis (CasualConc) -- 4. Remove common words --- # Collocation (total) | Word | LR total | :- | -: | sound | 402 | recognition | 378 | control | 243 | musical | 214 | data | 212 | mapping | 210 | based | 207 | music | 189 | analysis | 173 | system | 149 --- # Collocation (left) | Word | L total | :--- | ---: | | expressive | 115 | musical | 110 | sound | 87 | mapping | 86 | time | 85 | hand | 78 | physical | 68 | control | 65 | instrumental | 58 | instrument | 52 --- # Collocation (right) | Word | R total | :- | -: | | recognition | 344 | sound | 315 | control | 178 | data | 165 | based | 158 | music | 142 | analysis | 128 | mapping | 124 | musical | 104 | system | 97 --- # Conclusions -- NIME proceedings = interesting meta-research material! -- Gesture@NIME: -- - in 689 papers (62%) -- - used 4,211 times (6 times per paper) -- - used more widely than at SMC (34%) and ICMC (17%) -- - many different meanings: motion, technology, metaphor... --- # Conclusions So... -- - Define your use of the term -- - Use other words when appropriate --- class: center, middle # www.arj.no