Transcript Slide 1
Human-Computer Interaction for Music Workshop COMP 250 October 16, 2012 Collaborative Tabletop Performance Dan Afergan • Performers can swipe the tabletop to change the interface or instrument that he/she is using • Supports any instrument that can be turned into a touch-screen interface: turntables, digital MPC, keyboard, reacTable, Audiopad, drum pad… Collaborative Tabletop Performance • Performers can send their instrument to another surface by pinching + “throwing” towards another performer • Performers can share an instrument with a full hand swipe to someone else • Ability to record/playback instruments or entire performances Prof. Auner Sonic Ball • A ball with built-in sensors that track velocity, height, spin, impact, bounce etc. so that these values can be mapped on to musical controls. • Players from one to any number wear Bluetooth head sets with microphones that allow them to hear the music and to communicate with each other (and to rap, sing, etc. along the with music). The music can be broadcast only to the headsets or can be sent to external speakers. • The prototype will use a basketball (and thus build on all the physical skills already in place), but the technology will be adapted to soccer balls, tennis balls, footballs, bowling balls, etc. Can be made part of all traditional games using those balls or developed into new games that take advantage of the musical interface. Sonic Ball • Any kind of music can be used, but the protoype will use dubstep. The ball at rest produces a mellow ambient dubstep groove; when it is thrown the sounds get brighter as it speeds up (then duller as it slows down). • When you throw it high, more layers are added (and then subtracted as it falls)When you catch it there is a cool percussive effect--break beat or drum fill; when it bounces there are single percussive effects. More motion, faster motion, harder impacts, higher trajectories, faster bounces, etc. produce more intense musical effects. • The ball and headsets will charge on a wireless charging mat, which will also have all the controls (on/off switch), headsets or external speakers, etc. • Rationale: The point is to add music to physical activities to increase skills and coordination, social bonding, to better understand physical phenomena through the musical mapping, integrate dance with sport, and to have fun. Heart-rate Based Dynamic Tempo Modules D.F.Brown – HCM Workshop Proposal Rationale / Input • Rationale: An average resting human heart rate lies somewhere within 60-100 BPM. Most musical ‘beats’ run within a spectrum of 40 – 200 BPM. Depending on how a heart rate is broken-down as a beat (heart BPM can be multiplied by two as one musical ‘beat’), the two lie in a remarkably similar realm. I find the beat of our hearts intrinsic in our experience and production of music. I do not think the history of human music could have developed far from the timing within the ever-present biological metronome of our heart. I’d like to find a method of bringing the less-controllable biological timing of our hearts into the process of music production and experience, in order to break away from strict timing structure towards a more abstract, ephemeral form. • Inputs: Digital heart rate monitor, and one or several human controlled digitally alterable inputs (midi-controller, MPC, Kinect). Interaction / Evaluation • Interaction: A digital musical interface with programmable functions is loaded with a bank of drum beats, treble melodies, bass lines, and whatever else can be strategically managed by the specific digital device. The tempo of each sample, however, will increase or decrease in real time in correspondence with the heart rate of the performer, being digitally monitored and fed into the system. The result, in theory, would create a more biologically dynamic timestructure in the musical performance, directly correlated to and impacted by the physical experience of the performer in process. • Evaluation: This system needs to be experimented with to see how the resulting music may sound with the available technology. There may be issues with lag-time that I am unaware of, perhaps not. I believe the system will have the desired effect if one can effectively hear changes in the music as the performer is evidently taxed or relaxed by his physical or psychological process. Washboard app for iPad Workshop Proposal by Marci Cohen Rationale: takes advantage of: • Interest in folk music • Ubiquity of iPads • Obscurity of real washboards Interaction: • Screen has graphic of real washboard • User holds iPad vertically in lap, moves fingers vertically along surface to trigger samples of real washboard • Sound varies with speed of scratching Shawn Fitzpatrick Rationale: It is important for runners to keep a consistent pace when running long distances. This music interface will help runners maintain their desired pace by changing the tempo of their music. Input: Runner’s desired pace, GPS coordinates used to calculate the runner’s speed Interaction: The user changes the tempo of the song based on their running pace. If the user is running faster than their desired pace, the music slows down. If the user is running slower than their desired pace, the music speeds up. Evaluation: Analyze how well the user adjusted their pace to the tempo of the music. If the tempo speeds up, the runner should speed up. If the tempo slows down, then the user should slow down. Audience: Long-distance runners who would like to keep their pace through music Eli Fox-Epstein Suppose I'm a software engineer. Are my tests passing? How many rps are we getting? Are 5xx errors spiking? Goal: Know without distraction. But isn't this just vanilla sonification? "... without distraction" Can't distract me from my music. Can't make my music sound bad. How: Figure out what user is hearing. Accompany that with data. Adjust to user's agitation. Kathleen Kuo 10/16/2012 Why create the Songboard? Can be placed in classrooms (like traditional whiteboards) for educational use, or at home for recreational use Supplement traditional music education by allowing teachers to show notation, rhythm and pitch in real time Grant composers the ability to play any instrument, no longer limited to just piano, and have the notes instantly appear Easy and fun sound- and touch-based input for musicians and composers of any age How does the Songboard work? Frequency-sensitive sensors can be attached to or placed near an instrument and capture pitches to display on the Songboard in real time Touch-screen technology or projection + computer allow the teacher/musician/composer to create, alter, or delete notes on the board Option to create and view music without having to write, click a mouse, or touch a screen Potential to be used in multimedia and art installations on top of classroom, recreational use CAR RADIO ADJUSTER Rationale: To notify driver through music if s/he is driving slower/faster than normal. People tend to pay attention to music playing when driving. Input(s): Driver’s gaze and eye movements if adjusting radio station, spoken words if controlling the radio through voice commands, button clicks of radio station presets, driver’s foot control as s/he presses the gas pedal. Interaction: Car radio adjusts the music tempo being played. If driving faster than normal, music will be unrecognizable. <Confusion> If driving slower, music slows down to a crawl. <Frustration> FRANCINE LALOOSES Assumes driver will alter gas pedal behavior upon emotional states. Evaluation: System will adapt to driver’s normal speed and provide feedback through music tempo. Success based on driver driving at consistent speeds on roads. Audience: Drivers on the go who listen to the car radio. Dylan Portelance Comp 250 - HCM Beste Yuksel Using Genetic Algorithms for Music Composition Start with population of audio samples. Use some kind of fitness function to select the ‘fittest’. Population reproduces and mutations occur. The fittest survive. Repeat until desired effect. Proposed Method Ramdom sample of rhythms. Fitness function determined by mood of user (fNIRS or other physiological measure) Those rhythms which trigger the desired emotional response reproduce. Continue for many generations until achieve desired musical output or hit limit.