Transcript CHI 2002
Look Ma, No Keyboard user interface: Tools for music pre-production Farhan Mohamed, Sidney Fels Current Sequencing Setup LMNKui Setup Human Communication Technologies Laboratory, UBC Piano Function Mappings “LMNKui System” An octave mapped to handle general editing tasks. Mapping Example: F6 Copy a sequence to the clipboard G6 Paste Undo Sequence Design C6 F#6 An ergonomic method for controlling music sequencing software from a piano keyboard by adding a momentary foot switch. An octave used by the composer very widely for editing notes and their values. A music pre-production practice involving recording, arranging and playing MIDI data. Sequencing programs allow composers to edit out mistakes, play music in various tempos, change keys without replaying parts, rearrange an entire song, draw volume graphs, and more. CUBASE VST by Steinberg Commonly Performed Tasks • Recording MIDI events on selected channels B3 C4 An industry standard music sequencing package used for our study. CUBASE allows the composer to perform complex pre-production tasks. Touch Pad Interface The composer’s creative work is constantly interrupted by the time spent switching between multiple input devices and figuring out their functions. The current setup as shown above, also occupies excessive physical space. • To distinguish between keystrokes that represent notes and those that represent macros, a momentary foot switch is used as a ‘mode switch’. Mapping Example: B4 C5 Why cumbersome? C5 Open/Close key edit window G5 Move to the next note E5 Move to the previous note B2 C3 Preliminary user studies Mapping Example: • With an average learning time of approximately 5 minutes, subjects performed both tasks significantly better with the piano controller in Test1 (p<0.05, paired t-Test). E4 Go to the previous part F4 Move down the track C4 Create a new track Tracks Parts Transport Octave (C3-B3) Mapping Example: C3 Open/Close transport window G3 Forward C2 • Although Test2 (p<0.06, paired t-Test) was done faster than Test1, the performance was not significant with the piano controller. This was due to the minimal learning time, and small number of subjects (6) used for our study. • Subjects found the piano interaction intuitive and an effective link between playing and editing. G3 Click On/Off (B3) Selecting volume Piano controller 100 80 60 40 20 0 Test 1 Test 2 Mean Time with 95% Confidence Error Bars • Exploring musical structures for good mappings • Investigating different key boundaries References User Testing Key Maps E2 120 • Continuing user studies A touch pad is used for constrained vertical or horizontal actions, for tasks like sliding a fader and drawing a modulation graph. Emulates the left button down of the mouse Computer keyboard • Applying the principles to other music instruments Touch Pad Octave (C2-B2) C2 140 Current Developments Mapping Example: The University of British Columbia Department of Electrical and Computer Engineering Momentary foot switch • Two interaction methods were tested: The Computer Keyboard and The Piano Controller. • Effects and dynamic processing • Editing notes etc. Note An octave used by the composer to arrange music by selecting and creating tracks. • Spatial positioning • Playing back recorded channels • For continuous parameters, a single point sensitive touch pad is used in conjunction with the touch pad octave. Arrange Octave (C4-B4) An octave used by the composer to play, stop, record, mute and solo a sequence or a specific MIDI channel. A volume graph for a selected channel • The design of the layout incorporates the windows and the functions most commonly used by the composer. The functions associated with each window map to a specific octave on the piano controller keyboard as shown in the Piano Function Mappings. Key Edit Octave (C5-B5) B5 What is music sequencing? Touch Pad Interface seconds B6 General Edit Octave (C6-B6) http://hct.ece.ubc.ca/research/lmnkui/ E2 Human Communication Technologies Laboratory