Core MIDI and Friends

1. 1. CoreMIDI and FriendsChris Adamson, CocoaHeads Ann Arbor Sept. 8, 2011
2. 2. Road Map• MIDI basics• MIDI APIs on OSX• MIDI APIs on iOS
3. 3. MIDI Basics• MIDI = Musical Instrument Digital Interface • Circa 1982!• Sends messages between devices, not sounds • Notes to play (pitch, impact, vibrato), patch selection, timing synchronization, etc.
4. 4. MIDI Terms• Device — participant in a MIDI network• Endpoint — one connection into or out of a device• Source — endpoint that sends data• Destination — endpoint that receives data
5. 5. MIDI MessagesSTATUS DATA 1 DATA 2
6. 6. MIDI Messages• Channel Voice Messages — Note On, Note O?, After-touch, Pitch wheel • High nybble of status is command, low nybble is channel number• Channel Mode, System Messages• Various extensions to the spec over the yearshttp://www.midi.org/techspecs/midimessages.php
7. 7. MIDI APIs on OSX• CoreMIDI — Implementation of MIDI messaging• Instrument Units — Audio Units that generate sound in response to MIDI events• Music Device — Sends commands to Instrument Units• Music Sequence / Music Player — Plays MIDI (.mid) ?les
8. 8. Core MIDI• C-based API in Audio Toolbox • Largely similar to other Core Audio APIs • Uses Core Foundation opaque types, memory management, etc.
9. 9. Core MIDI Types• MIDIClientRef — Holds state for your MIDI session• MIDIDeviceRef — A device (real or virtual) with “entities” (MIDIEntityRef), which are logical sub-systems• MIDIEndPointRef — Source or destination• MIDIPortRef — An input or output port
10. 10. Creating client & input port!MIDIClientRef client = NULL;!MIDIClientCreate(CFSTR("Core MIDI to System Sounds Demo"), MyMIDINotifyProc, self, &client);!!MIDIPortRef inPort = NULL;!MIDIInputPortCreate(client, CFSTR("Input port"), MyMIDIReadProc, self, &inPort);
11. 11. Discovering and connecting sources!unsigned long sourceCount = MIDIGetNumberOfSources();!for (int i = 0; i < sourceCount; ++i) {!! MIDIEndpointRef src = MIDIGetSource(i);!! CFStringRef endpointName = NULL;!! OSStatus nameErr = MIDIObjectGetStringProperty(src, kMIDIPropertyName, &endpointName);!! if (noErr == nameErr) {!! ! NSLog (@" source %d: %@n", i, endpointName);!! }!! MIDIPortConnectSource(inPort, src, NULL);!}
12. 12. Read Proc• Callback function indicated by MIDIInputPortCreate() • Receives MIDIPacketList • numPackets and MIDIPacket[] • Receives user-info/context pointers that you set up in MIDIInputPortCreate() and MIDIPortConnectSource()
13. 13. MIDIPacket• Contains timeStamp, length, and Byte[]• Byte[] data is the MIDI message • data[0]: status • data[1]: MIDI Data 1 • data[2] (if present): MIDI Data 2
14. 14. Parsing a NOTE ON message!MIDIPacket *packet = (MIDIPacket *)pktlist->packet;!!Byte midiCommand = packet->data[0] >> 4;!// is it a note-on?!if (midiCommand == 0x09) {!! Byte note = packet->data[1] & 0x7F;!! Byte velocity = packet->data[2] & 0x7F;
15. 15. Now What?
16. 16. Instrument Units• Audio Units that generate sound in response to MIDI events • kAudioUnitType_MusicDevice• Do not currently exist on iOS
17. 17. Instrument to audio H/W I/O Unit Unit AU Graph
18. 18. Instrument to audio H/W Effect Unit I/O Unit Unit AU Graph
19. 19. Instrument to audio H/W I/O Unit Unit AU Graph
20. 20. Mu si cD ev ic eM ID IE ve nt () Instrument to audio H/W I/O Unit Unit AU Graph
21. 21. MusicDevice.h• Small API to deliver MIDI events to instrument units• Not in Xcode documentation. Check out the header ?le • Only 4 functions • MusicDeviceMIDIEvent() sends status, data1, data2 to a MusicDeviceComponent (i.e., an instrument Audio Unit)
22. 22. Demo
23. 23. MIDI on iOS• Core MIDI added in iOS 4.2• Device connectivity is via dock port • Custom hardware • iPad Camera Connection Kit
24. 24. MIDI via the CCK• MIDI-to-USB adapters semi-o?cially blessed by Apple • Adapter must be USB MIDI Class- compliant (i.e., doesn’t need drivers) • Bus-powered devices may not work, due to low power supplied by iPad. Powered USB devices generally work.• http://iosmidi.com/devices/
25. 25. Playing sounds on iOS• No instrument units on iOS. Options: • Synthesize your own with a render callback • Play sampled sounds with Audio Queue, AV Player, System Sounds, etc.
26. 26. Demo
27. 27. Demo
28. 28. AUSampler• New instrument audio unit in Lion… and… … • Takes a sampled waveform and pitch-shifts it to make it into an instrument • Call with MusicDeviceMIDIEvent(), just like other instrument units
29. 29. AUSampler• Con?guration is a huge hassle • Huge and tota!y undocumented hassle • Build an .aupreset with AU Lab utility, or load DLS bank or SoundFont 2 ?les, or provide your own ?les • See WWDC Session 411 (“Music in iOS and MacOSX”), then ?le documentation bugs against absence of sample code
30. 30. In Summary• Musicians love MIDI. Devices are cheap and plentiful• OSX and iOS love media, MIDI included • By comparison, Android has crap MIDI support (no javax.sound.midi, just .mid ?le support in android.media.JetPlayer)• If you’re doing Mac or iOS media apps, you should consider supporting MIDI device I/O
31. 31. Also, you should buy my bookhttp://www.mypearsonstore.com/bookstore/product.asp?isbn=9780321636843 http://my.safaribooksonline.com/9780321636973 http://www.informit.com/promotions/promotion.aspx?promo=137039