I wish they would have removed SysEx messages. They cause way more trouble than they're worth, now that we have property exchange/profile configuration built into the spec.
What's the problem that SysEx messages cause? I've only limited experience with them (having used them only to send config data to a device, in a way that didn't need the device to keep functioning while being updated, from a programming point of view, I found them quite convenient and simple, except perhaps the fact that you only get 7 bits per byte so may have to pack the data).
[For the benefit of HN: SysEx is a special manufacturer-specific MIDI message which is undefined, so a synth manufacturer can use it for whatever he wishes]
I build a lot of open source patch editors for older synthesizers. My beef with Sysex is that every manufacturer uses completely different approaches to defining their own proprietary messages with it. For example, nearly every synth in the universe has a sysex message for dumping a patch (a full set of parameter settings) from a synth to another or to a computer; but they define their messages is radically different ways, so I must construct an entirely different set of parsing and emitting tools for every single synthesizer, even within a given manufacturer. It's a nightmare.
So continuing this example, if the MIDI association had gotten together early on and said that MIDI dumps should have a header that looks like THIS and then all the parameters in order, two bytes per parameter with no bit packing whatsoever, no two's complement, and end with a specific checksum, then I'd have written 10x more patch editors so far. I wouldn't have to write custom parsers and dumpers: I'd just provide a list of parameters and their bounds.
(Disclaimer: been working in synth industry for decades now..)
Most SYSEX dumps are just dumps of the plain ol' structs that the synth engines are using to drive their output. A lot of synths don't have the processing power to do more than just dump the struct.
So, it wouldn't really make much sense to have them all use the same struct - this can't be enforced too well. Forcing synth mfr's to all use the same struct means that, even if they have their own internal plain-old-structs, they'd need code to dump the SYSEX according to the standard.
> Most SYSEX dumps are just dumps of the plain ol' structs that the synth engines are using to drive their output. A lot of synths don't have the processing power to do more than just dump the struct.
I don't think it's processing power: it's stingy RAM utilization. Many bad actors (Kawai, Casio, later Yamaha) did crazy bit-packing of parameters rather than just keep them in a simple array, while the more sane (Oberheim, E-mu, Waldorf, early Yamaha) at least tried to pack in a consistent way. Other bad actors (ahem Korg, as late as 2000) decided to use, shall we say, creative parameter encodings, going even so far as embedding textified versions of parameter numbers into sysex byte streams. And many used all sorts of crazy schemes for banks and patch numbering, most of which are incompatible with one another.
And it's not just encoding: basic synth patch dump features are missing from different models. There are five basic tasks that most synth editors require:
- Change patch
- Request a dump from working memory
- Dump to working memory
- Dump to patch RAM and save
- Send a single parameter change (for any parameter)
Manufacturers couldn't even agree to make machines which supported all five of these. Some machines (Yamaha) have no way to write to RAM. Some machines couldn't do single parameter changes. Some machines can't properly change patches in a consistent manner. Some machines have no patch request mechanism. Many machines can't dump to current working memory: only to patch RAM!
The situation is only getting worse. Whereas in the past manufacturers at least attempted a complement of sysex messages, now many manufacturers can't even be bothered to allow access to their machines (Korg, Roland). Others treat their sysex messages as proprietary secrets (Arturia, Digitech, Alesis).
There is only one truly good, shining actor in the open MIDI spec space, and that is Sequential. Which shouldn't be a surprise given who runs it.
"Send a single parameter change (for any parameter)"
This makes no sense. That would also imply a way to discover (and name, and probably provide semantics for) all parameters. That's a huge ask if MIDI (even MIDI 2.0) is the only communication protocol available.
Yes, the first 4 of your list are common. The first one is covered by the core MIDI spec. The 2nd and 3rd have no standard msg, but your complaint seems to be about the contents of the message, which is no business of the requestor. The 4th assumes "patch RAM", which cannot be assumed, as you note, and that seems correct to me.
Why? It's highly standard. About 90% of the synthesizers I've written patch editors for provide exactly this facility. In fact some (PreenFM2, Korg Microsampler, Futursonus Parva) provide only this facility.
> The first one is covered by the core MIDI spec.
Actually it's not. Program Change only works for 128 patches. If a synth has more than 128 (and many do), they must rely optionally on Bank Select, but their definitions of "banks" vary because a bank is not a formally defined concept. Some rationally treat banks as divisions of the patches. Others treat banks as media choices: cards versus RAM versus ROM. Some require that Bank Select be immediately before Program Change with nothing in-between; others do not. Some ignore banks entirely and instead define a "Program Change Table" of 128 slots pointing to arbitrary patches in memory, and then Program Change indicates which patch slot to use.
And there are several major synthesizers (Yamaha TX81Z and DX11 are famous examples) where Program Change is in fact broken and requires unusual workaround hacks. Further, most synths require a program change prior to a patch load: but others (notably the Oberheim Matrix 6 and 1000) require a program change after a patch load. It's a mess.
I don't think it's fair to say that "banks are not a defined concept". What has happened is that many MIDI device manufacturers have stretched and ignored the clear intent of the MIDI 1.0 specification. Reading that, it is quite obvious what the relationship of banks and program changes is. But that hasn't stopped various companies from playing games with the two (as you note) for their own purposes. Nothing will ever do that, fully.
It's not standard at all. There is absolutely no MIDI standard for the contents of a patch. I really don't know what you're thinking of.
Back in the 90's, when things like "MIDI Librarians" were common (and widely used), each new device needed to be added to the MIDI Librarian's code to deal with the specifics.
What I mean is that, if they have to reformat their internal struct to conform to a standard, they don't have the processing power to do this munging. Not that they'd care, as you have noted elsewhere.
The point of Sysex messages is moving complexity, such as disentangling bitfields, from the lean hardware of a synth, entitled to do whatever is more convenient, to a deluxe computer that can afford the plasticity of software.
I can't know how big a SysEx message is until run time, which makes buffering them somewhat complicated when you don't know what's in the SysEx message.
This isn't uncommon in serial protocols, but MIDI has been lifted several layers above the UART it was designed for, and one of its strengths for everything but SysEx is that you exactly how big messages are going to be and preallocating space for them is trivial.
> I can't know how big a SysEx message is until run time, which makes buffering them somewhat complicated when you don't know what's in the SysEx message.
That's the reason why you want to break compatibility with MIDI 1 spec? Really? Just like for an HTTP stream, you don't need to know how big the stream is to process it, just when it ends, there is no new problem to solve here.
No, I want there to be only one way to do something in the protocol and to remove arbitrary binary exchange, this is one pain point. It turns your "standard" into a Frankenstein's Monster of data exchange, and you'd think we'd have learned over the last few decades that ambiguity in specification of a communication protocol is just going to lead to headaches and incompatible/buggy hardware.
It's also weird to say SysEx is needed for backwards compat with MIDI 1.0. You can just require a translation layer between MIDI 1.0 sysex and MIDI 2.0 configuration/property exchange for backwards compat. Opaque binary as a part of the protocol does not encourage compatibility among hardware.
The MIDI spec was successful for 30+ years for a reason and manufacturers managed with SysEx without any issue, there is no justification for any "compatibility layer" whatsoever here. The problem wasn't the spec obviously.
As a musician and software developer I'd disagree. Sure, manufacturers are getting on fine because they can do whatever they want, but it makes the setup for anyone who isn't the manufacturer much more bespoke and challenging, as others have said in thread.
Leaded gasoline was successful for decades too, but there comes a time to take things to the next level.
> As a musician and software developer I'd disagree. Sure, manufacturers are getting on fine because they can do whatever they want, but it makes the setup for anyone who isn't the manufacturer much more bespoke and challenging, as others have said in thread.
MIDI 1.0 was successful for 30+ years because of its simplicity, so I'm not sure what you are disagreeing with. Building a MIDI payload is extremely simple and serious manufacturers will document their MIDI implementation in public documents.
What part of the MIDI spec did you have hard time with when developing software using the MIDI 1.0 protocol?
Your comment about gasoline has absolutely nothing to do with MIDI.
Allocating memory (by asking the OS) in a realtime context is a no-no. This means guessing the maximum size of a sysex ahead of time or breaking RT programming rules.
The other tricky issue is the way MIDI 1.0 defines the transmission of the LSB and MSB for 14 bit messages. They got it backwards, requiring the receiver to use a timer. Might have been sensible for baked-in-h/w MIDI gear, not so great for general purpose computing.
It's been kind of a headache in Firefox's WebMIDI support plans (since SysEx is sometimes used for firmware updates and it's hard to explain "this page may hack your keyboard" in a permission request dialog): https://github.com/mozilla/standards-positions/issues/58#iss...
Lots of 1.0 instruments have only SysEx to load/save patches. A single patch may have hundreds of parameters. Whole libraries of patches exist; they can be loaded/saved at once with Sysex. So, no way.
I don't really see why that matters for a new protocol that fallbacks to MIDI 1.0, it's not a superset of MIDI 1.0. If your device only takes MIDI 1.0 SysEx for patches, anything that would send those patches via MIDI is going to do it on top of the fallback.
What I'm saying is a new MIDI 2.0 device probably shouldn't be able to use SysEx. Either opt out of MIDI 1.0 and use the paradigms established or fallback to MIDI 1.0 and its limitations. Otherwise we're just going to get a mess of different implementations, and MIDI 2.0 will fail to be a successful standard.
