What exactly do you mean by PITA? In most cases in my experience, you have two very simple choices - 1. don't modify the existing API / public structures as you change things and the ABI will stay safe; 2. just change things and bump up major number when necessary.
The third way of carefully doing only additive modifications to the library to preserve ABI even over upgrades is hard - that's true. But I don't think it's needed that often once most of the features are in. And again - it applies only if you've got some really popular software that gains anything by doing it this way, rather than just bumping up the version.
This. Working with SO versions is much easier in my experience than working with Ruby gems where every minor upgrade could (and sometimes would) break everything.
So with shared library dependencies, you can be very broad (e.g. "any libm.so.5 please") and get security updates etc. automatically without needing to recompile, while in "modern" programming languages you will be locked onto old and rancid libraries because the app developer doesn't bother updating the locks.
And then distro package managers go about and complicate things by having overly rigid dependencies and only allowing one version of each package name to be installed.
And the recently introduced container based replacements are and exercise in shooting twee twee birds with AAA...
Sure, which is why we have `libopenssl0.9.8`, `libopenssl1.0.0`, etc. In Debian at least, the policy is that package names for libraries should include something approximating the SONAME to allow for exactly this.
Eventually you drop the old one, because you can no longer provide security support or for some other reason. But if there are compelling reasons otherwise, distributions can and do provide multiple ABIs for the same library.
At the very least shoulnd't they be providing and supporting multiple major versions on the same system? Major versions are, by definition, for breaking/incompatible changes, so having them being mutually exclusive creates impossible situations doesn't it?
Well yes, but using the package name to distinguish them makes it harder for third parties to do packages. This is the complaint that has lead to the development of container based formats in the first place.
Thing is that with sonames the actual files are distinguished anyways, so mangling the package names to get around collisions are just an artifact of an overly rigid package manager.
I think you are talking about compatibility in general (which is great to have!) but not about ABI compatibility. Nothing apart from cultural conventions prevents maintainer of a ruby gem from strictly adhering to semver or maintainer of a C library from breaking everything in a minor version release.
Agreed. Ruby just hasn't developed the culture of compatibility that's mostly adhered to e.g. in Qt/KDE land where I lived for a long time.
Related: I'd like it very much if compatibility was enforced automatically. For example, if I publish version 1.2 of my library to $repository, then $repository checks if 1.2 passes the 1.2 tests (of course), and also if it passes the 1.1 and 1.0 tests. If any of these tests fail, then release should be downright refused unless you upload it as 2.0.
And of course, once you have that, you can think of any check you like. In C++, you can pretty reliably detect code changes that change your ABI in backwards-compatible or -incompatible ways, so you could allow backwards-incompatible changes only in major and backwards-compatible changes only in minor releases.
What I mean is that in addition to rules of the language I program in (say C or C++) there is another bunch of seemingly arbitrary rules that I must constantly be aware of to ensure nothing breaks. Rules like "don't reorder fields of structs" or "you can add new methods to a class as long as they are not virtual. and sometimes you can add virtual methods too as long as they end up at the end of vtable".
How do I ensure that I conform to these rules apart from being disciplined about them? If everything compiles, am I good? No! If everything links, am I good? No! Depending on linker options and the nature of incompatibility the program using incompatible ABIs can blow up at runtime or just silently corrupt data.
And don't get me started on the venerable "pimpl idiom". A page of boilerplate just to ensure the most basic thing.
Hope that clarifies my short sentiment a bit :) I agree that once you grasp the rules following them is not that hard but it is just another bit of incidental complexity that we agreed to maintain.
Of course it won't save you from business logic bugs (changing the meaning, but not size/location of a value), but "how do I ensure that I conform" is 90+% possible to verify automatically.
It is an additional thing to be concerned about, but the alternative (static linking everywhere) is so much worse! At least, for most (but not all) situations.
It's sort of like democracy (or capitalism?): the worst system in the world, except for every other alternative. :)
Having said that, many of your pain points are specific to C++ rather than system-level ABIs. C++ compilers have to abuse most OS-level system ABIs (example: symbol name mangling) because of linkage-related concepts in C++ that don't exist at the lower-level (and simpler, yet very different) system ABI provided by the OS.
In a way, you're really complaining (rightfully so, IMO) about one of C++'s core design principles: that the programmer should never pay a performance cost (compared to C) for language features that aren't used. For example, methods are non-virtual by default. Support for virtual methods are required for polymorphism, one of the defining features of OOP. However, calling a virtual method is always going to have at least a teensy bit of overhead vs. calling a non-virtual method. As a result, in C++ all methods are non-virtual unless defined otherwise.
Higher-level (and admittedly <= C++ in performance) OO languages tend to make everything virtual by default, and may not even have a mechanism for making a non-virtual method. These languages give away a small amount of performance in exchange for a reduction in cognitive load on the programmer. Reducing cognitive load is also a reduction in potential bugs, so there may be pragmatic reasons for using a higher-level language other than making the developer's life easier.
Most (but not all) other OO languages take away the option of direct memory management, and instead have some variant of garbage collection to keep memory usage somewhat constrained. C++ can't do this by default because then it would take a performance hit compared to C. So again, you get stuck with one (or two or many) options for higher-level memory management, but by default you're doing manual memory management a la C.
I can see (heh) C++ being very appropriate for certain types of projects and certain types of developers, but IMO its popularity is mysteriously much greater then its usual level of appropriateness. I don't dislike the language (OK, maybe a little), but it's unfortunate that it is considered the default alternative to C as often as it is.
C++ ABI compat rules really hurt when they prevent you from evolving a library in a sensible way. Say that you really need to add a method to a virtual class. Unless you planned ahead by adding placeholder methods, you're SOL. What a lousy design constraint!
This problem is solvable at the language/runtime level by resolving vtable offsets at runtime, but that doesn't help the existing C++ userbase.
This is a major flaw of C++'s ABI design, unfortunately, and one reason why we see so few supported ABI-stable C++ libraries shipped with major platforms.
Most people just have to incorporate their C++ dependencies directly instead, as there is no supported ABI.
The third way of carefully doing only additive modifications to the library to preserve ABI even over upgrades is hard - that's true. But I don't think it's needed that often once most of the features are in. And again - it applies only if you've got some really popular software that gains anything by doing it this way, rather than just bumping up the version.