It's likely a human resources problem. For every competent FreeBSD or Illumos sysadmin there are 10x equally experienced with Linux. And those numbers are much worse outside of major cities. The commercial support from Ubuntu tips the scales.
I made this same decision at my last job. I ran Solaris and Illumos on our file servers and loved it, but a year before I left I ported all the pools to Ubuntu so my successor only needed to be Linux competent and ZFS trainable.
Sometimes when choosing tech it's not what's technologically superior solution nor what you personally could run well, it's what's best for your coworkers, your successor and the organization in the long term.
> For every competent FreeBSD or Illumos sysadmin there are 10x equally experienced with Linux
That's the Nth I've read this quote on HN, it became a classic... You can't find a FreeBSD sysadmin but you can find a Linux admin.
Where I work I have to deal with AIX, Solaris, Open/FreeBSD (Had Net before), Linux (all major flavours) and (god Forbid) Windows Server (2008, 12R2, 2016 and Nano). I've build packages for most of these systems. I don't know all of them inside/out but NEVER had problem implementing/setting/testing features in any of them.
Can you tell me in a what way an intermediate (say 5 years) of experience, linux sysadmin would have problems managing a Free/Open/NetBSD?
Guess it depends on the flavor of admin. With some regularity a linux admin "with decades of experience" will show up and announce that openbsd is terribly broken and nothing works, not even the most basic pkg-add command. Uh, did you mean pkg_add? See! Openbsd is so broken they called the command pkg_add while I typed pkg-add. I've never had this trouble with linux!
I'd be worried about letting such a person admin linux servers, but I guess you can limp by as long as you keep your infra to what they already know. Ideally you'd weed such people out before hiring, but maybe if you need to hire an admin you don't know enough to do that?
Totally agree. BTW, thanks for signify it is an awesome system for pkg signatures. It took me a while to get it working correctly for automatic pkg signing but once it all clicked together, the system was simple as any I've see!
ps. The best documentation I've found, was the manpage[1].
It's hard enough to find Linux admins, even in major cities. There are people who will put Linux experience on their resumes, but almost nobody who can boast more than casual home use. We ended up giving up on finding a Linux admin after a few months of the job posting collecting dust and decided to just hire someone fresh out of college a train them from scratch.
Then you run into the problem that BSD admins are even rarer, and AIX/Solaris even rarer than that.
Someone with experience with Linux could likely find their way around any UNIX like system but given the choice we would rather deal with some awkwardness in a familiar environment where admins have far more experience and we can leverage our exiting infrastructure, scripts, and config management.
FreeBSD is just as easy to administer as Linux. Solaris I find to be a bit more challenging. I can appreciate your commentary, but I challenge you to prove me wrong. Just for some color I've ran the Gambit of Unix and Linux systems in my career with FreeBSD being one of the easiest and most consistent out of the box technologies. Solaris had already been quite the opposite. I guess you could say AIX is a close second to Solaris, but I think that's a bit of a grey area considering how opinionated that operating system is.
To be clear I'm not a FreeBSD zealot. I'm for what works. Right now what works is the emerging container based solutions albeit not relevant to this discussion.
Not exactly, but to any competent administrator, learning FreeBSD based on Linux experience is not that difficult. It's sort of like tasking someone to start working on a program written in Go even if they have only C++ experience. You can cope.
I get a little annoyed with this line of reasoning though.
"Anyone can learn anything" doesn't help me if I need an expert now. And it doesn't magically jump the gap between "functional" (I can make a thing work in an ugly and naive way) and "good" (I can weigh the trade-offs behind the scenes and choose the optimal from multiple alternatives).
Unless the assertion is that FreeBSD / Go is easy, logical, and/or obvious enough that a master Linux / C++ programmer will be productive and community standard-compliant without any effective lag time.
And I'm not trying to be obtuse. I honestly see it a lot and think it's a blind spot: reverse Pareto principle if you will. "Getting up to 80% proficiency is easy, so let's ignore the last 20% because it must also be easy."
Experts have to get made somehow. It's not as if Linux is a frozen target where you can count on being "productive and community standard-compliant without any effective lag time" without going back to the docs sometimes.
A great example in my opinion: Red Hat RHEL7 introduced systemd. A lot changed versus RHEL6. RHEL6 "experts" turned into clumsy RHEL7 "80%-ers". We figured it out.
Not to even mention that SuSE, RHEL, and Ubuntu are about as similar as "Linux" and FreeBSD, if you are worried about the finer points of best practice. We figure it out.
Absolutely. It happens. But to harken back to the original post, there are definite advantages to "technology with X experts available in the market" than "technology with Y experts available in the market". Where X > Y.
And those advantages don't disappear even if Y is easy to learn.
On that basic point, I agree -- although there are advantages to swimming upstream sometimes. Otherwise, given the landscape of 10-15 years ago, we'd be having this discussion about Windows servers instead of Linux!
Besides, probably the best way to find out if it's a "big deal" is to ask your sysadmins. Or, generally, the people who are going to be stuck running it.
Granted on the upstream point! Especially with how quick transformative technology goes through its various phases, it may be essential (/Strangelove emphasis) to make the harder choice now so that you're not behind your competitors in the near future.
I think your argument lies more in the fact that linux is non standard to Unix. They have gone their own way and made it difficult to transfer knowledge. Ask anyone who has ported a Linux application to any other Unix. It's at best a PITA. At worst a nightmare.
Can you do it? Sure. But it isn't pleasant. The Linux community is off in the weeds imo. Doing their own poor re-implementations of tech others have already done. See: Dtrace, Filesystems, Jails/zones, Networking, VM, Init systems, ...
So I think trying to argue it's easier to install Linux because Linux folks can't transfer their knowledge to other OS's speaks volumes to it being a poor choice to invest skills in if you can't transfer them to other OS's
That's not really a fair statement. First of all, Linux started as Linus just reimplementing the Unix semantics, so edge cases and subtle semantics should be expected to be different. Secondly, most Unix-based OSes are barely compatible in their facilities.
Sure, you have DTrace (which only macOS, illumos and FreeBSD have) and ZFS (which only illumos and FreeBSD have) but the rest is similarly incompatible. Solairs/illumos even has a complete NIH-reimplementation of FreeBSD's kqueue (event ports). They have different views on containerisation (Zones/Jails). They've historically had very different opinions on /proc and ioctls, not to mention that they were developed separately for such a long time that their shared history is not very recent.
As a result, porting from Solaris to FreeBSD is also difficult. Maybe it's harder or easier than porting to GNU/Linux, but I wouldn't just flat-out claim that GNU/Linux is the only member doing things that are incompatible.
Sun's event ports are so similar to kqueue cantrill has said they should have just adopted kqueue.
And there are not different views on zones and jails. They are the same thing. Sun just took the idea of jails and flushed them out further adding a separate network stack for each zone, which jails now also have. But they operate on the same principle and ideas. Jails was bare bones at inception. Jails shared their IP stack with the host, this was before cloud computing and the need for separate network stacks. They both started with being secure and then adding features where the Linux container mess started with features and then continues to try and address the fact they are insecure by design.
So jail and zones are similar.
Porting from FreeBSD to Solaris and vice Versa is easier than you think. If it was insanely hard FreeBSD wouldn't have zfs or Dtrace from Sun. Dtrace was almost single handedly ported by one engineer. An amazing engineer, but he Was the only one. Same with ZFS. And illumos ported the FreeBSD installer. Also done by one individual. All very good engineers but still just one.
I also have watched Cantrill's talks. But I think you're intentionally ignoring my point, in two ways:
* You were discussing porting a Linux application, not a kernel feature like DTrace, ZFS, Zones, kqueue, etc. Obviously porting a kernel feature between two OSes that share a kernel history is going to be easier than porting to an entirely different kernel. It's almost tautological. Porting an application has more to do with whether the syscall/libc interfaces are compatible and if you take kqueue/eventports as an example you still need to do standard porting work. glibc provides BSD-like interfaces so it's not like you have to switch away from bzero or whatever -- that's not the hard part of porting code.
* You specifically stated that Linux is "off in the weeds", "doing their own poor re-implementations of tech others have already done". Ignoring how disrespectful that is, your response to me saying "the whole Unix family re-implements each others ideas all the time -- DTrace and ZFS are the exception and only three members of the family use them" isn't helping your original point.
Also this whole section is just a non-sequitur:
> And there are not different views on zones and jails. They are the same thing. [Long description of how they are different and were developed separately.] [Random aside about Linux containers and how they're a mess.] So jail and zones are similar.
I am aware of the similarities and differences between Jails and Zones, and I'm also very painfully familiar with Linux containers. Not sure why you're bringing them up in a discussion about porting applications between different Unix-like operating systems. Sounds like you just have an axe to grind.
event ports is not an NIH reimplementation. It was a framework intentionally developed to meet specific business and technical needs within the Solaris threading model at that time.
It's far closer to Windows IOCP than FreeBSD kqueues IMHO.
It's also been one of the most successful features added to Solaris and is used throughout the system.
I believe it was. See comment above. FreeBSD has excellent documentation on nearly every topic. Nearly every program has a manpage and there's always google to help you.
For advanced topics (CARP, DTrace, ZFS, Jails, Accounting API) the docs are excellent and you'll have to some reading to properly implement any of those anyway.
That's part of the job actually, reading/learning.
I run napp-it on omni-os at home and work with good results. It's not too hard to adapt your linux knowledge if you're not used to the solaris-ish userspace.
Sure but that alone doesn't mean it's any less suitable on Ubuntu. I'm just trying to understand what the differences are in implementation, reliability, etc.
ZFS on FreeBSD and on Illumos has been used, abused, tested and stressed all the way to the Moon and beyond. Loads of people with loads of data over loads time.
This makes bugs appear, and get addressed, and eventually gives you confidence that nothing nasty remains uncorrected, and it won't eat your data.
ZFS on Linux, due to the unfortunate licensing situation, is considerably less tested and thus scary, data-eating bugs are at least a bit more likely to exist.
> On the other hand Linux is far more popular so I wouldn't be surprised if ZFS on Linux already clocks more hours of usage than on BSD.
Yes and no. Few people, even among the Linux crowd, are aware it exists or feel like trusting it with important data sets/jobs. Those few are the most likely to be equally at ease running FreeBSD or Illumos (or Solaris), where the damn thing is known to work really well.
ZFS-on-Linux has not been mainstream (or existed) for as long as ZFS has been integrated inside FreeBSD. illumos is the free software fork of OpenSolaris -- so it's the repo of record for modern ZFS development.
ZoL uses an emulation layer to make Linux look more like something with Unix ancestry, ancestry that BSD and Solaris share. The biggest difference I believe is in memory management / fs cache integration.
In practice this means that ZoL may require more memory to be stable, or may be less stable depending on configuration if low in memory.
I've run ZFS on Solaris (by way of Nexenta) and Linux since 2008 or so. I haven't seen much reliability difference in practice. I've had fewer hiccups streaming video from Linux though.
Yeah I was like uhhhh what? Illumos and illumos based OS'es and FreeBSD are the only games in town where ZFS is a first class citizen. Ill chalk it up to extreme bias.
Yeah, many people don't consider FreeBSD because of silly reasons… :(
but mentioning Solaris that "requires licensing" and not illumos?? illumos is the new Solaris, the old Solaris devs are working on illumos! Oracle Solaris is irrelevant.
