It's kind of surprising to see that Rust is so rapidly accepted by wide range of developers.
Unfortunately I'm from DataScience field, so I cannot see much motivation to learn Rust, but I am considering learning it, because language itself seems exciting!
Is there anyone on HN who is from DataScience field like me and has learned Rust? It would be much appreciated if you could share the experience.
We developers love so much to learn computer stuff that we want to do it even when we don't need to.
I worked on data science, computational science, some system engineering and embedded projects. I tried Rust on all of them but it was valuable only for the third.
In data science the focus is more on speed of development and as much as Rust is more enjoyable to use than C++, it doesn't match scripting languages (namely Python) for the flexibility you get, the easiness with which you can adapt to changes and the lack of a need to focus too much on machine-related details. This is the least suited field for Rust between the one I mentioned, in my view.
Computational science needs to focus on algorithms and formulas, Rust can hide them a bit too much under "unwrap"s "iter"s and so on. Plus still no stellar library support and (my) experience with other tools (namely C + OpenMP, Julia, C++, numpy...) made me feel using Rust for that was unnecessary and slower.
When you are designing systems, however, that's where Rust is a complete game changer. Honestly I don't think C++ can stand a chance in its current level. Everything, from the package system, to the borrow checker, to the safe threading model, makes you pity your C++ ancestors for how hard they thought this kind of programming had to be. Rust makes it orders of magnitude more approachable, saner, and with better results too.
Embedded... it's nice but honestly all the dark arts of unsafe Rust are not standardized so it doesn't feel as future-proof as C already.
Rust still needs to do a lot of catching up in HPC, HFT, GUI, game engines, console SDKs, OS drivers on Apple and Microsoft OSes, Arduino, LLVM and GCC,...
Rust threading is only safe between threads accessing data structures in process own memory. It does nothing for shared resources using OS IPC, or external resources shared among threads.
It will certainly improve, as hopefully C++ will, even it never gets 100% as safe as Rust, the ecosystem has too much weight for decades to come.
> Rust threading is only safe between threads accessing data structures in process own memory.
"Only". Intra-process shared memory is by far the most important kind of shared memory. 80% of the time you see someone advocating for message-passing multi-process over shared memory multi-thread concurrency or parallelism, it's because shared mutable memory is incredibly hard to get right in just about every mainstream language except for Rust.
> It does nothing for shared resources using OS IPC, or external resources shared among threads.
Only partially true. The very same language constructs which make shared memory safe can also be used to create safe wrappers for other shared resources.
> Language constructs only help if there are no other applications accessing the same resources.
You're mistaken. Rust's language constructs can often help you write safe wrappers around inter-process communication and synchronization mechanisms. Of course, all bets are off if the other processes don't adhere to the agreed-upon IPC protocol, but that's not the point. The point is that Rust makes it easier to not accidentally screw up the protocol yourself.
So now you're arguing that Rust won't help because it doesn't fix the deficiencies of other languages? So what? At least you'll have fewer defects in those parts that do use Rust.
I feel like this was true a few years ago but things have changed a lot recently.
In my opinion Bevy may be one of the better ECS implementations I've seen, as ex-gamedev it's just very well thought through. It's never going to supplant Unity or Unreal but that's less of a language issue than just the scope of those codebases.
You're seeing a lot more of it showing up in OSes across Android, Linux and I've seen a fair bit of movement in the windows space as well.
As for IPC, C/C++ doesn't help you their either, that's more of a framework level concern. If anything the threading model helps when you're doing concurrent IPC dispatch(ex: MTA COM or equivalent high throughput IPC) and preventing data races.
Rust has amazing integration with Python through PyO3 [1] so see it like a safe alternative for high performance calculations and operations.
The rust data science ecosystem itself is starting to come together with projects like Polars [2] (Pandas alternative), nalgebra [3], Datafusion [4] and Ballista [5]. Of course nowhere near Python but that is hard to beat today.
Also dfdx [1], which is shaping up to become a great neural network library (though GPU support is still WIP, give it a couple months before production use).
Of course the ecosystem isn't as diverse as python yet, but if you're willing to call a bit back and forth between rust and python it's great.
> Is there anyone on HN who is from DataScience field like me and has learned Rust?
Yeah me. I’m data science/data engineering.
Wrote and debugged a lot of R and Python, fair bit of Spark too.
Learning Rust was fine. Not trivial, but definitely worthwhile. I work on more “data systems” than once-off scripts and analysis in my work, so the performance and correctness of Rust has proven more valuable to me than the flexibility and development speed promises of Python. The former lends itself to difficult to maintain, hacky, patchwork code, and the latter falls apart after the nth time you have to fix some Python code in prod that’s failed 3.5 hours into a process, because some random package has thrown some as-yet-unseen error.
There’s some interesting stuff happening in the data space in Rust-polars data-frame library gives you pandas like functionality, but with better performance and type checking, the data-fusion stuff is making strides as well. There isn’t a 1:1 replacement for all your ML libs, but it’s getting there!
Linfa and SmartCore are making strides on the sklearn-like space.
Would you use Rust to replace your ml stuff in Python at work tomorrow? No, probably not just at the moment. Is it worth learning because it’ll make you a better dev, and give you more options and tools the next time you have a non-standard problem to solve? Yes, definitely.
But: I'm not well versed in Python Pandas either, and my rust is better than my python (though Ruby is my daily language, so I'm very used to dynamic languages). The thing I miss most is the rapid REPL workflow in Python, where I work out the pipelines, filters, map-reduce and so forth, on the interactive REPL and then extract that into python scripts.
The polars scripts that I worked with, were small enough and the builds are incremental, therefore fast: milliseconds. But the build-step inbetween is somewhat clumsy when exploring data.
But the end-results are just so extremely cleaner, saner and especially faster that it certainly is worth it. I worked on a dataset where crunching it in python took upwards of 14 hours, and the polars under an hour. That feedback loop alone is worth days of fighting the borrow-checker.
And that immediate feedback by the compiler/checker "this column isn't guaranteed a String, it could be missing, could be unparsable; deal with this now" is priceless. Too often did I have some ETL run for 8 hours, and then choke on some column that I swore should've been UTF8. Fix it, and re-run. Wasting 8 hours. A type-checker in ETL/data flows is priceless.
Edit: to be clear: I am developer. Not datascience. But I do need to understand my users, my business, my market. Developer/devops today do need to crunch large datasets: at least I do.
I started learning Rust a few years ago, with the intention to use it for data science.
I ended up writing a book, "Data Analysis with Rust Notebooks", where I demonstrate using Rust + Jupyter Notebooks for tasks I'd normally use Python for. It's a good experience, although there are some additional limitations when using Rust as a notebook kernel.
I'm also in the data science field, and I'm currently teaching myself rust as a possible replacement for C and C++ for all those times I have to write something fast and novel and that I can call from python.
That's exactly why our Data Engineering team is also looking at Rust, but we're looking at Go, too. I have a hunch that Rust is going to be a harder sell to the broader team given its learning curve and that most of the team don't already have a compiled language in their tool belt. I'm quite open to either, though, as they'd both make my pipelines go vroom.
Rust has the upper hand for incrementally replacing Python modules. With PyO3[0], you can seamlessly replace existing Python code with native extensions written in Rust.
If your use case is to replace entire tools, and your team isn't familiar with static languages, then Go is probably the better option. There's no denying that it's much easier to learn.
I did. It's a very fun language to learn. If you're comfortable with the simplicity of Python, Rust would be too verbose. I'm personally fine and actually like Rust more. With the current adoption pace it should be competing with Python in the Data Science field soon.
I suppose if you're coming from Python, you could use Rust (with PyO3) instead of C/C++/Cython in something similar to the Numpy/ML lower-level implementations.
But I wish there were a language that solved the two-languages problem (fast vs. easy to work with). I find Rust quite verbose.
> But I wish there were a language that solved the two-languages problem (fast vs. easy to work with).
That is the motivation and purpose of Julia.
For some people's workflows it already seems to have solved the problem, but in general the "easy to work with" part is still in progress IMO. Some of it is the (in)famously annoying compilation delays, but a large part of it is just a tooling and ecosystem thing (both of those sides of the issue are being actively worked on).
The difference from C/C++/other low level languages is that it's not the language itself that's difficult to work with, it's the current tools and workflows that are suboptimal. The language is pretty well-designed and fun to work with, so there's hope that it can actually solve the two-language problem for a wider variety of devs and their needs.
> But I wish there were a language that solved the two-languages problem (fast vs. easy to work with). I find Rust quite verbose.
Rust itself can be made "easy to work with" at the cost of boilerplate. You end up with code that's littered with idiomatically superfluous uses of things like .clone(), interior mutability, Cow<>, perhaps even such exotic features as the Any trait for encoding "dynamic" behavior.
The flip side is that it's easier to refactor this code and make it more correct and performant; "simply" replace the boilerplate with proper Rustic code and deal with the resulting compile errors. It's a very viable strategy that has no direct equivalents in other languages, not even in C/C++.
Rust has a number of features (ergonomic and technical) and some really nice language design that permits writing quite “high-level” code. Personally, having a compiler and type system as powerful as Rusts makes it worthwhile alone.
This. Rust is quite pleasant to use and once some barriers are overcome quite productive. Not Python fast dev speed, but easily Go levels of productive for me. I rarely fight the borrow checker unless building tricky data structures. It will still stump me once in a while with lifetimes, but the trade off is so worth it to have memory safety guarantees and know there are no foot guns like C code would have. And if you are just cranking something out you can write code a little less idiomatically and of efficiently for quick scripts or smaller throwaway CLI tools. Rust prefers a more functional style but you can just as easily make it not so.
It's nowhere as ergonomic, for the general population, as the likes of Python. References alone add a level of bothersomeness to the language that dramatically restricts its appeal.
Rust has a great role to play, replacing C and C++ in all performance-critical and security-critical underpinnings of modern computation (from kernels to UI toolkits). But higher up the stack, where those concerns are outweighed by overall development costs, other languages will continue to rule; and if one's focus is there, learning Rust is probably a waste of time.
> higher up the stack, where those concerns are outweighed by overall development costs, other languages will continue to rule;
Eh, I've written and maintained data pipelines, and API layers for downstream teams in prod in Rust, and it was a much more pleasant and low-effort experience than either doing either in Python. I could have had a "complete program" written first in Python, sure, but that gain in velocity is rapidly overtaken by the hours spent debugging and fixing issues in prod, issues that in Rust, I've not typically had.
> if one's focus is there, learning Rust is probably a waste of time.
au contraire - I genuinely think that learning languages, especially ones that are _not_ similar to your day-to-day language is an incredibly valuable thing to do. Learn a Lisp, learn some Haskell, etc, it exposes you to ideas that you wouldn't have otherwise come across, and gives you extra skills and tools with which to solve problems better. Maybe not all those tools are always applicable, but when they are it's a force-multiplier.
You're not wrong. Ocaml, Haskell, etc are pretty slept on, and have some amazing features, and I 100% think people should be more open to using them.
In my space (data engineering and associated systems), Rust is a much more palatable and easier sell than an Ocaml or a Haskell.
> With faster compiler toolchains and REPL like tooling.
TBH I don't really miss a REPL, I know a lot of people love it, but between the type system, tooling like RA, and how Rust makes writing tests _so_ frictionless, I've not really had any need to touch a REPL in the last ~2 years. I've probably been abusing the frictionless test functionality to write little snippets to test out using a new library or tool, I certainly find it a lot manual work than the "type multi-line snippet into repl. Make a typo. Start it all over again. Discover that part x actually wants a y. Write it all over again. Repeat until correct. Painstaking trawl through repl history to find the set of code that actually does what we want and lift it into our actual code base" experience I have with reps-based development.
It is not rapidly and widely accepted. It is pushed by a small but very vocal minority for all the wrong reasons.
When you have to prove your intelligence level (or rather "street cred") in a competitive market you have to show that you can learn the newest complicated things quickly. Rust is pure cryptonite in such a world.
It has been adopted by most large tech companies and several high profile open source projects within ~5 years of it's stable release. It certainly has been rapidly and widely accepted.
For important projects too: Google is using it in Android, Microsoft is using it in Windows, Amazon is using it for Lambda, Meta is using it for their source control tool, Dropbox is using for their storage backend AND in their client apps, Cloudflare is using it for their core proxy, etc.
I only know this word from context. And it is actually written with "k". So what does it mean? (Context/my reasoning: If you have to prove that you are a smart programmer you can't stay away from learning Rust. Hence it is kryptonite.)
Unfortunately I'm from DataScience field, so I cannot see much motivation to learn Rust, but I am considering learning it, because language itself seems exciting!
Is there anyone on HN who is from DataScience field like me and has learned Rust? It would be much appreciated if you could share the experience.