Hello all,
I am a data center engineer of about 8 years now. I've spent the last 3 years or so slowly learning Python(I say slowly not because of my effort, but because learning Python was actually very difficult for me.) I am not an expert in any way shape or form, I understand the concepts of OOP, inheritance, classes, functions, methods, etc and I have found that the python documentation that can be found within the language is usually enough for me to be able to write the programs that I want to write. Very rarely have I had to write programs that have to bypass the GIL, but occasionally, I have created threadpools for applications that are not I/O intensive. What I'm saying is, for most things that I create, performance is enough with Python.
However, I have been inspired by how much love Rust is getting from the people who use Rust. I have tried to find some books for using Rust for network automation and unfortunately I have not been able to find any reputable books.
Most of the "automation" work that I do involves parsing data with regex, restructuring the data, converting the data into a modeled format and transforming something with that data. Does anyone have any common use cases for Rust that might interest me? Has anyone used Rust for network automation tools? With familiarity, can Rust's intuitiveness match Python's "from idea to deployment" speed? Or should I only learn Rust if I intend to create applications that need tight performance?
With familiarity, can Rust’s intuitiveness match Python’s “from idea to deployment” speed?
Likely not at the start. Rust can take some time to learn to use it effectively it is not the fastest at throwing shit together quickly.
Or should I only learn Rust if I intend to create applications that need tight performance?
Also no. IMO rusts performance is only a nice by product of the language, yeah it encourages people to try it out, but they don't stay for the speed. They stay for the tooling and the feeling that once it compiles it will likely just work they way you intended. Rust forces you to think more about correctness and edge cases of your code - which does slow down initial idea to working prototype a bit. But IMO it quickly pays back dividends when you get something into production and it just works with no random crashing in the middle of the night.
It also makes refactoring a joy to do, where I hate refactoring in languages like python as you never know what you might have broken - likely something that you will find out only after you have deployed to production. Instead the compiler catches basically every thing that you missed before it will even let you run the code - so those edge cases are taken care of when you are developing, not after it fails in production.
I also find it is very nice with data processing/transformation as it lets you use functional coding styles which tend to lean towards clearer/easier to read series of data transforms.
If you want to learn it I would recomend starting out with the offical book, but you might also find zero to production or datawithrust interesting reads as well.
Context: I am an embedded software engineer. I write a lot of low level code that runs on microprocessors or in OS kernels, as well as networking applications and other things. I write a lot of C, I write some Rust, I write Elixir if I possibly can, I write a lot of Python (I hate C++ with a passion).
I don't think you want Rust. Python is unbeatable on "idea to deployment" speed. Python's downsides:
- Painful packaging/distribution if you want to get a load of people who don't have Python installed to run your thing (e.g the GUI program we currently maintain for talking to our hardware)
- Performance under some circumstances. There are some things that are not quick in Python. They're not always the things you expect because Python actually drops down to C modules for a lot of the number crunching that you might do. E.g. for ML you are basically using Python to plug a load of bits of fast C code together
Rust is good when you need at least one of:
- High speed
- Control over use of memory
- Low level systems programming (drivers etc.)
- Can't cope with a Garbage Collector
- Compiling to a microcontroller
If you're doing one of those and so have become expert in Rust, then it is actually excellent for a lot of other things. E.g. you might build your data processor in it, and then distribution is easy because it's just a single binary.
One option you might look at is Go. You get a lot of performance, you get good parallelism if you need it, it's designed to be easy to learn, and it also compiles programs to a single binary for easy distribution.
One more note on learning Rust: what Rust does is front-load the pain. If you write something in another low-level "direct control of memory" language you can often get something going much more easily than Rust because you don't have to "fight the borrow checker" - it'll just let you do what you want. In Rust, you need to learn how all the ownership stuff works and what types to use to keep the compiler happy.
But then as your project grows, or does a more unusual thing, or is just handed over to someone who didn't know the original design idea, Rust begins to shine more and more. Your C/C++/whatever program might start randomly crashing because there's a case where your pointer arithmetic doesn't work, or it has a security hole because it's possible to make a buffer overrun. But in Rust, the compiler has already made you prove that none of that is possible in your program.
So you pay a cost at the start (both at the start of learning, and at the start of getting your program going) but then over time Rust gives you a good return on that investment.
I think you're right. Despite my bandwagon fandom, I Believe Go is more appropriate for my needs. I think a compiled language with more intuitive parallelism is what I need
When you compare "idea to deployment" speed, a dynamic language will always win. However, much of this win is due to a dynamic language will let you deploy with a lot of bugs. So, you will then have to spend lot of time fixing production issues. Rust will force you to fix most of these issues before you can deploy, hence it feels slower in this aspect. I previously worked for 10 years with a huge perl code base, and I trade the deployment speed for stability in production any time.
Rust is a great language but it really has a lot of up-front costs. Whether you are learning it for the first time, or starting a new project, both. Python is always going to be faster "from idea to deployment."
I think like the other person said, start with the Rust book. It really is a perfectly good introduction. But what I think you'll find is if you want to be productive with Rust you will need to get the ground rules down or else you will be constantly tripping on the borrow checker and ownership rules. If you think you are getting somewhere with the book, try rewriting something you've done in Python. If that works out, great! If not, it's OK to accept that Rust might not be worth your time.
"intuitive" is extremely subjective, and based on your past experiences. I've coded in C++ for years, and some Python, too and was able to grasp many Rust concepts very quickly, while for others I struggled (and still am). I'd say that if you are looking for "intuitive", Rust ain't it. It's a system language, so it requires planning, it's definitely not the ideal language to slap a prototype quickly together, expecially as a beginner.
Most of the “automation” work that I do involves parsing data with regex, restructuring the data, converting the data into a modeled format and transforming something with that data.
These are the highlights of rust - regexes (the regex library that powers ripgrep), serialization/deserialization (serde, nom, pest, capnproto, etc), data manipulation (too many. perhaps apache arrow deserves a mention), etc. The language is designed for this sort of stuff.
Has anyone used Rust for network automation tools?
I'm in the same boat as you. I use python to manage an LXD cluster. I didn't choose Rust because Python has an API binding, but Rust didn't. It wouldn't have been too hard, considering that the API is mostly just JSON.
With familiarity, can Rust’s intuitiveness match Python’s “from idea to deployment” speed?
Unfortunately no! (in my personal experience). I have been using Rust for a decade (since before 1.0) and am more or less comfortable with it. I no longer fight with the compiler as most beginners do. Still, the compiler complains a lot and you have to think about why. This is not bad, IMO - all that thinking make you design better programs. There is always a sweet satisfaction of writing excellent code when you finish a Rust program - probably the reason why Rust developers love it so much. But you have to sacrifice some speed for that.
Intuitiveness is also subjective. My introduction to programming and computers was from the hardware side - from ALUs, DRAM, hardware pipelining etc. Rust's rules are very well defined, but is obscurely related to real problems you encounter in the hardware. In other words, Rust rules look confusing at first, but it makes sense once you see the error messages. You know what problem could have happened on the hardware if Rust didn't stop you. It's as if Rust's simple rules magically cover the problems you dread on the hardware.
All this means that Rust is easy to grasp for people who are already using C - OS, game engine, web engine and embedded system developers. For others who are mostly used to GC languages, it's an uphill battle with the compiler, until some day it clicks somehow. With Rust, it's very important to start right away with the (c lang) memory model and Rust's aliasing rules.
Or should I only learn Rust if I intend to create applications that need tight performance?
I wouldn't say no to learning a new language. But I always recommend a few languages because they introduce a radically different idea. Rust is one of them. My rant above probably gave you an idea why I would recommend Rust. The first is that Rust forces you to design programs properly - even though the rules are meant only to enforce memory safety. Second is that Rust gets you very close to the soul of the hardware. Python and even Go has some opaque layers in between you and the hardware. With Rust, you'll get every opportunity to get the best the hardware can offer. You get the same in C and C++ - but they also allow you to screw up.
I use Rust for even trivial tasks where a shell script would suffice. Besides the language features, Rust has excellent tooling and a rapidly growing library ecosystem. In many ways, it's easier to package Rust programs than Python ones. So Rust will work for your use case. But if development speed is more important to you than performance, then Go (similar to Python), Nim (much more similar to Python) and Zig are probably better choices. Repeating something here - Go is designed for the web. Numerous web backends are already on Go. And almost the entirety of Kubernetes is written in Go.
Thank you, great response. Based on your opinion and another, I believe I will focus on learning Go. I mostly need the benefits of compiled languages in order to easily distribute, as well as easier parallelism, as I've always found that to be a pain in the ass with Python. Not sure if Threading, Concurrency, or Asyncio are the "best" way to handle threading of non CPU intensive tasks, such as sending a request and waiting for a response. But I know that it seems since Python has taken so long to attempt to allow easier bypassing of the GIL, you get a lot of decent ways of doing something, but no great ways.
Compared to C or C++ it is miles ahead, but higher-level programming languages can be more intuitive. Kotlin IMO is much more intuitive than Rust.
While Rust code may look as simple as other high level languages, it takes much more effort to get there. It can feel like the type system is fighting against you, rather than being there to guide you toward a correct answer. Thanks to Rust's macros, IDE support for Rust is also not as good, which contributes to that feeling.
Just adding that you can also do Python - Rust interop fairly easily with something like https://github.com/PyO3/pyo3
So you can gradually adopt Rust for some of the more performance demanding parts and call them from Python. You will get a massive increase just from a simple rewrite. Though you would have the additional burden of 2 build toolchains. But as others have said, Rust is not just about the performance.
I’m usually a little suspicious of a new fancy language - because the language is only a part of the equation. Does it have good tooling and does it have awesome libraries?
I had a preconception that Rust is strong as a language (formally well structured, low shoot-yourself-in-the-foot potential, consistent, predictable) and that the tooling seemed strong (debuggers, editors, code completion, help, test frameworks), but I’ve always thought that it would lag with libraries. I mean compared to something like Python (« Batteries included ») or java, surely it is not yet compatible, right?.
So I chose a few of the less main-stream libraries that I use regularly… and Lo and behold! They exist for Rust, including Couchbase, SQLite, ECDH, DiffMatch. I can’t vouch for the completeness of those libs, but the fact that everything I looked for existed… that’s impressive.
