I've been looking around for a scripting language that:
- has a cli interpreter
- is a "general purpose" language (yes, awk is touring complete but no way I'm using that except for manipulating text)
- allows to write in a functional style (ie. it has functions like map, fold, etc and allows to pass functions around as arguments)
- has a small disk footprint
- has decent documentation (doesn't need to be great: I can figure out most things, but I don't want to have to look at the interpter source code to do so)
- has a simple/straightforward setup (ideally, it should be a single executable that I can just copy to a remote system, use to run a script and then delete)
Do you know of something that would fit the bill?
Here's a use case (the one I run into today, but this is a recurring thing for me).
For my homelab I need (well, want) to generate a luhn mod n check digit (it's for my provisioning scripts to generate synchting device ids from their certificates).
I couldn't find ready-made utilities for this and I might actually need might a variation of the "official" algorithm (IIUC syncthing had a bug in their initial implementation and decided to run with it).
I don't have python (or even bash) available in all my systems, and so my goto language for script is usually sh (yes, posix sh), which in all honestly is quite frustrating for manipulating data.
Why aren't python and bash be available in all your systems? Which languages would be?
I would've recommended python, otherwise perl or Haskell (maybe Haskell's too big) or something, but now I'm worried that whatever reason makes python undoable also makes perl etc. undoable
Mate, I came on here to post Haskell as a semi-ironic 'joke' and it's included in the top comment. You've made my day.
I mean, it's a functional programming language and an incredibly good one. But it probably has a far too big footprint for their use-case...
Why aren’t python and bash be available in all your systems?
Among others, I run stuff on alpine and openwrt.
I don't need to run these scripts everywhere (strictly speaking, I don't need the homlab at all), but I was wondering if there's something that I can adopt as a default goto solution without having to worry about how each system is packaged/configured.
As for python, I doubt the full version would fit in my router plus as said I don't want to deal with libraries/virtualenvs/... and (in the future) with which distro comes with python3 vs pyton4 (2 vs 3 was enough). Openwrt does have smaller python packages, but then I would be using different implementations on different systems: again something I'd rather not deal with.
As for perl, it would be small enough, but I find it a bit archaic/esoteric (prejudice, I know), plus again I don't want to deal with how every distro decides to package the different things (eg. openwrt has some 40+ packages for perl - if I were doing serious development that would be ok, but I don't want to worry about that for just some scripts).
Sounds like you want MicroPython. It’s definitely available on OpenWrt and AlpineLinux and has a very small footprint.
If you don’t like Python, have a look at Lua/luajit.
You've defined yourself into an impossible bind: you want something extremely portable, universal but with a small disk imprint, and you want it to be general purpose and versatile.
The problem is that to be universal and general purpose, you need a lot of libraries to interact with whatever type of systems you might have it on (and the peculiarities of each), and you need libraries that do whatever type of interactions with those systems that you specify.
E.g. under-the-hood, python's
open("<filename>", 'r')is a systemcall to the kernel. But is that Linux? BSD? Windows NT? Android? Mach?What if you want your script to run a CLI command in a subshell? Should it call "cmd"? or "sh"? or "powershell"? Okay, okay, now all you need it to do is show the contents of a file... But is the command "cat" or "type" or "Get-FileContents"?
Or maybe you want to do more than simple read/write to files and string operations. Want to have graphics? That's a library. Want serialization for data? That's a library. Want to read from spreadsheets? That's a library. Want to parse XML? That's a library.
So you're looking at a single binary that's several GBs in size, either as a standalone or a self-extracting installer.
Okay, maybe you'll only ever need a small subset of libraries (basic arithmetic, string manipulation, and file ops, all on standard glibc gnu systems ofc), so it's not really "general purpose" anymore. So you find one that's small, but it doesn't completely fit your use case (for example, it can't parse uci config files); you find another that does what you need it to, but also way too much and has a huge footprint; you find that perfect medium and it has a small, niche userbase... so the documentation is meager and it's not easy to learn.
At this point you realize that any language that's both easy to learn and powerful enough to manage all instances of some vague notion of "computer" will necessarily evolve to being general purpose. And being general purpose requires dependencies. And dependencies reduce portability.
At this point your options are: make your own language and interpreter that does exactly what you want and nothing more (so all the dependencies can be compiled in), or decide which criteria you are willing to compromise on.
Is compiling scripts an option? Aka compiling them in C, C++, Rust, whatever for your router on another machine, and copying and executing those binaries on your router?
luajitis small, fast(well, it can jit), and has a small but complete standard library and can do FFI pretty easily, should be ideal for most homelab usecaseldd $(which luajit) linux-vdso.so.1 (0x00007ffee9dc7000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fb4db618000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fb4db613000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fb4db5f3000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb4db3ca000) /lib64/ld-linux-x86-64.so.2 (0x00007fb4db799000)I honestly love Powershell, but haven't tried the Linux version yet. I only use Bash on linux but it has a load of odd quirks that make it unpleasant to use imo. Can't comment on anything else.
Perl is already installed on most linux machines and unless you start delving into module usage, you won't need to install anything else.
Python is more fashionable, but needs installing on the host and environments can get complicated. I don't think it scales as well as Perl, if that's a concern of yours.
I've looked into this a lot actually. There see many options. I'll highlight the pros and cons of each option.
Lua: extremely lightweight, but standard library is lacking, and doesn't include stuff like map or fold. But that would be easy to fix.
Python: thicc standard library, but is not lightweight by any means. There are modifications made to be more shell like, such as xonsh
Rash: based on scheme, very much functional but if you're not used to lisp style, might take a bit to get used to it. This is actually my favorite option. It has a cli interpreter, and really pleasant to use. Cons is... Well it's not very common
You can honestly use any language. Even most compiled languages have a way to run immediately.
Wanted to say this too but it really depends on what you're using it for.
That does seem to be just one, maybe two small files, and no dependencies. And a built in map() function.
JavaScript through Node.js, or TypeScript through Deno if you like typed languages. They both check all your boxes (just check the size of the executables to make sure that it's what you would consider "small footprint").
Both languages and runtimes are quite popular, so you will find any answers on StackOverflow.
They are both single-executable with little dependencies, and Deno can also compile your scripts to self-contained executables.
As a bonus, both support the vast and extensive NPM package repository where you can find all sort of libraries for even the most complex tasks.
And they work with your favourite IDE or editor, not just syntax highlighting, but also contextual suggestions.
Installing node uses some 60MB (according to zypper on my current desktop). I'd rather have something small and possibly that consists of a single executable.
As a bonus, both support the vast and extensive NPM package repository
That's not necessarily a feature :) Package repos are great if you are a developer (I am one) working primarily with that language, but are frustrating if you just want to run things.
I thought so. Although almost nothing for modern standards, 60MB is not exactly tiny. Sorry about that.
On a different note, a repository is always a good thing imho. If you'd rather not have to worry about the dependency-pull step you can always include the dependencies with your sources, or just limit your code to using features included in the standard library.
A scripting language written in Rust would certainly fulfill you requirement of only needing to copy one file since they are always statically linked and you can even statically compile against musl so it will work on any Linux system without needing a correct libc. Maybe check out rhai.
Elixir checks most of those boxes. If you want a good functional scriptibg language, Elixir soynds like the go to. Some lisp language like guile should also be sufficient, and probably have a lighter footprint.
This requirement stands out though:
has a simple/straightforward setup (ideally, it should be a single executable that I can just copy to a remote system, use to run a script and then delete)
Thats basically what ansible does. If you plan on doing this to multiple machines you should just use ansible. Also how do you plan on ensuring the scripting interpreter is installed on the machines?
Elixir is quite big (yeah, it's certainly smaller than something like java... sorry for not specifying what I mean by "small disk footprint").
Thats basically what ansible does. Thats basically what ansible does. If you plan on doing this to multiple machines you should just use ansible.
Ansible requires python on the target machine (or a lot of extra-hacky workarounds) so... I could just use python myself :)
BTW getting ansible to do anything besides the very straightforward usecases it was meant for is a huge pain (even a simple if/else is a pain) and it's also super-slow, so I hate it passionately.
Also how do you plan on ensuring the scripting interpreter is installed on the machines?
Ideally I'd just copy the interpreter over via ssh when needed (or install it via the local package manager, if it's available as a package)
If python is too big for you and you're dealing with heterogeneous systems then you're probably stuck with
shas the lowest common denominator between those systems. I'm not aware of any scripting languages that are so portable you can simply install them with one file over scp.Alternate route is to abandon a scripting interpreter completely and compile a static binary in something like Go and deploy the binary.
There was also some "compile to bash" programming languages that I've sneered at because I couldn't think of a use case but this might be one.
Tried bash, Make, and awk/sed. All hit brick walls. Finally landed on pyinvoke. Two dependencies to install on any new machine. Never had problems. Also, easy to debug and modify as projects evolve.
(Chicken) Scheme.
Schemes have one of the best and most interactive interpreters Is general purpose, allows functional, procedural and OO programming, small disk size and compilable to native executables, Throughout documented and supplemented by years of research, simple setup.
It also is CGI compatible, if necessary.
I don't know if it matches your desire for easy install of small disk space, but it might make up for it in other arenas - Ruby is my new-found love when making simple scripts. Being able to mostly emulate the shell integration that bash has by just using backticks to call a shell command is the killer feature in my book.
