What does HackerNews think of oil?

I agree, and that was out of the motivations for https://www.oilshell.org/

It runs your bash scripts but you can also opt into correct error handling. The simple invariant is that it doesn't lose an exit code, and non-zero is fatal by default.

Oil 0.10.0 - Can Unix Shell Error Handling Be Fixed Once and For All?

https://www.oilshell.org/blog/2022/05/release-0.10.0.html

This took quite awhile, and I became aware of more error handling gotchas than I knew about when starting the project:

https://www.oilshell.org/release/latest/doc/error-handling.h...

e.g. it's impossible in bash to even see the error status of a process sub like diff <(sort left.txt) <(sort OOPS)

If you have bash scripts that you don't want to rewrite, try

1) run them with OSH

2) Add shopt --set oil:upgrade at the top to get the error handling fixes.

Tell me what happens :) https://github.com/oilshell/oil

I spent a long time on that, but the post didn't get read much. I think it's because it takes a lot of bash experience to even understand what the problem is.

Sure you can check it out here: https://github.com/oilshell/oil

    $ wc -l NINJA* */NINJA*
       64 NINJA_config.py
      ...
      260 cpp/NINJA-steps.sh
      361 cpp/NINJA_subgraph.py
      305 mycpp/NINJA-steps.sh
      605 mycpp/NINJA_subgraph.py
     1615 total

I borrowed the little Python library that Ninja itself uses to bootstrap its Ninja files.

There are a few missing things like some textual code generation, but overall it should make sense.

It's structured a bit like a lightweight Bazel -- each directory has a NINJA_subgraph.py file that specifies what to build. And then the "actions" are shell scripts in NINJA-steps.sh.

At the top of one file it shows the directory structure for variants, which is basically

    _bin/$compiler-$variant/foo_test

and

    _build/obj/$compiler-$variant/foo_test.o

The variants control the compile and link flags: https://github.com/oilshell/oil/blob/master/cpp/NINJA-steps....

Overall this is the best build system I've used :) My pet peeve is having to 'make clean' when changing the variant, and this avoids that, making everything available at once.

I'm sure it's reinventing some of the 120K lines of CMakeLists.txt that comes with CMake, but this way I don't have to learn and debug a wonky shell-ish language!

One downside of the pattern is that you probably don't want to invoke shell on Windows (which doesn't affect my project). I find that very useful though, so I'm not sure what would replace it for Windows projects. (I guess batch files, but I don't think they're powerful enough)

Bash is ugly and rife with pitfalls, but it can be incredibly productive for certain classes of problems if you know what you’re doing. Trying to use Python or Go to whack some external commands together feels very cumbersome to me.

For pure programming, “real” languages are absolulely preferable in almost every way, but all of them fail when it comes to running external programs and redirecting their output in a way that doesn’t make me pull my hair out. As a heavy user of both “real” programming languages and shell scripting languages, I’m left craving something that brings the best of both worlds.

There are a number of newer shell projects in this vein that are very exciting, like Oil [0], Elvish [1], and Xonsh [2]. I was also hopeful that Neugram [3] would go somewhere, but the project seems to have died out. While many people cite Bash’s lack of portability as a reason not to use it, I find Bash to be very portable for my use cases and avoid using these newer shells for their lack of portability. Maybe one day we can have nice things.

[0]: https://github.com/oilshell/oil

[1]: https://github.com/elves/elvish

[2]: https://github.com/xonsh/xonsh

[3]: https://github.com/neugram/ng

That's the point of Oil:

https://github.com/oilshell/oil

A new Unix shell. Our upgrade path from bash to a better language.

It runs thousands of lines of existing shell scripts unmodified, but there's also a brand new language that will be familiar to Python or JS programmers. See http://www.oilshell.org/blog/2020/01/simplest-explanation.ht...

It works and you can try it now. The downside is that the implementation is more like a slow prototype, but that's being addressed, and the release I made today has stats about the C++ version (blog post forthcoming).

Yes, Oil (https://oilshell.org/) now has nullglob on when you run bin/oil instead of bin/osh.

So you get this:

    oil$ find . -name *.jpg
    find: missing argument to `-name'

    oil$ find . -name '*.jpg'
    (works)

Details: I had it on when you set 'shopt -s strict:all', but I neglected to turn it on for 'oil:basic' and 'oil:all'. Those are oil-specific option groups so you don't have to remember all the option names.

But I just fixed that and it will be out with the next Oil release.

https://github.com/oilshell/oil/commit/ddac119254f9a7045dca7...

If anyone wants to add more strictness options to Oil to avoid these types of mistakes, let us know! (e.g. on https://github.com/oilshell/oil, or there's more contact info on the home page).

If you have some realistic shell benchmarks, please file a bug and link the files:

https://github.com/oilshell/oil

Oil isn't fast at the moment, but I'm currently optimizing it (auto-translating some code to C++, and eventually hand-optimizing other parts.) I run benchmarks on every release, so speed is definitely a goal.

I haven't seen that many shell scripts that are "CPU bound" (as opposed to waiting for processes or I/O). (Although now I recall a shell-script-library bundler project that was.)

Of course, one big problem with shell is that you often have to spawn an external process like sed or awk to do string manipulation that's just a library call in Python. That could easily be a three order-of-magnitude difference -- e.g. milliseconds vs. microseconds.

Sure, I agree, for specific tasks. Just like a hammer is a better tool than a screw driver for putting nails in a block of wood.

Thats my point. They're tools not lifestyle choices. Learn your tools, know your tools, don't be afraid of other people's tools.

They all, usually, have some sort of value.

I might as well note this too:

I am speaking in broad strokes, you're going to be able to pinhole this as much as you want because of that. I'm simply suggesting that if your immediate reaction to how another organization uses these tools to replace A with B at a certain level is "this is clearly inferior" instead of "well, what can I learn from this" well, there's the problem.

I used to think programmers/computer scientists where open minded people. Then I realized as a whole it tends to be we're only open minded if its within the domain we chose to be most active in

Still far and away better than any other folks I've met, of course, but this is really a community culture problem, which is why I find it so counter productive.

Alternatives to bash? Many do exist. zsh is bash compatible but does implement its own tools and subsets.

If you're looking for something that removes the bash syntax there is powershell, which does run on Linux/macOS now as well:

https://github.com/PowerShell/PowerShell

Then of course, there is oil shell:

https://github.com/oilshell/oil

which is more like a subset of bash (and somewhat close to the idea of stripping away all the legacy stuff)

Then there is xonsh: lets you use python as a basis for most everything

http://xon.sh/

The stalwart fish:

https://fishshell.com/

fish shell has alot of niceties and a more object oriented syntax (kind of)

and for the daring there is ergonomica: https://ergonomica.readthedocs.io/en/latest/

which has a lisp like syntax

as does https://github.com/dundalek/closh which uses clojure and its syntax

and i'm sure there are more.

I recommend using plain shell scripts. Each .PHONY target is a simple shell function, and then the last line of the script is "$@", which means run function $1 with parameters $2 to $n.

So instead of 'make test', I just use './test.sh all', or './test.sh foo'. The test script can invoke make.

The idea is that 'dataflow' parts are done in Make, and the imperative parts are done in shell. This works out fairly well if you're disciplined about it. The only point of Makefiles is to support quick incremental builds. If there's no incrementality, then you might as well use shell. (Note: whenever you use Make, you're using shell by definition. There's no possibility of only using Make. So I take Make out of the picture where it's not necessary.)

For example, all the instructions here are of the form 'foo.sh ACTION':

https://github.com/oilshell/oil/wiki/Contributing

Pretty much every shell script in the repo uses "the argv dispatch pattern"... I've been meaning to write a blog post about that pattern, i.e. using functions with the last line as "$@".

https://github.com/oilshell/oil

I was going to write that I don't have a good pointer, but actually I think the idea is basically the "onion architecture" or "clean architecture". I don't recall if I watched all of this talk, but in general I like Brandon Rhodes' PyCon talks :

https://www.youtube.com/watch?v=DJtef410XaM

http://rhodesmill.org/brandon/slides/2014-07-pyohio/clean-ar...

I don't come from a Java background, but I think that many of these ideas originated in the Java community. They came to Python later (if at all), and it's a little funny to see Go struggling with the same thing.

I think I came at it from more of a functional programming perspective, and the style gradually converged with what people were thinking about OOP. I'm not actually of all the details about "clean architecture" and "onion architecture", but they are definitely related to dependency injection.

Basically all your effects and state can be instantiated on the "outside" in main() and passed INWARD. But I don't believe it necessarily has to be pure. I encapsulate all my state, but I don't encapsulate all my side effects, like printing to the screen.

Some people appeared to like this comment, which is somewhat related:

https://news.ycombinator.com/item?id=11841893

I'm not sure how much this will help, but here is 12K+ lines of code I wrote with zero globals:

https://github.com/oilshell/oil

The main directories to look at are core/ and osh/. The main program imports a lot of stuff and wires it together:

https://github.com/oilshell/oil/blob/master/bin/oil.py

(There is a bunch of surface messiness, but the architecture is sound.) I've also written even larger programs with zero globals.

Also, I believe that web frameworks are some of the worst offenders in terms of preventing you from using this style. I don't think I've seen a single web framework that made the handlers easily testable.

I think you may be fighting against the grain in some parts of the Python world using this style... I can do it in certain projects because I'm writing most of the code from scratch, and forking some dependencies.

I don't think there are that many languages that are good for writing a shell. There seem to be a few shells written in Go, but Go only reluctantly supports fork(), because it interacts with its threaded runtime. Python is actually closer to what you want than Go, but I think it will end up having problems with signal handling, because the Python interpreter does its own signal handling. The prototype of my shell is written in Python [1].

Garbage collection almost always interacts poorly with signal handling. You can't interrupt a garbage collector at an arbitrary point.

I wanted to bootstrap my shell [1] with Lisp -- and I hacked on femtolisp because Julia is bootstrapped in exactly this manner. And for awhile I thought about using an OCaml front end.

But in the end I settled on C++ (writing 3K lines of code, which I plan to return to after my prototype is done.) C++ lets you do fork(), exec() and handle signals exactly as in C, but it actually has useful string abstractions!

C++ has a lot of problems, but it appears to be the best tool for this job.

[1] https://github.com/oilshell/oil/