I'm contributing to a linker written in ~OCaml [1], and now I understand writing systems code in OCaml is really a bad idea. It makes things more complicated when you really don't want them to be. As James Mickens says [2]:
>You can’t just place a LISP book on top of an x86 chip and hope that the hardware learns about lambda calculus by osmosis.
[1]: https://github.com/rems-project/linksem [2]: http://scholar.harvard.edu/files/mickens/files/thenightwatch...
Can you provide an example of the things, OCaml makes more complicated and explain which languages you compare it to? I guess it's quite different in comparison to C, C++ or maybe Rust than to other garbage-collected 'system programming' languages like Go?
Here are a few examples:
- Inferred types: you're constantly wondering "what is the type of this variable?". Type errors occur far from where the real type error is. Annotating is complicated because you're often dealing with complicated types (with nested tuples for instance).
- Using higher-order functions: often you'll see a function with nested functions in it [1], or some functional list processing function like "unzip" being used [2]. This increases the code complexity a lot ("what does unzip do again?").
- Using monads: yet another distraction from what's really happening. It forces you to write nested functions and doesn't play well with conditions [3]. Also various people use monadic operators way weirder than >>= and all of this becomes really hard to read.
- Using functional-friendly data structures: the linker is using immutable lists of optional bytes to represent memory images, because this is idiomatic in OCaml. However this is slow and inefficient as hell. Really, I can't load a 60MiB binary because the linker runs out of memory and is killed by the kernel on machine with 6GiB of RAM. Also, just linking hello-world takes ages.
- Designing functional loops: when the loop you want isn't in your stdlib, you first need to create a recursive function, and then make sure all recursive calls are tail calls otherwise you blow up your stack. The result is usually not very fun to read.
tl;dr: the C language is small, the OCaml language is not, and when you're dealing with complicated stuff you really don't _need_ more complexity from the language itself. James Mickens explains this (among other things) in his essay.
[1]: https://github.com/rems-project/linksem/blob/master/src/link... [2]: https://github.com/rems-project/linksem/blob/master/src/link... [3]: https://github.com/emersion/linksem/blob/54c3a8430e621198653...
(Note: set the syntax highlighting to "OCaml" when trying to read this source code)
EDIT: list formatting EDIT 2: grammar
The code you link to is pretty bad and I really cannot understand why.
[1] OCaml does the currying for you. Why would this ever need to be done?
[2] I have never even thought of putting a let statement in a list before let alone seen one in the wild. Also that looks like a list comprehension, so there is some kind of non-standard syntax going on there.
[3] Monads are a pretty good fit for some things, like parsing.
> the linker is using immutable lists of optional bytes to represent memory images
OCaml has support for bytes in the Bytes module. How do you deal with optional bytes in C? Well, one way is to have a mask array. You could also have a bitmask. You can do the same in OCaml and memory usage will be similar.
I've never worked on anything OCaml that I didn't start myself. However I've dived into mature code-bases including the OCaml compiler and they are generally readable and understandable. The examples you highlight, even if unrepresentative of the code-base, are very difficult to comprehend.
I think a lot of your problems do seem to stem from this code-base and some of the mysterious things look to me like Haskellisms and attempts to make OCaml into something it's not. Sadly there is a lot of OCaml like this out there. Batteries and Core really haven't helped in this regard.
There is definitely an element of understanding OCaml. There isn't much of it to learn, but you need to approach it from the right direction and that can take time. Learn it without Core or Batteries.
Even so, I would be very strongly tempted to clean up that code if at all possible.
I didn't link the code to say "hey look this code is bad", my intent was just to illustrate that functions inside functions indeed do happen, and unzip is also used, etc.
>OCaml has support for bytes in the Bytes module. How do you deal with optional bytes in C? Well, one way is to have a mask array. You could also have a bitmask. You can do the same in OCaml and memory usage will be similar.
Yeah, that one is probably more because the original authors have written this code in their perfect mathematical world. But this is also one thing I don't like about systems functional programming: it forces you to use abstractions that don't always make sense.
Take a look at the ideal mathematical (verified) C compiler written in Coq. I find it neat: