One thing I ran into was setting up a Python project using poetry2nix. Mostly works great, but then you sometimes get inscrutable error messages. I had to copy this into a shell.nix file for reasons that aren't entirely clear to me (and I had to hunt it down from https://github.com/NixOS/nixpkgs myself instead of finding docs or a bug report):
astunparse = super.astunparse.overridePythonAttrs
(old: { buildInputs = old.buildInputs ++ [ self.wheel ]; });
What surprises me is that people tout RPM/Chef as easier replacements to Nix/NixOS. The assumption about those being "replacements" of each other is questionable because they address different needs. What's even more questionable is the part about RPM or Chef being easy. I have dealt with those two for years as part of my day job, and they're both far from easy. I still have troubles with them to this day. Documentation is far more lacking than Nix, and it's not because things are obvious. If you encounter any trouble, you almost always have to dive into the source code. Navigating the source code of RPM or Chef is a nightmare because the code is scattered across countless different repositories and it's impossible to guess where to look for to begin with. Choice of programming languages doen't help here too. Chef is written in Ruby, which has a tendency of making code ungreppable due to its metaprogramming features. RPM has its own macro syntax which works similar to the C preprocessor. It's very terse and full of gotchas. Navigating around Nix code is much, much easier in comparison. Everything you need is in a single repo, https://github.com/NixOS/nixpkgs, and written in a JSON-like language.
Many people also claim that Nix is hard because it requires a functional programming background. That too doesn't match my experience. Nix expressions are basically JSON with functions. Learning the syntax is a non-issue if you know JSON and some common programming language constructs like conditionals, functions, map, filter and reduce. The only functional programming exposure I had prior to Nix was LISP, in a university classroom. Yet I didn't experience problems learning the syntax. The hard part is learning how to actually utilize the language. The documentation and some grepping around of the Nixpkgs codebase helped a lot with this.
> Use typescript, jsx to create the simplest way to implement components and sub circuits. (I know this isn't going to happen)
I can see why not many people would be willing the learn a new language from JITX (stanza?), but why typescript/jsx instead of python?
> Use a cloud package manager so people can share thier sub circuits and ICs
Why not just a github repository? Kind of like https://github.com/NixOS/nixpkgs
It's not because of the nix language, It's not because of the CLI, it's because everything is scattered, you have to consult many places to find out how to do things with Nix, here is an example:
Usually, when I need a new complex program, like Steam, I first check the system-wide configuration [1], the wiki [2] and the package list [3], if I just want it on my user, I need to check if Home Manager has an option [4], if it doesn't, I can try using the "home.packages" option. Now, if I need to override something on the package, I need to remember how to do it with [5] [6] (while checking the source code for the package in parallel to find the options).
And then sometimes, on very rare occasions, I need to fine tune something with the nix language, so I need to check the builtins/lib docs [7], but some builtins are not there, so I need to either use nix-doc [8] or find the docs inside the code-bases [9] [10] (they are split between both repos)
For me, this is one of the main pain points of using Nix / NixOS that needs to be solved.
[0] - https://github.com/shiryel/nixos-dotfiles
[1] - https://search.nixos.org/options
[2] - https://nixos.wiki/wiki/Steam
[3] - https://search.nixos.org/packages
[4] - https://mipmip.github.io/home-manager-option-search/
[5] - https://nixos.org/manual/nixos/stable/#sec-customising-packa...
[6] - https://nixos.org/guides/nix-pills/nixpkgs-overriding-packag...
[7] - https://teu5us.github.io/nix-lib.html
[8] - https://github.com/lf-/nix-doc
For people new to it, I am trying to provide a quick glossary of terms here, as I understand them after about 2 years of using nix.
* nix: a language to create derivations and the interpreter/package-manager which provides the implementation of said language. It currently offers two command-line interfaces, the stable on with hyphenated commands like "nix-build", "nix-shell", etc. And the newer, "experimental" one which includes support for nix flakes and so on, without hyphens: nix build, nix shell, nix run, etc.
repo: https://github.com/nixos/nix
docs: https://nixos.org/manual/nix/stable/
repo: https://github.com/NixOS/nixpkgs docs: https://nixos.org/manual/nixpkgs/stable/ docs nixos: https://nixos.org/manual/nixos/stable/
This tooling includes higher-level helpers for language-/environment-specific packaging, like "buildGoModule", "buildRustPackage" and so on, as well as e.g. tooling to run integration tests in a whole cluster of inter-connected linux VMs!
Packages which are submitted to nixpkgs must fulfill certain criteria, such as not using "IFD" (input-from-derivation, to simplify: "letting nix evaluate nix-code which was generated by another deriviation/"nix package".
nixpkgs is alive and well with lots of daily contribution and an everlasting effort to keep Hydra, the nix-specific CI/CD system and public binary caches up to date and responsive. Thanks to all maintainers & contributors!
* flakes are an approach to standardize a way to package nix code outside of nixpkgs but to still keep it re-usable. They are still "experimental" as the details are figured out, but nevertheless used in production. There are some frame-works to keep boilerplate low, like "flake-utils", "flake-parts" and others, as well as e.g. deployment tools like "colmena" and "deploy-rs" and re-usable helpers for system-configuration like e.g. https://github.com/nix-community/impermanence
There's lots of other stuff in the community, things like home-manager, direnv + flakes and devshells changed my workflow fundamentally to the better since I've switched. If you got the time and are still interested, join us on matrix or elsewhere :) https://github.com/nix-community/awesome-nix
In Bazel you have to do a lot of that work yourself. Google has essentially ~unlimited manpower to do this for the things in their third_party tree, but for any other size of organisation this is not the case and people resort to all sorts of ugly hacks.
Depending on your needs these different types of projects can also coexist with each other in Nix. To use an example from my work, we have a Nix-native build system for Go[1] and code using this[2] co-exists with code that just uses the standard Go build system[3]. Both methods end up being addressable on the same abstraction level, meaning that they both turn into equivalent build targets (you can see both in a full build[4] of our repo).
And for what it's worth, some of the things Bazel gets extremely right (such as having a straightforward mapping from code location to build target, universal build command) are pretty easy to do in Nix (see readTree[5], magrathea[6]).
[0]: https://github.com/NixOS/nixpkgs
[1]: https://code.tvl.fyi/about/nix/buildGo
[2]: https://cs.tvl.fyi/depot/-/blob/ops/besadii/default.nix
[3]: https://cs.tvl.fyi/depot/-/blob/third_party/gerrit-queue/def...
[4]: https://buildkite.com/tvl/depot/builds/13067
[5]: https://cs.tvl.fyi/depot/-/blob/nix/readTree/README.md
[6]: https://cs.tvl.fyi/depot@f0e6d3498d6c0f905977ea9432c311b6808...
Nix provides a level of flexibility that other solutions simply do not offer. If a package in Nix doesn't fit your requirements for whatever reasons, you can create a modified version of a package with ease. For example, say that you need a version of Nginx built against a custom version of OpenSSL. You can do just that with a few lines of code:
let
mypatch = pkgs.fetchpatch {
url = "https://example.com/bugfix-for-openssl.patch";
sha256 = "...";
};
openssl = pkgs.openssl.overrideAttrs (old: {
# add build flags
configureFlags = old.configureFlags ++ [ "--enable-foo" ];
# add dependencies
buildInputs = old.buildInputs ++ [ pkgs.foo ];
# add patches
patches = old.patches ++ [ mypatch ];
});
in
nginx.override { inherit openssl; }
You can't do that with other common package managers. You're stuck with whatever the package manager provides you. So if you're not in the happy path, you're out of luck. You'd either have to give up or build from scratch. If you need a bugfix for a particular package, you'd have wait until the fix reaches the package repository. When I was using Ubuntu, that was often until the next major release. None of this stuff is a problem with Nix, which allows for customization with very little effort.
It's easy to learn about Nix if you know where to look for. Nix Pills [1] would be a good start. The core language is the easiest part. It's JSON, but with functions and variables for proper abstraction. The documentation, while imperfect, is quite extensive compared to a majority of other high profile open source projects. If the documentation fails you, the Nixpkgs repository [2] is an even more rich source of information. The code is well organized, and I was able to get familiar with writing Nix packages fairly quickly by grepping the codebase. And finally, Nix has a sizable community so you could always ask if you're stuck.
Edit: it seems to be the largest software repo according to this (2020) [1].
[0] https://github.com/NixOS/nixpkgs [1] https://discourse.nixos.org/t/nixpkgs-has-been-the-largest-r...
It's on the NixOS website, but IMO it's worth making it more visible to newcomers:
https://nixos.org/guides/nix-pills/
From my experience, it became considerably more easier to figure out or search for solutions once I've grasped the basics from "Nix Pills." Once I became comfortable enough, I also relied heavily on the Nixpkgs repository[1] as a great source of information. It has an approachable codebase and some quick grepping gave me many examples when trying to figure out how to write packages for a particular language.
Currently going with a container image as the minimal deployable unit that gets put on top of a clean up to date OS. For me that's created with a Dockerfile using Alpine image variants. In a way I could see someone's rsync as an ok equivalent, but I'd do versioned symlinked directories so I can easily roll back if necessary if I went with this method. Something like update-alternatives or UIUC Encap/Epk: https://www.ks.uiuc.edu/Development/Computers/docs/sysadmin/.... Anyone remember that? I guess the modern version of Epkg with dependencies these days is https://docs.brew.sh/Homebrew-on-Linux. :-) Or maybe Nixpkgs: https://github.com/NixOS/nixpkgs?
Deployment-wise I've already done the Bash script writing thing to help a friend automate his deployment to EC2 instance. For myself I was going to start using boto3, but just went ahead and learned Terraform instead. So now my scripts are just simple wrappers for Docker/Terraform that build, push, or deploy that work with AWS ECS Fargate or DigitalOcean Kubernetes.
No CI/CD yet. DBs/backups I'll tackle next as I want to make sure I can install or failover to a new datacenter without much difficulty.
let
newerPkgs = import (builtins.fetchGit {
name = "release-21.05-pkgs";
url = "https://github.com/NixOS/nixpkgs/";
ref = "refs/heads/release-21.05";
rev = "8d6407e5a442e5e2fc50c3ca36411b6995afbc17";
}) {};
in
I may have the syntax slightly wrong, but something like that.
Edit: Ah, except Synapse is a service, so things get a little more complicated. https://nixos.wiki/wiki/FAQ/Pinning_Nixpkgs has some pointers.
Why wouldn't it be available? Nix packages are on github. [1]
> And that's on top the fact that "if you tried to follow this article step by step, you’ll have noticed that the versions of ruby and node you installed are probably slightly different from the ones above"
That's the same as on any other host. If I write about installing ruby on ubuntu today and you read the blog post a whole later you'll probably get a newer version.
This is one common question people ask when I introduce them to Nix and its ability to pin packages. They are often surprised to hear that you only need to pin one thing, which is the version of the Nixpkgs package collection.
Nixpkgs [1] is the central repo that contains all the Nix expressions for the "official" packages. It contains everything Nix needs to know to build a package. When you rebuild your system configuration, Nix will resolve all the packages using your local checkout of Nixpkgs into store derivations (.drv).
A .drv file uniquely identifies a specific version of a package. When any input to the expression change (source URLs, dependencies, etc.), it will result in a different derivation that will be installed under different paths in /nix/store. In other words, different inputs -> different outputs.
This is where binary distribution comes into play. Nix builds packages in sandboxes with tight isolation (no access to network or outside fs). Because the resulting store paths depend on the inputs being the same, for the same store path you will pretty much [2] get the exact same thing no matter where it's built. So Nix will lookup the binary cache for a matching object with the same store path. If it fails to find a match (e.g., because you modified the expression), Nix will simply build it. Binary cache can thus be seen as a transparent optimization that will get you binaries that are the same and will behave the same, as if you built it locally.
To summarize, when you use NixOS, you already have the source (Nixpkgs) to be able pin everything. In fact, you cannot rebuild the system without it. If the expressions don't change, the results won't. The only thing you need to to is to actually pin it to a specific commit and keep track of it in your VCS.
The ambiguity versus "*nix" as shorthand for Linux/Unix is unfortunate, but here we are. :)
I have stopped installing stuff from apt and thankfully my system never breaks now.
To learn more about Nix, I'd recommend grepping the nixpkgs [3] repository once you grasp the basics of the Nix language (Nix Pills [4] would be a good start for that). Since everything in the Nix official repository belongs in nixpkgs, it's easy to search for the information I want. I just wish CentOS and Debian had a way of searching across the entire codebase this easily when I couldn't figure out something from the docs.
[1]: https://nixos.org/manual/nixos/stable/#sec-x11 [2]: https://nixos.org/manual/nix/stable/#idm140737322634320 [3]: https://github.com/NixOS/nixpkgs [4]: https://nixos.org/guides/nix-pills/
A simple example of this is when I want to run a Python script from the internet, I can execute
nix-shell --run "python3 foo.py" -p "python3.withPackages (ps: with ps; [ numpy ])"
nix-shell --run "ghci" -p "haskellPackages.ghcWithPackages (ps: with ps; [ QuickCheck ])"
These problems have already been solved to varying degrees in other places, but Nix lets you unify them (think of the huge amounts of libraries downloaded by cargo, cabal, node and so on scattered amongst projects and your home folder, Nix stores everything in /nix/store/) into a single framework.
I'd like to also say that the Nixpkgs repository[4] is super easy to contribute to, just open a PR on GitHub as opposed to sending patches via mailing lists, which is checked via the CI. I'm not sure if there's something analogous on other package repositories but there's also a bot[5] that opens thousands of PRs updating and testing packages automatically.
[0] https://github.com/Shopify/comma
[1] https://github.com/Lassulus/nix-autobahn
[2] https://github.com/nix-community/nixos-generators
[3] https://matthewbauer.us/blog/beginners-guide-to-cross.html
Discussion on /r/github: https://www.reddit.com/r/github/comments/hei81f/
Personally I think it's an improvement on mobile - finally the entire README is readable by default.
That being said was there any warning/reasoning behind the change? I cannot find any announcements.
Any official package from Nix belongs to this single repository, making it easy to see what other packages are doing. If I want to see how python packaging is done, I simply search "python" in the nixpkgs repository.
It's also easier to grasp what is going on in nixpkgs than other packaging systems. The Nix expression language offers better readability than say, RPM spec files. Meanwhile the nixpkgs repository includes support for numerous build systems and frameworks. This combined allows you to create packages with minimal effort most of the time.
If this repository is the upstream nixpkgs[1], or if your private repository follows the same rules about not importing anything without fully-pinned hashes, you can already get those guarantees about the image content with Nixery today!
The one exception is packages that are not in the binary cache, end up rebuilt and aren't reproducible (the binaries might differ) - but NixOS is ~98% reproducible already[2]!
[1]: https://github.com/NixOS/nixpkgs/
[2]: https://r13y.com
There are tons of dedicated maintainers. You can see that in the maintainers attribute of most packages. So I don't get your point regarding the lack of dedicated maintainers.
I maintain several packages there, and a bot sends me an automated message every time upstream updates the source, and auto-generates a new package definition for me. Things are really streamlined and bleeding edge. The unstable channel is often more up-to-date than Arch Linux even, due to the high number of Nix developers.