Announcing Lix 2.92 "Bombe glacée"

We at the Lix team are proud to announce our third major release, version 2.92 “Bombe glacée”. This release focuses on evolution work of the evaluator and store in order to be able to replace the Nix store protocol, to make the evaluator faster, and to remove rarely-used and troublesome features of the Nix language.

A bombe glacée is a frozen dessert made of ice cream molded into a hemisphere which may contain some sort of filling

Lix is a Nix implementation focused on reliability, predictability, friendliness, developed by a community of people from around the world. We have long term plans to incrementally evolve Nix to work in more places, to make it more reliable and secure, and to update the language and semantics to correct past mistakes and reduce errors, all the while providing an amazing tooling experience.

Upgrading from CppNix or previous Lix versions

The upgrade procedure depends on how you installed Lix or CppNix, and is fully described in the Lix installation guide.

If you are using Lix from nixpkgs on NixOS, you just need to upgrade your nixpkgs once the upgrade pull request (FIXME: still needs doing as of this writing) has passed through the build farm into your channel; no other action is required.

If you want to help us test the next version of Lix, consider running main by following the beta guide.

Changes

The general theme of Lix 2.92 is improvements to the daemon and language to make room for future evolution.

Language and daemon evolution work

In 2.92, we rework the daemon to use a modern asynchronous runtime, KJ, with asynchronous C++ code replacing the previous hard-to-follow custom async implementation. This makes room for our future plans to replace the daemon protocol, which will allow for richer communication with daemons, support for multiple Nix implementations and feature flags, better tracing and progress visiblity, and more. These refactors are expected to bring about a performance improvement while building large numbers of derivations in parallel, but are primarily an improvement in maintainability. Big thanks to eldritch horrors for their work on this!

On the language side, Lix is deprecating several problematic Nix language features, both to remove foot guns for users and to improve our own code maintainability. Using any of these deprecated features will throw an error or warning at parse time; there is a configuration opt-out of the new behavior for each individual feature. This means that old Nix code can still be run with a CLI flag. However, this should never be necessary except for very old Nixpkgs checkouts.

URL literals, which are unnecessary and disallowed inside nixpkgs today, and which cause confusion with lambdas both for humans and computers.
For example, x:x is a URL literal and thus a string, which is confusing since it looks kind of like a lambda.
Supporting these also slows down our parser.
Exposed implementation details like a - b being literally translated to __sub a b and thus allowing a very limited and not terribly useful form of operator overloading.
```
nix-repl> let __sub = builtins.add; in 1 - 2
3
```
This “operator overloading” only exists for a very limited list of operators and was an implementation shortcut leaking to user code rather than an intended feature.
This affects the following operators:
- __sub
- __lessThan
- __mul
- __div
This also affects the desugaring of <nixpkgs> that uses __findFile and __nixPath. If you are overriding these for some use case, please reach out on the issue tracker so we can understand how to make a better replacement!
__overrides in rec sets, originally used years ago to implement overrides of package properties in nixpkgs (which were since replaced by the .overrideAttrs and .override fixed-points of today):
```
nix-repl> :p rec { a = 2; __overrides.a = 3; }
{
  __overrides = { a = 3; };
  a = 3;
}
```
The legacy let syntax of let { a = 2; body = a; }.

To ease migration, only arcane language features that have not been used in nixpkgs for years are currently turned into parser errors. Features that are still occasionally in use just give a warning instead, which will then be upgraded to an error in a future update. Either way, the opt-out mechanism is intended to stay for a long time, to allow for backwards compatibility. The point where this compatibility would be removed is when Lix’s daemon/protocol story is improved enough that a viable alternative can be provided, likely by allowing for pinning the evaluator in projects.

Big thanks to piegames and eldritch horrors for implementing these deprecations!

There are some further ideas in the concept/prototype stage in the project about ways to possibly evolve the Nix language sustainably and compatibly. You can see the in-progress development documentation for some more information on the language work in Lix.

Breaking changes

Many of the above-listed legacy language constructs generate an error unless you enable the appropriate deprecated-features thanks to piegames and eldritch horrors.
Support for some especially weird fetcher usages has been removed thanks to eldritch horrors. Network protocols other than http, https, ftp, ftps and file have been removed from anything using the internal fetching infrastructure for http-like purposes like binary caches, builtins.fetchTarball, <nix/fetchurl.nix> and similar.
S3 and git+ssh and suchlike use a different system and are unaffected.
Sorry to anyone using Lix to connect to Gopher servers! Or, to devious catgirls abusing Gopher for server-side request forgery or posting to IRC from the evaluator.
Some unused Content-Encoding values that violate the HTTP standard like xz and bzip2 have also been removed. If you are using these, neither curl nor web browsers likely work on your site.
nix fmt with no arguments now just gets passed directly to the formatter rather than being equivalent to nix fmt . thanks to zimbatm.
This removes some weird footguns with respect to the current directory and aligns behaviour with CppNix.
The following formatters are known to have behaviour changes with this change (since they format stdin when no argument is given):
- nixfmt
- nixpkgs-fmt
- alejandra
If you want to restore the previous behaviour in a flake, you can use the following snippet:
```
{
  outputs = { self, nixpkgs, systems }:
    let
      eachSystem = nixpkgs.lib.genAttrs (import systems) (system: nixpkgs.legacyPackages.${system});
    in
    {
      formatter = eachSystem (pkgs:
        pkgs.writeShellScriptBin "formatter" ''
          if [[ $# = 0 ]]; set -- .; fi
          exec "${pkgs.nixfmt-rfc-style}/bin/nixfmt "$@"
        '');
    };
}
```

Improvements

Lix 2.92 primarily brings some subtle but nice user experience improvements:

Relative and tilde paths are supported inside user config files thanks to [wiggles]. This is helpful for settings like repl-overlays like so:
```
repl-overlays = ~/.config/nix/repl.nix
```
or
```
repl-overlays = repl.nix
```
which, inside the default user config location, ~/.config/nix/nix.conf, refers to ~/.config/nix/repl.nix.
When prompted to accept nixConfig entries from a flake, you can now answer N to refuse all the settings from the flake, thanks to ma27. Previously this could only be accomplished with answering n several times or passing --deny-flake-config.
Lix now allows for reconfiguring the temporary directory as the temp-dir setting instead of hardcoding it to /tmp thanks to lilyball.
This is useful, among other things, on macOS, for using a case sensitive build directory. It’s also nice for moving builds off of a tmpfs.
Alt+Left and Alt+Right can now be used to navigate by words in nix repl thanks to [wiggles].

nix --version now has more details in the output (as would previously be the case if you used nix-env --version or nix --verbose --version), thanks to just1602.

$ nix --version -v
nix (Lix, like Nix) 2.92.0-dev-pre20241119-b0d7a81
System type: x86_64-linux
Additional system types: i686-linux, x86_64-v1-linux, x86_64-v2-linux, x86_64-v3-linux
Features: gc, signed-caches
System configuration file: /etc/nix/nix.conf
User configuration files: /home/jade/.config/nix/nix.conf:/etc/xdg/nix/nix.conf
Store directory: /nix/store
State directory: /nix/var/nix
Data directory: /nix/store/a54db2zb3kbp0wanzsy70jnh8w4ghyzw-lix-2.92.0-dev-pre20241119-b0d7a81/share

Better errors

When building derivations that set allowSubstitutes = false, Lix will not substitute them from a binary cache even if they are available. However, it is possible that Lix also cannot build them because they are for an architecture it doesn’t have a builder for, or because max-jobs is 0.
This still results in weird errors since Lix may unexpectedly try to build them anyway instead of substituting them, but there is now a hint in the error as to a possible cause and fix, thanks to jade:
```
$ nix build -f fail.nix
error: a 'unicornsandrainbows-linux' with features {} is required to build '/nix/store/...-meow.drv', but I am a 'x86_64-linux' with features {...}

       Hint: the failing derivation has allowSubstitutes set to false, forcing it to be built rather than substituted.
       Passing --always-allow-substitutes to force substitution may resolve this failure if the path is available in a substituter.
```

If a non-reproducible multiple-output derivation is built with --check, Lix now lists all the mismatched outputs along with the paths for comparison rather than just one of them as before, thanks to lheckemann.

Old output:

error: derivation '/nix/store/4spy3nz1661zm15gkybsy1h5f36aliwx-python3.11-test-1.0.0.drv' may not be deterministic: output '/nix/store/ccqcp01zg18wp9iadzmzimqzdi3ll08d-python3.11-test
-1.0.0-dist' differs from '/nix/store/ccqcp01zg18wp9iadzmzimqzdi3ll08d-python3.11-test-1.0.0-dist.check'

New output:

error: derivation '4spy3nz1661zm15gkybsy1h5f36aliwx-python3.11-test-1.0.0.drv' may not be deterministic: outputs differ
         output differs: output '/nix/store/ccqcp01zg18wp9iadzmzimqzdi3ll08d-python3.11-test-1.0.0-dist' differs from '/nix/store/ccqcp01zg18wp9iadzmzimqzdi3ll08d-python3.11-test-1.0.0-dist.check'
         output differs: output '/nix/store/yl59v08356i841c560alb0zmk7q16klb-python3.11-test-1.0.0' differs from '/nix/store/yl59v08356i841c560alb0zmk7q16klb-python3.11-test-1.0.0.check'

When an attribute selection fails, thanks to piegames, the error message now correctly points to the attribute in the chain that failed instead of at the beginning of the entire chain.

 error: attribute 'x' missing
-       at /pwd/lang/eval-fail-remove.nix:4:3:
+       at /pwd/lang/eval-fail-remove.nix:4:29:
             3| in
             4|   (removeAttrs attrs ["x"]).x
-             |   ^
+             |                             ^
             5|

Debuggability

Lix 2.92 now names all its threads thanks to jade:

  (gdb) info thr
  Id   Target Id                    Frame
* 1    LWP 3719283 "nix-daemon"     0x00007e558587da0f in accept ()
   from target:/nix/store/c10zhkbp6jmyh0xc5kd123ga8yy2p4hk-glibc-2.39-52/lib/libc.so.6
  2    LWP 3719284 "signal handler" 0x00007e55857b2bea in sigtimedwait ()
   from target:/nix/store/c10zhkbp6jmyh0xc5kd123ga8yy2p4hk-glibc-2.39-52/lib/libc.so.6

For certain errors, Lix now prints a stack trace with bug reporting instructions thanks to jade.

This will be expanded to more errors and further improved in the future, but it is now available for C++ exceptions falling out of main.

Lix crashed. This is a bug. We would appreciate if you report it along with what caused it at https://git.lix.systems/lix-project/lix/issues with the following information included:

Exception: std::runtime_error: test exception
Stack trace:
 0# nix::printStackTrace() in /home/jade/lix/lix3/build/lix/nix/../libutil/liblixutil.so
 1# 0x000073C9862331F2 in /home/jade/lix/lix3/build/lix/nix/../libmain/liblixmain.so
 2# 0x000073C985F2E21A in /nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/libstdc++.so.6
 3# 0x000073C985F2E285 in /nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/libstdc++.so.6
 4# nix::handleExceptions(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void ()>) in /home/jade/lix/lix3/build/lix/nix/../libmain/liblixmain.so
 ...

Fixes

builtins.fetchGit is better behaved thanks to ma27: you can now pass tag names as ref = and it will work. This restores compatibility with CppNix 2.3.
auto-optimise-store no longer risks store corruption on macOS, thanks to lilyball.
S3 binary cache stores now use the proxy environment variables thanks to the sleuthing of jade.
Flakes and restricted evaluation mode now correctly check whether paths are allowed thanks to eldritch horrors. Thanks to the reporter for telling us about this via the security report process by emailing a report with a reproducer to security at lix dot systems.
```
{
  inputs = {};
  outputs = {...}: {
    lol = builtins.readFile "${/etc/passwd}";
  };
}
```
In prior versions since at least 2.18, nix eval --raw .#lol didn’t throw an error and acted as if --impure was passed.
This has been fixed to work as intended, blocking non-permitted absolute paths.
We treated this bug as a minor security bug to test our response process, even though it did not break the security model. The Lix evaluator is not a security boundary and we do not recommend evaluating untrusted code with it on systems that handle trusted data.

The Lix daemon should no longer crash when clients disconnect unexpectedly since it has had its interrupt handling fixed thanks to jade.
This should be helpful in reducing the noise in core dump logs of large CI installations.

Miscellaneous development-facing changes

Lix now defines its builtins, experimental features, and other pieces in Markdown files in the codebase. Among other things, this will help a lot with being able to cross compile Lix documentation: previously, this data was dumped out by running Lix, which doesn’t work in cross compiled contexts. Now, it will be possible to directly generate the documentation pieces from the Markdown by virtue of not being buried in C++ source code.
Thanks so much to alois31 for improving this longstanding issue.
Lix now has its #include directives reshuffled to include Lix and the library name in them as the only supported format. This fixes some previous sketchy behaviour like Lix installing a config.h in the global include path when it’s used as a library, which conflicted with perl among other software.
Thanks to jade for completing this.
It’s now possible to write integration tests for Lix in Python thanks to jade.
This should make it easier and nicer to write tests.

This is not a fully comprehensive list of every small change, and we are thankful for every contributor’s hard work in making this release happen.

You can read the full changelog in the manual.

Thanks, as always, to the following groups:

The several dozen people who beta tested the upcoming release by running main in production since the 2.92 branch-off. We really appreciate having immediate feedback on our work, and the trust of running main alongside us means a lot to us.
If you want to run Lix main yourself, see the beta guide for details.
Everyone who contributed by filing bugs and giving us feedback on Matrix.
All the first time contributors who made their first contributions to a Nix implementation in Lix. We are eternally grateful to everyone who helped us out on the numerous important but tedious issues.
All the contributors who have helped us with the backlog of bugs.
The CppNix contributors and CppNix team, without whom we would not have this software, and who wrote some of the improvements ported into this release.

Onwards and upwards for the next release. We look forward to continuing to work together with everyone to build a better foundation for the evolution of Nix.