Multi-package rust project

Developing with flox
1

Hi,

I would like to floxify a rust project with multiple packages. I am not sure if flox init is doing the right thing here because when I try to modify default.nix to add something like the RUST_SRC_PATH = "${rustPlatform.rustLibSrc}"; variable, it is not available when running flox develop.

Would you recommend to run flox init on each individual package?

2

Seems you have tripped across an interesting example! As you have highlighted the repo is comprised of a bunch of subpackages referenced as “members” in the Cargo.toml. As such it’s designed to be built once at the top level and create a big package with everything in it.

When you type flox init and choose the rust builder it almost gets it right if not for two things:

  1. the package requires the dbus.dev buildInput
  2. on darwin only it also requires the darwin.apple_sdk.frameworks.{DiskArbitration,Foundation} buildInputs
  3. the following error indicates that this package requires rust version 1.65.0:
ockam_core>    Compiling ockam_vault v0.67.0 (/build/source/implementations/rust/ockam/ockam_vault)                                                             
ockam_core> error[E0658]: labels on blocks are unstable                                                                                                         
ockam_core>   --> implementations/rust/ockam/ockam_vault/src/secret_impl.rs:51:37                                                                               
ockam_core>    |                                                                                                                                                
ockam_core> 51 |             SecretType::NistP256 => '_block: {                                                                                                 
ockam_core>    |                                     ^^^^^^^                                                                                                    
ockam_core>    |                                                                                                                                                
ockam_core>    = note: see issue #48594 <https://github.com/rust-lang/rust/issues/48594> for more information                                                   
ockam_core> error[E0658]: labels on blocks are unstable                                                                                                         
ockam_core>    --> implementations/rust/ockam/ockam_vault/src/secret_impl.rs:148:37                                                                             
ockam_core>     |                                                                                                                                               
ockam_core> 148 |             SecretType::NistP256 => '_block: {                                                                                                
ockam_core>     |                                     ^^^^^^^                                                                                                   
ockam_core>     |                                                                                                                                               
ockam_core>     = note: see issue #48594 <https://github.com/rust-lang/rust/issues/48594> for more information                                                  
ockam_core> For more information about this error, try `rustc --explain E0658`.                                                                                 
ockam_core> error: could not compile `ockam_vault` due to 2 previous errors                                                                                     
ockam_core> warning: build failed, waiting for other jobs to finish...                                                                                          
ockam_core> build failed in buildPhase with exit code 101

So what next? The good news is that rust 1.65.0 is available in the “unstable” version of nixpkgs, and with that it should be exactly this easy to build your package:

flox --stability unstable build

However, this has uncovered a regression in our support for the --stability flag. :unamused: We’re working to fix that now and hope to get a solution out for you ASAP. Watch this space …

3

Thanks a lot Michael for taking the time to look at this one.

However, this has uncovered a regression in our support for the --stability flag

And I’m glad I’m participating to the progress of flox :slight_smile: .

1 Like
4

Ok … progress! The latest version of flox-prerelease (0.0.7-r98) fixes the stability issue so you should now be able to build the entire package by invoking flox init and then replacing pkgs/ockam/default.nix with the following:

{ self, lib, rust_1_65, hostPlatform, dbus, openssl, pkg-config, libiconv, darwin }:

let
  inherit (rust_1_65.packages.stable) rustPlatform;

in rustPlatform.buildRustPackage {
  pname = "ockam";
  version = "0.77.0"; # XXX: should get that from Cargo.lock
  src = self;
  doCheck = false; # disable tests requiring network access
  cargoLock = {
    lockFile = self + "/Cargo.lock";
    outputHashes = {
      #   "dependency-0.0.0" = lib.fakeSha256;
    };
  };
  buildInputs =
    [
      openssl.dev
      dbus.dev
    ] ++ lib.optional hostPlatform.isDarwin [
      libiconv
      darwin.apple_sdk.frameworks.Security
      darwin.apple_sdk.frameworks.DiskArbitration
      darwin.apple_sdk.frameworks.Foundation
    ] ++ lib.optional hostPlatform.isLinux [ ];
  nativeBuildInputs = [
    pkg-config # for openssl
  ];
}

… and using the following commands:

flox pull -e flox/prerelease
flox --stability unstable build

You can similarly enter a dev environment with flox --stability unstable develop.

Note: once you enter a flox development shell then the FLOX_STABILITY environment variable will be present in your shell, so any further invocations of flox build will use the unstable version of nixpkgs by default. You can similarly export FLOX_STABILITY=unstable in any shell if you would like for flox (build|develop|publish|...) to use that stability by default.

Give that a try and let us know how it works. Back to your original question, we could certainly flox init each of the subpackages in turn to make building sub-parts of the package faster, but I’m unsure whether that would make for a better development environment on the whole. What would you prefer to see and why? Now that we’ve solved the more fundamental problem of getting the package to build as a whole we’d be happy to look into that for you.

Many thanks as usual for the excellent feedback! :smiley:

footnote: I had no problems building this on x86_64-linux but on aarch64-darwin I had repeated (Nix) build failures with the message “usually happens due to networking issues”. The quickest way to build in the face of such errors is to add the --keep-going flag to the flox build invocation. Kinda annoying, but it did build successfully in the end on darwin.

1 Like
5

Hi Michael,

I did a flox init in the implementations/rust/ockam directory and on flox build I’m getting:

± flox build                                                                                                                                                                                                                                                                                                               prerelease etorreborre/ockam etorreborre/default
error: flake 'git+file:///Users/etorreborre/projects/rust/ockam?dir=implementations%2frust%2fockam' does not provide attribute 'packages.x86_64-darwin.packages.x86_64-darwin', 'legacyPackages.x86_64-darwin.packages.x86_64-darwin' or 'packages.x86_64-darwin'

ERROR: cannot find attribute path - have you run 'flox init'?

warning: not writing modified lock file of flake 'git+file:///Users/etorreborre/projects/rust/ockam?dir=implementations%2frust%2fockam':
• Updated input 'flox-floxpkgs/nixpkgs/nixpkgs':
    'github:flox/nixpkgs/2788904d26dda6cfa1921c5abb7a2466ffe3cb8c' (2022-11-22)
  → 'github:flox/nixpkgs/1710ed1f6f8ceb75cf7d1cf55ee0cc21760e1c7a' (2022-12-13)
error: flake 'git+file:///Users/etorreborre/projects/rust/ockam?dir=implementations%2frust%2fockam' does not provide attribute 'packages.x86_64-darwin.default' or 'defaultPackage.x86_64-darwin'
flox [prerelease etorreborre/ockam etorreborre/default]

(that’s using r98 and export FLOX_STABILITY=unstable).

One thing I noticed was that default.nix has been created under pkgs not pkgs/ockam. That being said even moving the default.nix file there gives me the same error.

Back to your original question, we could certainly flox init each of the subpackages in turn to make building sub-parts of the package faster, but I’m unsure whether that would make for a better development environment on the whole. What would you prefer to see and why?

I indeed prefer a top level init for now, because I just want to be able to explore the code. But I wonder what this might mean for development and I suppose that, for incremental recompilation/building we might prefer one flake per package.

6

Ah … yes, sorry - flox init needs to be invoked from the toplevel directory of the git clone. We should make it so that flox knows to do that when performing an init - I’ll log an issue to do exactly that.

Sorry I didn’t make my exact patch available to you last night - feel free to clone https://github.com/flox-examples/ockam and try that. You can also install my built package by running flox subscribe flox-examples github:flox-examples/floxpkgs and flox install unstable.flox-examples.ockam if you just want to run it (copying the pre-built binaries from cache.floxdev.com).

7

I am still trying to build but I am currently stuck on one package

[0/13 built, 1/0/27 copied (0.0/1759.4 MiB), 0.0/311.1 MiB DL] fetching ncurses-6.3-p20220507 from https://cache.nixos.org

which doesn’t seem to want to download. Can I force a rebuild from source maybe? Does that happen often?

8

I get that all the time - it’s caused by packet loss coming back from the cache.nixos.org CDN. I usually just interrupt and restart the build. Anecdotally it’s worse in London (with G.networks) than it is for my US-based colleagues. Will be interested to hear if it doesn’t work for you after a second, third, or fourth attempt. (You can interrupt it the moment it hangs and re-attempt right away.)

9

Unfortunately the build stays stuck on that specific package. I have even tried to use a VPN to maybe trick the system and use another CDN but that doesn’t change anything.

10

I might be the victim of something like Fix deadlocked nix-daemon zombies on darwin #3294 by roberth · Pull Request #6052 · NixOS/nix · GitHub

11

A restart of my machine unblocked me. Before doing that I had a look at my ps and saw that there were several rogue flox (r58) and nix processes. Maybe there is something not very thread-safe with nix which can deadlock if you run too many concurrent processes?

12

I can confirm that everything builds ok and rust-analyzer is starting fine on that project. Thanks Michael!

1 Like
13

Just to summarise, I understand the only outstanding item on this thread is for us to look at performing flox init from the toplevel directory of the git clone. We have this logged and will advise once it is sorted. In the meantime though I believe you are all set :slight_smile:

1 Like
14

Hi @etorreborre - we have completed the porting of flox init to rust and we believe this has addressed your issues noted above. Please do let me know if this is not the case and we can of course revisit. :slight_smile:

1 Like
15

Thanks Robin, everything seems to be working out the box now!

BTW I noticed a typo in a generated comment: s/flox environment configuation/flox environment configuration

1 Like
16

Thanks fixed! fixing spelling typo configuRation :-) by robinbrantley · Pull Request #117 · flox/floxpkgs · GitHub

1 Like