Packages and develop

Hi, I don’t understand why, when I type flox packages --all I get

cargo - Downloads your Rust project's dependencies and builds your project
  stable.nixpkgs-flox.cargo@1.53.0
  stable.nixpkgs-flox.cargo@1.57.0
  stable.nixpkgs-flox.cargo@1.60.0
  staging.nixpkgs-flox.cargo@1.60.0
  staging.nixpkgs-flox.cargo@1.61.0
  staging.nixpkgs-flox.cargo@1.62.1
  unstable.nixpkgs-flox.cargo@1.61.0
  unstable.nixpkgs-flox.cargo@1.62.1
  unstable.nixpkgs-flox.cargo@1.64.0

as the versions for cargo, but when I enter a development shell with flox develop I get

[flox] λ cargo --version
cargo 1.63.0

And that version is not even listed in the list above. Where does it come from?

Moreover I seems to have issues with the syntax for command lines options because when I try to filter by subchannel I get no results:

flox packages [ --all | stability[.channel[.package]] ] [--show-libs]
        list all packages or filtered by channel[.subchannel[.package]]

for

flox packages stable.nixpkgs-flox.cargo
flox packages stable.nixpkgs-flox
flox packages stable

To give a bit more colour here is what I’m getting when using the stability flag

± flox --stability unstable develop
[flox] λ cargo --version
cargo 1.63.0
[flox] λ exit
exit

± flox --stability stable develop
[flox] λ cargo --version
cargo 1.63.0
[flox] λ

Great question - thanks for asking.

Flox provides multiple ways of consuming nixpkgs, and the most important two are:

  1. catalog

    The catalog is a record of the packages that have built successfully in nixpkgs over time, limited to the last 3 versions in each of the 3 stabilities. Packages found in the catalog are guaranteed a) to have built successfully and b) be available for substitution from the cache.nixos.org binary cache.

  2. stabilities

    The github:flox/nixpkgs repository has three branches - stable, staging and unstable - which track the upstream github:NixOS/nixos-unstable at different points in time. The purpose of these branches is threefold: they a) provide selective access to the “bleeding edge” just for those times when you need it, b) control how often things change over time, and c) the cascading nature of stabilities provides an opportunity to test packages in unstable and staging prior to appearing in the stable branch.

It’s not a problem for flox environments to include packages built from different versions of nixpkgs, so when you install a package to an environment with flox install that is using the catalog.

However, when developing software with flox build or flox develop, it is essential that all of the packages used in that “instantiation” be derived from a single version of nixpkgs, so in this case it is the stability that determines which version of nixpkgs is used. The flox application provides the --stability option for selecting the stability of a given invocation, as in flox --stability unstable develop.

Back to your question … you’re getting version 1.63.0 when invoking flox develop (with the “stable” stability) because the github:flox/nixpkgs stable branch has that as the current version of rustc/cargo, and you’re wondering why is the catalog showing the latest “stable” version as 1.60.0?

In short … that’s a bug with the rendering of the catalog - it should be showing 1.63.0! Thank you for highlighting - I’ll look into that first thing in the morning because that’s definitely not correct.

One further comment regarding the use of flox --stability … it’s not working at the moment. We discovered on Friday there had been a regression in the communication of the nixpkgs override throughout the calling stack and it only uses the “stable” branch. We’re working on a fix for that now and I expect to get a patch out for that bug by Tuesday at the latest.

Found and fixed the bug (we had restructured the catalog metadata and in the process broken the sorting of builds by revision count), and I’m pleased to report the catalog is now showing the most recent cargo versions:

$ flox search cargo --refresh | head -10
cargo - Downloads your Rust project's dependencies and builds your project
  stable.nixpkgs-flox.cargo@1.62.0
  stable.nixpkgs-flox.cargo@1.62.1
  stable.nixpkgs-flox.cargo@1.63.0
  staging.nixpkgs-flox.cargo@1.62.1
  staging.nixpkgs-flox.cargo@1.63.0
  staging.nixpkgs-flox.cargo@1.64.0
  unstable.nixpkgs-flox.cargo@1.62.1
  unstable.nixpkgs-flox.cargo@1.63.0
  unstable.nixpkgs-flox.cargo@1.64.0

Thanks again for highlighting the bug!

Nice! Is the new version released? (suggestion: having the latest version number in the install docs could be a way to check what’s the most up to date version)

1 Like

thanks for the suggestion; we will definitely look into this. In the meantime, we’ll let you know once the update is available for you to test :slight_smile:

1 Like

Good news, the new version (0.0.6-r49) is out and available for installation using … flox!!!

$ flox search -c flox flox-prerelease
flox.flox-prerelease
  stable.flox.flox-prerelease@0.0.6-r45
  stable.flox.flox-prerelease@0.0.6-r49
  staging.flox.flox-prerelease@0.0.6-r45
  unstable.flox.flox-prerelease@0.0.6-r45

$ flox install flox.flox-prerelease

$ flox activate
Supercharged by flox!

[flox] [~/src/rust-edits]$ flox --version
Version: 0.0.6-r49

This version fixes the bug with the --stability flag as shown below:

[flox] [~/src/rust-edits]$ flox develop
Supercharged by flox!
[flox] λ cargo --version
cargo 1.63.0
[flox] λ 
exit

[flox] [~/src/rust-edits]$ flox --stability unstable develop
warning: not writing modified lock file of flake 'git+file:///home/brantley/src/rust-edits':
• Updated input 'floxpkgs/nixpkgs/nixpkgs':
    'github:flox/nixpkgs/d6490a0bd9dfb298fcd8382d3363b86870dc7340' (2022-09-21)
  → 'github:flox/nixpkgs/f994293d1eb8812f032e8919e10a594567cf6ef7' (2022-10-25)
Supercharged by flox!
[flox] λ cargo --version
cargo 1.64.0
[flox] λ 
exit

I’ve published this version for x86_64-linux and {aarch64,x86_64}-darwin, so the above procedure should work on any of those system types.

Can you please give that a try and confirm that it works the same for you? With your confirmation we’ll go ahead and merge it into the main release repository tomorrow, at which point the default installers will be updated to reflect the new version on all distributions/architectures.

Again, many thanks for all of your feedback!

Hi Michael,

I’m afraid that it doesn’t work for me

~/projects/rust/edits on main
± flox search -c flox flox-prerelease
flox.flox-prerelease
  stable.flox.flox-prerelease@0.0.6-r49

~/projects/rust/edits on main
± flox install flox.flox-prerelease
failed to find a binary download for 'flake:flox#evalCatalog.x86_64-darwin.stable.flox-prerelease'
try building from source by installing as 'flake:flox#evalCatalog.x86_64-darwin.stable.flox-prerelease.fromSource':

	$ flox install flake:flox#evalCatalog.x86_64-darwin.stable.flox-prerelease.fromSource

failed to find a binary download for 'flake:flox#evalCatalog.x86_64-darwin.stable.flox-prerelease'

ERROR: failed to install packages: flake:flox#evalCatalog.x86_64-darwin.stable.flox-prerelease

Possible the next thing I tried explains why?

± flox install flake:flox#evalCatalog.x86_64-darwin.stable.flox-prerelease.fromSource
ERROR: Repository not found.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
warning: could not read HEAD ref from repo at 'ssh://git@github.com/flox/floxpkgs-internal.git', using 'master'
ERROR: Repository not found.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
error: program 'git' failed with exit code 128
(use '--show-trace' to show detailed location information)

ERROR: failed to install packages: flake:flox#evalCatalog.x86_64-darwin.stable.flox-prerelease.fromSource

Something having to do with access rights on your catalog?

My apologies - I published the metadata but neglected to copy the closure to cache.floxdev.com. I’ve done that now, can you please try the installation again? Many thanks!

I’m still getting the same message. Is there anything else I should check?

Thanks for your patience … it appears the package itself is being published with a path beginning with “70hsn04v”, but that is just a package containing 6 symlinks pointing at the package with a path beginning with “vhawj9db”. Furthermore, one of those links (for manifest.nix) is clearly wrong.

I’m not sure why the published entry contains the (superfluous?) extra top-level package and we’ll look into the broken link, but as I’ve now copied that extra package into the binary cache the installation should succeed now - can you try the command once more?

Background data for the record:

MacBook-Air:floxpkgs-internal brantley$ result/bin/flox install flox.flox-prerelease
MacBook-Air:floxpkgs-internal brantley$ result/bin/flox list --out-path | grep flox.flox-prerelease
    2 stable.flox.flox-prerelease	/nix/store/70hsn04v5kfxa0rj3v2qys37d7pyb170-flox-prerelease-0.0.6-r49
MacBook-Air:floxpkgs-internal brantley$ find /nix/store/70hsn04v5kfxa0rj3v2qys37d7pyb170-flox-prerelease-0.0.6-r49 -ls
1542488        0 dr-xr-xr-x    8 root             wheel                 256  1 Jan  1970 /nix/store/70hsn04v5kfxa0rj3v2qys37d7pyb170-flox-prerelease-0.0.6-r49
1542494        0 lrwxr-xr-x    1 root             wheel                  69  1 Jan  1970 /nix/store/70hsn04v5kfxa0rj3v2qys37d7pyb170-flox-prerelease-0.0.6-r49/manifest.nix -> /nix/store/vhawj9dbnvlqj5b2dw3masyq603ay7ym-flox-prerelease-0.0.6-r49
1542489        0 lrwxr-xr-x    1 root             wheel                  73  1 Jan  1970 /nix/store/70hsn04v5kfxa0rj3v2qys37d7pyb170-flox-prerelease-0.0.6-r49/bin -> /nix/store/vhawj9dbnvlqj5b2dw3masyq603ay7ym-flox-prerelease-0.0.6-r49/bin
1542490        0 lrwxr-xr-x    1 root             wheel                  77  1 Jan  1970 /nix/store/70hsn04v5kfxa0rj3v2qys37d7pyb170-flox-prerelease-0.0.6-r49/libexec -> /nix/store/vhawj9dbnvlqj5b2dw3masyq603ay7ym-flox-prerelease-0.0.6-r49/libexec
1542491        0 lrwxr-xr-x    1 root             wheel                  73  1 Jan  1970 /nix/store/70hsn04v5kfxa0rj3v2qys37d7pyb170-flox-prerelease-0.0.6-r49/etc -> /nix/store/vhawj9dbnvlqj5b2dw3masyq603ay7ym-flox-prerelease-0.0.6-r49/etc
1542492        0 lrwxr-xr-x    1 root             wheel                  73  1 Jan  1970 /nix/store/70hsn04v5kfxa0rj3v2qys37d7pyb170-flox-prerelease-0.0.6-r49/lib -> /nix/store/vhawj9dbnvlqj5b2dw3masyq603ay7ym-flox-prerelease-0.0.6-r49/lib
1542493        0 lrwxr-xr-x    1 root             wheel                  75  1 Jan  1970 /nix/store/70hsn04v5kfxa0rj3v2qys37d7pyb170-flox-prerelease-0.0.6-r49/share -> /nix/store/vhawj9dbnvlqj5b2dw3masyq603ay7ym-flox-prerelease-0.0.6-r49/share

We’re making progress, no errors, but…

± flox install flox.flox-prerelease

~/projects/rust/edits on main
± flox --version
Version: 0.0.6-r45

Well, sorry, everything works of course!

± flox activate
[flox]
~/projects/rust/edits on main
± flox --version
Version: 0.0.6-r49

Thanks!

2 Likes

Just wrapping up thread and confirming that flox has been updated and the current version is flox-0.0.6-r49. We plan to announce future releases in flox discourse.

Any issues, just let us know. :slight_smile:

1 Like