Publishing a package


I was trying to publish a package with a recent version of flox. I can see that publish is still considered experimental but I am lacking a bit of information from the documentation.

  • flox publish requires a cache URL. I conjectured that would do the trick
  • signing-key is less clear to me

For signing-key I first tried to give the path of my ~/.ssh/id_ecdsa_sk which actually uses Yubikey. That didn’t work so I tried to generate other keys with:

But nothing worked.

Could you please tell me how to generate a proper signing key and publish my package?

The other thing which I find confusing with the documentation about publishing is the notion of channel.

The documentation says that after publishing a package a catalog is created in the package github repository. Then it shows that we can create a channel, more or less by subscribing to that repository. This breaks a bit my mental model since one package = one repository = one catalog = one channel but a channel is supposed to give access to several packages.

Thanks for your help!

How should those things be understood then?

at this point the publish user experience works for us (flox) to make our own packages available but many of the rough edges must be smoothed out before it’s ready for real world usage.

we’ll send some clarifications on key generation shortly

The other thing which I find confusing with the documentation about publishing is the notion of channel.

that’s good feedback about some confusion between channel & catalog. multiple channels is so you can have your own catalog alongside flox’s.

1 Like

Hi @etorreborre, I saw your blog about flox and loved it!

re the channel query, let me try to explain and perhaps together we can find out where we need to clarify this more clearly in the docs.

Once a package builds successfully, it can be published to a channel. People can subscribe to a channel and select a package from that channel to use. Channels and Catalogs are two sides of the same coin and often the terms are used interchangeably. They are simply a collection of packages that have successfully built and been published. flox has a couple default channels (flox and nixpkgs-flox) as examples but you can create your own as well.

Is the statement in the doc " flox publish will create a detached catalog/<system> branch in the upstream repository ( )." under step 2 on the page Publish custom packages | flox:docs causing the confusion?

1 Like

Hi @robinbrantley,

Hi @etorreborre, I saw your blog about flox and loved it!

Glad you liked it :-).

Is the statement in the doc " flox publish will create a detached catalog/<system> branch in the upstream repository ( )." under step 2 on the page Publish custom packages | flox:docs causing the confusion?

Yes. I don’t mind keeping the two terms but it would be good to explain how they relate to each other. For example we could say that Catalog = where you publish your packages and Channel = a subscription to a given Catalogue?

My main difficulty right now is how to publish my (n) packages to a unique (1) catalog. I doesn’t seem that this is something that I can control with the current flox publish command. In that case that would force me to create as many channels as the number of packages I published.

we’ll send some clarifications on key generation shortly


thanks for this feedback. I know there have been a lot of changes behind the scenes in the CLI as we are moving to our GA so I will need to chat internally with the engineers whether things are currently in limbo or not. The intention is that you could (and should) be able to this.

1 Like

Hopefully the following will let you workaround, but it’s all still very rough:

  • You do need to setup a binary cache. I’m not sure what failure you were seeing with wiki instructions, but happy to try to help if you give more details. Another option is to use s3 as described in Serving a Nix store via S3 - Nix Reference Manual. Then pass the s3 URL as --cache-url and set your s3 creds using environment variables.

  • I was able to use a key generated with nix key generate-secret --key-name mkenigs > mkenigs-key.sec. Is that failing for you?

  • The current way to publish many packages in a single catalog is by doing something like Add capacitated telegraf to packageset by mkenigs · Pull Request #1 · etorreborre/floxpkgs · GitHub and then running flox publish in that repo (make sure your branch has an upstream, git push -u). I added a little bit of explanation for why we currently do it that way in the PR, but that’s something we haven’t nailed down, so if you have feedback on how to make that easier to use and understand, we’d love to hear it!

Hi Matthew,

For the cache I created a cache using cachix and I’d like to see if I can use it.

But my problem is the signing at the moment. I created a key like you did but I am getting this error message

Signing binary...
error: you are not privileged to add signatures
ERROR: Could not sign binary with sign-key "/Users/etorreborre/.ssh/etorreborre-key.sec"

I wonder if this is related to this issue: nix sign-paths as unprivileged user? · Issue #1708 · NixOS/nix · GitHub. Should I add myself as a trusted user on Mac? Or is this issue solved in a more recent version of Nix? I am using nix 2.17.0 but that doesn’t mean that this is what flox itself is using.


The current way to publish many packages in a single catalog

Thanks for the PR and the explanation. Unfortunately, as a flox user this raises many more questions which I don’t know how to answer :-)!

  • catalog, channel, environment and now “packageset”?

  • you talk about flox.lock. As a flox user I don’t really know what that is. It seems to be locking a bunch of dependencies related to flox itself (like shellHooks) and not to my project (no telegraf in there). I kind of understand though that this is a way to make sure that all the packages I publish more or less refer to the same dependencies when possible (and when it’s not possible what happens?)

  • then, an entry like {capacitated}: capacitated.telegraf.packages.telegraf seems a bit magical to me. I supposed that {capacited} is something that flox is expecting but how the value capacitated.telegraf.packages.telegraf is build and what each part means is, is a mystery to me

  • the other thing that is not clear to me is where do I specify a name that would be the name of the catalog for all my published packages? My expectation would be that I can reuse that name in other projects to declare “where” they are published

  • also the doc for flox publish talks about a cache and a public-cache, the difference being where to pull or where to push binaries. Should I understand that cache is where the binaries for the current packages are pushed and public-cache is where the binaries needed to build the current package are being retrieved?

I am happy to be used as guinea pig if you need someone to try out the publish workflow in real-time, so that you can see what kind of questions I ask myself while doing it.

1 Like

I believe flox is still on nix 2.15, so yes the solution would be to add yourself to trusted users

Most of those are fantastic questions, and they’re questions we need to simplify and address at some point. Unfortunately for now I think some things will remain mysterious because they aren’t really ready to be used. But I’ll try to answer as best I can.

  • catalog and channel should probably be collapsed into a single concept. The distinction between packageset and catalog/channel is that packageset is essentially source, and catalog/channel contains (references to) built binary artifacts.
  • flake.lock locks all of your nix dependencies, so in the case of telegraf that would most notably be buildGoModule, which determines your version of go used to build telegraf. The whole packageset and capacitated rigamarole would ensure that the same version of buildGoModule is used for any package in the packageset. In the case of go that’s somewhat unimportant because of static binaries, but it becomes more useful if multiple packages in the packageset both use the same binary or library dependency, e.g. libssl.
  • The capacitated stuff says to ignore the flake.lock of telegraf and instead provide the buildGoModule used by the packageset. The first telegraf says to use the telegraf flake input defined in flake.nix. Nix flakes can define multiple different types of outputs, one of which is packages. And the final telegraf is the name of the package defined in the telegraf flake. So that corresponds to github:etorreborre/telegraf#packages.{system is handled automatically}.telegraf, but with the capacitated stuff ignoring the lockfile.
  • I’m not sure if I fully understand that question - do you mean you would expect to be able to say catalog_I_publish_to = github:etorreborre/floxpkgs in your telegraf repo?
  • the difference is primarily for authentication. E.g. we use s3://... as our cache and supply credentials, but then public-cache is and doesn’t require creds.

Thanks for being so willing to try things out! Pinging @ghudgins and @ysndr as they’ll likely be involved when we do more reworking of publish.