# Publishing a package

**URL:** <https://discourse.flox.dev/t/publishing-a-package/799>\
**Category:** Help\
**Tags:** closed\
**Created:** [September 17, 2023, 11:54am UTC](https://discourse.flox.dev/t/publishing-a-package/799 "2023-09-17T11:54:37Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![etorreborre](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.flox.dev/etorreborre/32/428_2.png) [@etorreborre](https://discourse.flox.dev/u/etorreborre)\
**Post date:** [September 17, 2023, 11:54am UTC](https://discourse.flox.dev/t/publishing-a-package/799/1 "2023-09-17T11:54:37Z")

</div>

Hi,

I was trying to publish a package with a recent version of flox. I can see that `publish` is still considered experimental but I am lacking a bit of information from the documentation.

- `flox publish` requires a cache URL. I conjectured that `https://cache.nixos.org` would do the trick
- `signing-key` is less clear to me

For `signing-key` I first tried to give the path of my `~/.ssh/id_ecdsa_sk` which actually uses Yubikey. That didn’t work so I tried to generate other keys with:

- [nix key generate-secret - Nix Reference Manual](https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-key-generate-secret.html)
- and [Binary Cache - NixOS Wiki](https://nixos.wiki/wiki/Binary_Cache)

But nothing worked.

Could you please tell me how to generate a proper signing key and publish my package?

The other thing which I find confusing with the documentation about publishing is the notion of channel.

The documentation says that after publishing a package a `catalog` is created in the package github repository. Then it shows that [we can create a channel](https://flox.dev/docs/tutorials/publish-custom-packages/#flox-subscribe), more or less by subscribing to that repository. This breaks a bit my mental model since one package = one repository = one catalog = one channel but a channel is supposed to give access to several packages.

Thanks for your help!

How should those things be understood then?

---

<div class="post-metadata">

**Author:** ![ghudgins](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.flox.dev/ghudgins/32/588_2.png) [@ghudgins](https://discourse.flox.dev/u/ghudgins)\
**Post date:** [September 18, 2023, 3:06pm UTC](https://discourse.flox.dev/t/publishing-a-package/799/2 "2023-09-18T15:06:32Z")

</div>

at this point the publish user experience works for us (flox) to make our own packages available but many of the rough edges must be smoothed out before it’s ready for real world usage.

we’ll send some clarifications on key generation shortly

> The other thing which I find confusing with the documentation about publishing is the notion of channel.

that’s good feedback about some confusion between channel & catalog. multiple channels is so you can have your own catalog alongside flox’s.

---

<div class="post-metadata">

**Author:** ![robinbrantley](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.flox.dev/robinbrantley/32/39_2.png) [@robinbrantley](https://discourse.flox.dev/u/robinbrantley)\
**Post date:** [September 18, 2023, 3:17pm UTC](https://discourse.flox.dev/t/publishing-a-package/799/3 "2023-09-18T15:17:22Z")

</div>

Hi @etorreborre, I saw your blog about flox and loved it!

re the channel query, let me try to explain and perhaps together we can find out where we need to clarify this more clearly in the docs.

Once a package builds successfully, it can be published to a channel. People can subscribe to a channel and select a package from that channel to use. Channels and Catalogs are two sides of the same coin and often the terms are used interchangeably. They are simply a collection of packages that have successfully built and been published. flox has a couple default channels (flox and nixpkgs-flox) as examples but you can create your own as well.

Is the statement in the doc " `flox publish` will create a detached `catalog/<system>` branch in the upstream repository (`https://github.com/flox-examples/hello-python` )." under step 2 on the page [Publish custom packages | flox:docs](https://flox.dev/docs/tutorials/publish-custom-packages/) causing the confusion?

---

<div class="post-metadata">

**Author:** ![etorreborre](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.flox.dev/etorreborre/32/428_2.png) [@etorreborre](https://discourse.flox.dev/u/etorreborre)\
**Post date:** [September 18, 2023, 4:18pm UTC](https://discourse.flox.dev/t/publishing-a-package/799/4 "2023-09-18T16:18:46Z")

</div>

Hi @robinbrantley,

> Hi @etorreborre, I saw your blog about flox and loved it!

Glad you liked it :-).

> Is the statement in the doc " `flox publish` will create a detached `catalog/<system>` branch in the upstream repository (`https://github.com/flox-examples/hello-python` )." under step 2 on the page [Publish custom packages | flox:docs](https://flox.dev/docs/tutorials/publish-custom-packages/) causing the confusion?

Yes. I don’t mind keeping the two terms but it would be good to explain how they relate to each other. For example we could say that Catalog = where you publish your packages and Channel = a subscription to a given Catalogue?

My main difficulty right now is how to publish my (n) packages to a unique (1) catalog. I doesn’t seem that this is something that I can control with the current `flox publish` command. In that case that would force me to create as many channels as the number of packages I published.

---

<div class="post-metadata">

**Author:** ![etorreborre](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.flox.dev/etorreborre/32/428_2.png) [@etorreborre](https://discourse.flox.dev/u/etorreborre)\
**Post date:** [September 18, 2023, 4:19pm UTC](https://discourse.flox.dev/t/publishing-a-package/799/5 "2023-09-18T16:19:24Z")

</div>

> we’ll send some clarifications on key generation shortly

Thanks!

---

<div class="post-metadata">

**Author:** ![robinbrantley](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.flox.dev/robinbrantley/32/39_2.png) [@robinbrantley](https://discourse.flox.dev/u/robinbrantley)\
**Post date:** [September 18, 2023, 4:51pm UTC](https://discourse.flox.dev/t/publishing-a-package/799/6 "2023-09-18T16:51:42Z")

</div>

> [@etorreborre](#):
>
> My main difficulty right now is how to publish my (n) packages to a unique (1) catalog. I doesn’t seem that this is something that I can control with the current `flox publish` command.

thanks for this feedback. I know there have been a lot of changes behind the scenes in the CLI as we are moving to our GA so I will need to chat internally with the engineers whether things are currently in limbo or not. The intention is that you could (and should) be able to this.

---

<div class="post-metadata">

**Author:** ![mkenigs](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.flox.dev/mkenigs/32/520_2.png) [@mkenigs](https://discourse.flox.dev/u/mkenigs)\
**Post date:** [September 18, 2023, 6:16pm UTC](https://discourse.flox.dev/t/publishing-a-package/799/7 "2023-09-18T18:16:28Z")

</div>

Hopefully the following will let you workaround, but it’s all still very rough:

- You do need to setup a binary cache. I’m not sure what failure you were seeing with wiki instructions, but happy to try to help if you give more details. Another option is to use s3 as described in [Serving a Nix store via S3 - Nix Reference Manual](https://nixos.org/manual/nix/stable/package-management/s3-substituter.html). Then pass the s3 URL as `--cache-url` and set your s3 creds using environment variables.

- I was able to use a key generated with `nix key generate-secret --key-name mkenigs > mkenigs-key.sec`. Is that failing for you?

- The current way to publish many packages in a single catalog is by doing something like [Add capacitated telegraf to packageset by mkenigs · Pull Request #1 · etorreborre/floxpkgs · GitHub](https://github.com/etorreborre/floxpkgs/pull/1) and then running `flox publish` in that repo (make sure your branch has an upstream, `git push -u`). I added a little bit of explanation for why we currently do it that way in the PR, but that’s something we haven’t nailed down, so if you have feedback on how to make that easier to use and understand, we’d love to hear it!

---

<div class="post-metadata">

**Author:** ![etorreborre](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.flox.dev/etorreborre/32/428_2.png) [@etorreborre](https://discourse.flox.dev/u/etorreborre)\
**Post date:** [September 24, 2023, 12:12pm UTC](https://discourse.flox.dev/t/publishing-a-package/799/8 "2023-09-24T12:12:07Z")

</div>

Hi Matthew,

For the cache [I created a cache using cachix](https://app.cachix.org/cache/etorreborre#pull) and I’d like to see if I can use it.

But my problem is the signing at the moment. I created a key like you did but I am getting this error message

```auto
Signing binary...
error: you are not privileged to add signatures
ERROR: Could not sign binary with sign-key "/Users/etorreborre/.ssh/etorreborre-key.sec"

```

I wonder if this is related to this issue: [nix sign-paths as unprivileged user? · Issue #1708 · NixOS/nix · GitHub](https://github.com/NixOS/nix/issues/1708#issuecomment-1557083854). Should I add myself as a trusted user on Mac? Or is this issue solved in a more recent version of Nix? I am using nix 2.17.0 but that doesn’t mean that this is what flox itself is using.

Thanks.

---

<div class="post-metadata">

**Author:** ![etorreborre](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.flox.dev/etorreborre/32/428_2.png) [@etorreborre](https://discourse.flox.dev/u/etorreborre)\
**Post date:** [September 24, 2023, 12:34pm UTC](https://discourse.flox.dev/t/publishing-a-package/799/9 "2023-09-24T12:34:37Z")

</div>

> The current way to publish many packages in a single catalog

Thanks for the PR and the explanation. Unfortunately, as a flox user this raises many more questions which I don’t know how to answer :-)!

- catalog, channel, environment and now “packageset”?

- you talk about `flox.lock`. As a flox user I don’t really know what that is. It seems to be locking a bunch of dependencies related to flox itself (like `shellHooks`) and not to my project (no `telegraf` in there). I kind of understand though that this is a way to make sure that all the packages I publish more or less refer to the same dependencies when possible (and when it’s not possible what happens?)

- then, an entry like `{capacitated}: capacitated.telegraf.packages.telegraf` seems a bit magical to me. I supposed that `{capacited}` is something that flox is expecting but how the value `capacitated.telegraf.packages.telegraf` is build and what each part means is, is a mystery to me

- the other thing that is not clear to me is where do I specify a name that would be the name of the catalog for all my published packages? My expectation would be that I can reuse that name in other projects to declare “where” they are published

- also the doc for `flox publish` talks about a `cache` and a `public-cache`, the difference being where to pull or where to push binaries. Should I understand that `cache` is where the binaries for the current packages are pushed and `public-cache` is where the binaries needed to build the current package are being retrieved?

I am happy to be used as guinea pig if you need someone to try out the publish workflow in real-time, so that you can see what kind of questions I ask myself while doing it.

---

<div class="post-metadata">

**Author:** ![mkenigs](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.flox.dev/mkenigs/32/520_2.png) [@mkenigs](https://discourse.flox.dev/u/mkenigs)\
**Post date:** [September 25, 2023, 9:34pm UTC](https://discourse.flox.dev/t/publishing-a-package/799/10 "2023-09-25T21:34:33Z")

</div>

I believe flox is still on nix 2.15, so yes the solution would be to add yourself to trusted users

---

<div class="post-metadata">

**Author:** ![mkenigs](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.flox.dev/mkenigs/32/520_2.png) [@mkenigs](https://discourse.flox.dev/u/mkenigs)\
**Post date:** [September 25, 2023, 10:00pm UTC](https://discourse.flox.dev/t/publishing-a-package/799/11 "2023-09-25T22:00:28Z")

</div>

Most of those are fantastic questions, and they’re questions we need to simplify and address at some point. Unfortunately for now I think some things will remain mysterious because they aren’t really ready to be used. But I’ll try to answer as best I can.

- catalog and channel should probably be collapsed into a single concept. The distinction between packageset and catalog/channel is that packageset is essentially source, and catalog/channel contains (references to) built binary artifacts.
- `flake.lock` locks all of your nix dependencies, so in the case of telegraf that would most notably be `buildGoModule`, which determines your version of `go` used to build `telegraf`. The whole packageset and capacitated rigamarole would ensure that the same version of `buildGoModule` is used for any package in the packageset. In the case of `go` that’s somewhat unimportant because of static binaries, but it becomes more useful if multiple packages in the packageset both use the same binary or library dependency, e.g. `libssl`.
- The `capacitated` stuff says to ignore the `flake.lock` of `telegraf` and instead provide the `buildGoModule` used by the packageset. The first `telegraf` says to use the `telegraf` flake input defined in `flake.nix`. Nix flakes can define multiple different types of outputs, one of which is `packages`. And the final `telegraf` is the name of the package defined in the `telegraf` flake. So that corresponds to `github:etorreborre/telegraf#packages.{system is handled automatically}.telegraf`, but with the `capacitated` stuff ignoring the lockfile.
- I’m not sure if I fully understand that question - do you mean you would expect to be able to say `catalog_I_publish_to = github:etorreborre/floxpkgs` in your telegraf repo?
- the difference is primarily for authentication. E.g. we use `s3://...` as our `cache` and supply credentials, but then `public-cache` is `https://cache.flox.dev...` and doesn’t require creds.

Thanks for being so willing to try things out! Pinging @ghudgins and @ysndr as they’ll likely be involved when we do more reworking of publish.

---

<div class="post-metadata">

**Author:** ![etorreborre](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.flox.dev/etorreborre/32/428_2.png) [@etorreborre](https://discourse.flox.dev/u/etorreborre)\
**Post date:** [October 2, 2023, 9:51am UTC](https://discourse.flox.dev/t/publishing-a-package/799/12 "2023-10-02T09:51:44Z")

</div>

Thanks for the detailed answer.

> - I’m not sure if I fully understand that question - do you mean you would expect to be able to say `catalog_I_publish_to = github:etorreborre/floxpkgs` in your telegraf repo?

Maybe not in the repository. Maybe just at publishing time.

> [@mkenigs](#):
>
> Thanks for being so willing to try things out! Pinging @ghudgins and @ysndr as they’ll likely be involved when we do more reworking of publish.

I think that polishing “publish” is super important because it is the escape hatch when something is not available in the standard nix packages (possibly intentionally if this is some kind of private/internal package).

---

<div class="post-metadata">

**Author:** ![mkenigs](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.flox.dev/mkenigs/32/520_2.png) [@mkenigs](https://discourse.flox.dev/u/mkenigs)\
**Post date:** [October 2, 2023, 4:30pm UTC](https://discourse.flox.dev/t/publishing-a-package/799/13 "2023-10-02T16:30:00Z")

</div>

> Maybe not in the repository. Maybe just at publishing time.

Yeah that’s something to keep in mind - we currently only support publishing to the repo a package is defined in. That encourages creating a packageset so that all packages are cohesive. But I agree it’s a bit surprising that you can’t specify an alternative channel name to publish to.

---

<div class="post-metadata">

**Author:** ![robinbrantley](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.flox.dev/robinbrantley/32/39_2.png) [@robinbrantley](https://discourse.flox.dev/u/robinbrantley)\
**Post date:** [October 2, 2023, 5:10pm UTC](https://discourse.flox.dev/t/publishing-a-package/799/14 "2023-10-02T17:10:55Z")

</div>

> [@etorreborre](#):
>
> I think that polishing “publish” is super important because it is the escape hatch when something is not available in the standard nix packages (possibly intentionally if this is some kind of private/internal package).

Agreed 🙂  
We definitely will be revisiting `flox publish` in light of the changes we are currently making to the CLI. The current estimate is early Q1 but if we can get through things faster, hopefully we can get to it sooner!  
In the meantime hopefully Matthew’s notes will give you a workaround.