Thanks for the feedback - looks like you got pretty far! I’ll take a closer look in the morning but will briefly mention that we understand the publish workflow needs improvement, so we’re actively working to make that better. I can’t say when we’ll get it finished but we’ll be sure to get you a working solution for the above example ASAP.
The trace should be unrelated - it’s just a warning that we only want when --verbose is set. Did you know the install failed because cmt wasn’t showing up with flox list?
Thanks, I just tried again and I think I was mislead by the trace message. It seems to work ok.
This morning I also ran a search and got a similar message:
± flox search -c etorreborre cmt
trace: Not importing any attributes because the directory /nix/store/yz4r85v19rn2304p95vwj47w2cqklask-source/pkgs/ doesn't exist
trace: Not importing any attributes because the directory /nix/store/yz4r85v19rn2304p95vwj47w2cqklask-source/pkgs/ doesn't exist
trace: Not importing any attributes because the directory /nix/store/yz4r85v19rn2304p95vwj47w2cqklask-source/pkgs/ doesn't exist
etorreborre.cmt - Write consistent git commit messages
stable.etorreborre.cmt@0.7.1.0
Maybe it should only be there when --verbose is set then?
@etorreborre if you have a moment would you be able to run through this entire workflow again with version 0.0.7, including the flox publish part? I forked your repository and it worked for me after running flox flake update. (Feel free to pull the latest updates from https://github.com/limeytexan/cmt if you prefer!)
the new UI is great! Lots more explanations, Yes / No button, I really like it
I wonder about the consistency of the “channel repository:” input when doing flox publish. I need to specify a git URL there like git@github.com:etorreborre/floxpkgs.git. On the other hand, if I go flox channels the URL says etorreborre github:etorreborre/floxpkgs/main (which would not work)
still on the URL note, but possibly out of flox control, I’m using URL that are like etorreborre:etorreborre/floxpkgs and work:etorreborre/work-repository and in my .ssh/config file I have a mapping so that I can use different keys for different github users. Those URLs will not work with flox publish
Now the problem I’m seeing is:
± flox publish default
build repository: git@github.com:etorreborre/cmt.git
package name: cmt
channel repository: git@github.com:etorreborre/floxpkgs.git
HINT: avoid having to answer these questions next time with:
$ flox publish -A cmt --build-repo git@github.com:etorreborre/cmt.git --channel-repo git@github.com:etorreborre/floxpkgs.git
Cloning git@github.com:etorreborre/floxpkgs.git ...
Cloning into '/var/folders/kb/tjt27xcn6_dg095y88337mjm0000gn/T/tmp.xm4dMBtVYb'...
remote: Enumerating objects: 56, done.
remote: Counting objects: 100% (56/56), done.
remote: Compressing objects: 100% (36/36), done.
remote: Total 56 (delta 7), reused 49 (delta 3), pack-reused 0
Receiving objects: 100% (56/56), 10.85 KiB | 3.62 MiB/s, done.
Resolving deltas: 100% (7/7), done.
Building cmt ...
error: Package ‘cmt-1.17’ in /nix/store/9m8drnpifyl5qsx93g6ll2xw6wkps03z-source/pkgs/applications/audio/cmt/default.nix:27 is not supported on ‘x86_64-darwin’, refusing to evaluate.
a) To temporarily allow packages that are unsupported for this system, you can use an environment variable
for a single invocation of the nix tools.
$ export NIXPKGS_ALLOW_UNSUPPORTED_SYSTEM=1
Note: For `nix shell`, `nix build`, `nix develop` or any other Nix 2.4+
(Flake) command, `--impure` must be passed in order to read this
environment variable.
b) For `nixos-rebuild` you can set
{ nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.
c) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
{ allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.
(use '--show-trace' to show detailed location information)
ERROR: could not build git+ssh://git@github.com/etorreborre/cmt.git?rev=3cf77bee56e4b079dbb54b513482249db4ab8dd7#cmt
The weird message is this
Package ‘cmt-1.17’ in /nix/store/9m8drnpifyl5qsx93g6ll2xw6wkps03z-source/pkgs/applications/audio/cmt/default.nix:27
it mentions applications/audio/cmt. That seems like a conflict with this package.
Good spot! Yes I think you’re right that appears to be a namespace collision that definitely shouldn’t be happening. We’ll put together a reproducer and get back to you on that asap.
Url issue 1: The inconsistencies with the urls stems from the fact that they are used by different subsystems.
The github:/ style is a Nix shorthand that uses the github (http) api instead of the git api.
Functionally that does not make a huge difference for Nix Bar some performance and authentication details.
The channel-url is handled by flox, has to clone, modify and push the names repository. Hence, the shorthand style does not work here.
We will try to be clearer in our descriptions for these things.
Second url issue:
Since you’re using your urls as got Hosts (i assume) channel-url should be able to work with etorreborre:etorreborre/floxpkgs already.
For build-url and other places you could try adding git+ssh:// at the start ie
git+ssh://etorreborre:etorreborre/floxpkgs
Without a protocol we cannot decide that this url is in fact a git/ssh url.
Mind that ssh based urls should pick up your existing ssh access config correctly
//
Regarding your cmt issue:
Your right we’re picking up nixpkgs‘ cmt.
Please make sure to update your flake to use the latest floxpkgs by deleting the flake.lock file or running
flox nix flake lock —update-input floxpkgs
Please report if the issue persists that is something I’d need to investigate
Then I tried to update the flake.lock file (I used flox nix flake lock —update-input floxpkgs) but I still got
$ flox publish -A cmt --build-repo git@github.com:etorreborre/cmt --channel-repo git@github.com:etorreborre/floxpkgs
Cloning git@github.com:etorreborre/floxpkgs ...
Cloning into '/var/folders/kb/tjt27xcn6_dg095y88337mjm0000gn/T/tmp.CZEdnB7tvr'...
remote: Enumerating objects: 56, done.
remote: Counting objects: 100% (56/56), done.
remote: Compressing objects: 100% (36/36), done.
remote: Total 56 (delta 7), reused 49 (delta 3), pack-reused 0
Receiving objects: 100% (56/56), 10.85 KiB | 132.00 KiB/s, done.
Resolving deltas: 100% (7/7), done.
Building cmt ...
warning: not writing modified lock file of flake 'git+ssh://git@github.com/etorreborre/cmt?rev=3cf77bee56e4b079dbb54b513482249db4ab8dd7':
• Updated input 'floxpkgs/nixpkgs/nixpkgs':
'github:flox/nixpkgs/f994293d1eb8812f032e8919e10a594567cf6ef7' (2022-10-25)
→ 'github:flox/nixpkgs/2788904d26dda6cfa1921c5abb7a2466ffe3cb8c' (2022-11-22)
error: Package ‘cmt-1.17’ in /nix/store/9m8drnpifyl5qsx93g6ll2xw6wkps03z-source/pkgs/applications/audio/cmt/default.nix:27 is not supported on ‘x86_64-darwin’, refusing to evaluate.
a) To temporarily allow packages that are unsupported for this system, you can use an environment variable
for a single invocation of the nix tools.
$ export NIXPKGS_ALLOW_UNSUPPORTED_SYSTEM=1
Note: For `nix shell`, `nix build`, `nix develop` or any other Nix 2.4+
(Flake) command, `--impure` must be passed in order to read this
environment variable.
b) For `nixos-rebuild` you can set
{ nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.
c) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
{ allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.
(use '--show-trace' to show detailed location information)
ERROR: could not build git+ssh://git@github.com/etorreborre/cmt?rev=3cf77bee56e4b079dbb54b513482249db4ab8dd7#cmt
@etorreborre we completely agree that these URL prompts are confusing (as are the Nix conventions that @ysndr explained above) and our aim is to present a simple and recognizable “canonical/normalized” format to the user in all interactions and then convert to the various Nix conventions as and when required. Let us think about that one and get back to you.
Regarding your publish command, I just tried a build and publish of https://github.com/limeytexan/cmt on x86_64-linux and again it worked as expected. BUT … I tried again on x86_64-darwin and it failed with the error shown above. With that I think we have a reproducer to work with, and rest assured we’ll get back to you on that as well.
One other hint: try doing a flox build before attempting a flox publish. The latter runs the former automatically as part of doing a publish, and it helps to disambiguate the things that can go wrong by running the various steps separately.
FWIW I published my build of the “mycmt” package to https://github.com/limeytexan/floxpkgs (for x86_64-darwin only) and you should be able to subscribe to and install it with these commands:
In this example I copied the built package (closure) to the s3 cache which backs https://cache.floxdev.com, but even without using the binary cache the above commands should be sufficient for anyone to build and install the package to a flox environment. Let me know if that doesn’t work for you. Thanks!
BTW that application closure is nearly 3GB, and I note this is most likely because it contains the compiler. That doesn’t sound right - @ysndr can you look at this example to see why that might be the case?
I tried your suggestion (and renamed cmt to commit-tool):
± flox publish default etorreborre/default
build repository: git@github.com:etorreborre/cmt
package name: commit-tool
channel repository: git@github.com:etorreborre/floxpkgs
HINT: avoid having to answer these questions next time with:
$ flox publish -A commit-tool --build-repo git@github.com:etorreborre/cmt --channel-repo git@github.com:etorreborre/floxpkgs
Cloning git@github.com:etorreborre/floxpkgs ...
Cloning into '/var/folders/kb/tjt27xcn6_dg095y88337mjm0000gn/T/tmp.6JvxS0z55r'...
remote: Enumerating objects: 56, done.
remote: Counting objects: 100% (56/56), done.
remote: Compressing objects: 100% (36/36), done.
remote: Total 56 (delta 7), reused 49 (delta 3), pack-reused 0
Receiving objects: 100% (56/56), 10.85 KiB | 326.00 KiB/s, done.
Resolving deltas: 100% (7/7), done.
Building commit-tool ...
warning: not writing modified lock file of flake 'git+ssh://git@github.com/etorreborre/cmt?rev=3cf77bee56e4b079dbb54b513482249db4ab8dd7':
• Updated input 'floxpkgs/nixpkgs/nixpkgs':
'github:flox/nixpkgs/f994293d1eb8812f032e8919e10a594567cf6ef7' (2022-10-25)
→ 'github:flox/nixpkgs/2788904d26dda6cfa1921c5abb7a2466ffe3cb8c' (2022-11-22)
error: flake 'git+ssh://git@github.com/etorreborre/cmt?rev=3cf77bee56e4b079dbb54b513482249db4ab8dd7' does not provide attribute 'packages.x86_64-darwin.commit-tool', 'legacyPackages.x86_64-darwin.commit-tool' or 'commit-tool'
ERROR: could not build git+ssh://git@github.com/etorreborre/cmt?rev=3cf77bee56e4b079dbb54b513482249db4ab8dd7#commit-tool
Seeing the revision number, I then did a git push and then publishing worked ok. So my understanding is that the remote branch should always be flush with the local one because what we are effectively publishing is the remote code, right?
the remote branch should always be flush with the local one because what we are effectively publishing is the remote code, right?
Exactly. You’re publishing all the information necessary for someone else (or you on another machine) to be able to rebuild that package, and for that to work you have to cite the source of your package as somewhere that’s generally reachable from everywhere, and the local clone in your home directory probably isn’t.
