Am I holding `flox develop` wrong?

Developing with flox
1

I’ve played with flox enough that I’m looking to integrate it into some of our open source projects as a test bed where we already have other methods of local development present (devcontainers, Docker). Once others have access to flox, this will give me an idea on whether we can nudge more projects onto it and what complexity would be involved for new contributors.

The project is GitHub - cloudflare/cloudflare-go: Go library for the Cloudflare v4 API. For now, I’m aiming to get only a development environment working, not a full flake for others to consume. With that in mind, I initialise a new floxEnv.

$ flox init
wrote: /Users/jacob/go/src/github.com/cloudflare/cloudflare-go/flake.nix
wrote: /Users/jacob/go/src/github.com/cloudflare/cloudflare-go/flox.toml

Contents look good

$ cat flake.nix
{
  description = "Floxpkgs/Project Template";
  nixConfig.bash-prompt = "[flox] \\[\\033[38;5;172m\\]λ \\[\\033[0m\\]";
  inputs.flox-floxpkgs.url = "github:flox/floxpkgs";

  # Declaration of external resources
  # =================================

  # =================================

  outputs = args @ {flox-floxpkgs, ...}: flox-floxpkgs.project args (_: {});
}

$ cat flox.toml
# Note: the format for this file is subject to change

# You can add arbitrary bash commands to setup your environment here
postShellHook = '''
    export VAR=foo
'''

[programs.catalog]
# You can specify plain package names
coreutils = {}
# or add a version specifier (make sure to add quotes because of the "@")
"gnumake@4.3" = {}

Before I go messing with the Nix and more packages, I attempt to run flox develop to confitm it works but I hit this error.

$ flox develop
error: flake 'git+file:///Users/jacob/go/src/github.com/cloudflare/cloudflare-go' does not provide attribute 'packages.x86_64-darwin.packages.x86_64-darwin', 'legacyPackages.x86_64-darwin.packages.x86_64-darwin' or 'packages.x86_64-darwin'

ERROR: cannot find attribute path - have you run 'flox init'?

warning: the flag '--override-input floxpkgs/nixpkgs/nixpkgs flake:nixpkgs-stable' does not match any input
error: flake 'git+file:///Users/jacob/go/src/github.com/cloudflare/cloudflare-go' does not provide attribute 'devShells.x86_64-darwin.default', 'devShell.x86_64-darwin', 'packages.x86_64-darwin.default' or 'defaultPackage.x86_64-darwin'
       Did you mean devShells?

Is this a misunderstanding on my behalf how the flox develop should work out of the box? Or is it a bug?

2

Hi @jacob,

Thank you for letting us know - we’re working hard to improve the process and your feedback is absolutely essential for making that happen!

Latest & greatest

First thing first, the results of your flox init invocation suggests you may be using an old version of flox? Can you append the output of flox --version?

In general I’d suggest following the upgrade instructions often to stay up with the latest revision, but you may find it more convenient to use the following trick to install the latest flox with flox. :slight_smile:

flox pull -e flox/default  # and then accept the default prompts
flox activate -e flox/default
flox --version  # as of now should return 0.0.7-r93

https://github.com/flox-examples/cloudflare-go

Thanks for providing a pointer to your source - it makes it so much easier to work through an example when we can work with the same project. I made a fork of your package to github:flox-examples/cloudflare-go and you can cherry-pick commit a5bb93b to replicate everything you see in this append.

Note that all of the steps in this posting are listed in the git log for the above commit.

Building your package(s)

I can almost hear you say “but I only want a developer environment”, and while that is certainly understandable the thing to keep in mind with flox is that the developer environment is comprised of a build environment with layers of extra tooling thrown in on top, so it’s easier to start with a successful build and add on from there.

cloudflare-go

It’s customary to name the flox/Nix package after the repository itself, and the following will create the flox expression for this package:

  1. invoke flox init

    1. select buildGoModule() template, accept defaults

  2. edit pkgs/cloudflare-go/default.nix

    1. set version to "0.57.1" manually (this will be automatically inferred from idiomatic conventions in future)
    2. then update version to "0.57.1-${getRev src}" if you want the getRev() function to automatically append revision count to version number (this will be included in the default template very soon)
    3. uncomment vendorSha256 = lib.fakeSha256 line and run flox build to download and identify checksum of package containing vendored dependencies, then set vendorSha256 accordingly (this will be automated in future with the new flox tidy command)
    4. add the line: subPackages = [ "." "cmd/flarectl" ];

That last line (2.4) is necessary because the build was tripping over a problem building the internal/tools subPackage, and setting this allowed Nix to skip that part. This by far was the hardest part of this exercise, requiring some Nix spelunking skills.

cloudflare-go-internal-tools

As mentioned above the build of the main package was not able to build the internal/tools subpackage, and I couldn’t figure out how to fix that (sorry I’m not a go developer!), so I instead created a second flox expression within your repository to build that subPackage if you need it. (If you don’t need it, or if you can get it building from within the main package then we can delete this.)

  1. run flox init again

    1. select buildGoModule() template, change package name in prompt to cloudflare-go-internal-tools

  2. edit pkgs/cloudflare-go-internal-tools/default.nix

    1. perform same updates as 2.1, 2.2, 2.3 above
    2. update the definition of src to be self + "/internal/tools"

Building your package(s)

Now that you have flox expressions for your packages, building should be as simple as running flox build, but sadly:

[brantley@clubsoda:~/src/cloudflare-go]$ flox build
Select package for flox build
> cloudflare-go
  cloudflare-go-internal-tools
HINT: avoid selecting a package next time with:
$ flox build -A cloudflare-go
error: builder for '/nix/store/yffhqcm20gcxmvh3nmsbybpl7w4c2rgx-cloudflare-go-0.57.1-dirty.drv' failed with exit code 1;
       last 10 log lines:
       > --- PASS: TestUpdateZoneSSLSettings (0.00s)
       > === RUN   TestError_CreateErrors
       > --- PASS: TestError_CreateErrors (0.00s)
       > === RUN   ExampleDuration
       > --- PASS: ExampleDuration (0.00s)
       > === RUN   ExampleLogpushJob_MarshalJSON
       > --- PASS: ExampleLogpushJob_MarshalJSON (0.00s)
       > FAIL
       > FAIL   github.com/cloudflare/cloudflare-go     3.269s
       > FAIL
       For full logs, run 'nix log /nix/store/yffhqcm20gcxmvh3nmsbybpl7w4c2rgx-cloudflare-go-0.57.1-dirty.drv'.

Oh no! Looks like the package built but the unit tests did not pass. That last line provides a helpful hint for accessing the failed build logs, and looking through that we can get more detail of what failed:

[brantley@clubsoda:~/src/cloudflare-go]$ flox nix log /nix/store/yffhqcm20gcxmvh3nmsbybpl7w4c2rgx-cloudflare-go-0.57.1-dirty.drv | grep -i fail
                                HTTP request failed: Get "https://developers.cloudflare.com/ssl/static/origin_ca_ecc_root.pem": dial tcp: lookup developers.cloudflare.com on [::1]:53: read udp [::1]:45278->[::1]:53: read: connection refused
                                HTTP request failed: Get "https://developers.cloudflare.com/ssl/static/origin_ca_rsa_root.pem": dial tcp: lookup developers.cloudflare.com on [::1]:53: read udp [::1]:49159->[::1]:53: read: connection refused
--- FAIL: TestOriginCA_OriginCARootCertificate (0.00s)
=== RUN   TestListZonesFailingPages
--- PASS: TestListZonesFailingPages (0.03s)
FAIL
FAIL    github.com/cloudflare/cloudflare-go     3.269s
FAIL

This makes complete sense - the Nix build sandbox is completely isolated from the network so that test cannot succeed. We’ll simply need to disable that particular test in the Nix build.

Disabling tests in the Nix sandbox

Again a disclaimer: I am most definitely not a go developer so this is probably not the best way to do it, but the following patch was able to disable the one TestOriginCA_OriginCARootCertificate test and allow the build to succeed:

diff --git a/origin_ca_test.go b/origin_ca_test.go
index 0051080..b524906 100644
--- a/origin_ca_test.go
+++ b/origin_ca_test.go
@@ -6,6 +6,7 @@ import (
        "encoding/pem"
        "fmt"
        "net/http"
+       "os"
        "testing"
        "time"
 
@@ -224,7 +225,14 @@ func TestOriginCA_RevokeCertificate(t *testing.T) {
        }
 }
 
+func skipNix(t *testing.T) {
+       if os.Getenv("NIX_BUILD_TOP") != "" {
+               t.Skip("Skipping testing in Nix sandbox")
+       }
+}
+
 func TestOriginCA_OriginCARootCertificate(t *testing.T) {
+       skipNix(t)
        setup()
        defer teardown()

… and then following the addition of that patch the build succeeds and the package seems to work:

[brantley@clubsoda:~/src/cloudflare-go]$ flox build -A cloudflare-go

[brantley@clubsoda:~/src/cloudflare-go]$ result/bin/flarectl 
NAME:
   flarectl - Cloudflare CLI

USAGE:
   flarectl [global options] command [command options] [arguments...]

VERSION:
   dev

COMMANDS:
   ips, i                     Print Cloudflare IP ranges
   user, u                    User information
   zone, z                    Zone information
   dns, d                     DNS records
   user-agents, ua            User-Agent blocking
   pagerules, p               Page Rules
   railgun, r                 Railgun information
   firewall, f                Firewall
   origin-ca-root-cert, ocrc  Print Origin CA Root Certificate (in PEM format)
   help, h                    Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --account-id value  Optional account ID [$CF_ACCOUNT_ID]
   --json              show output as JSON instead of as a table (default: false)
   --help, -h          show help (default: false)
   --version, -v       print the version (default: false)

Did someone say developer environment?

So now, I think this is what you were looking for:

[brantley@clubsoda:~/src/cloudflare-go]$ flox develop -A cloudflare-go                                                                                                                                                            
[flox] λ go mod vendor
go: downloading github.com/urfave/cli/v2 v2.23.7
go: downloading github.com/olekukonko/tablewriter v0.0.5 
go: downloading github.com/hashicorp/go-retryablehttp v0.7.1
go: downloading golang.org/x/time v0.0.0-20220224211638-0e9765cccd65
go: downloading golang.org/x/net v0.0.0-20220722155237-a158d28d115b
go: downloading github.com/stretchr/testify v1.8.1
go: downloading github.com/mattn/go-runewidth v0.0.13
go: downloading gopkg.in/yaml.v3 v3.0.1
go: downloading github.com/rivo/uniseg v0.2.0
go: downloading github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673
go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.2
go: downloading github.com/russross/blackfriday/v2 v2.1.0
[flox] λ go build
[flox] λ

flox publish and more …

On the path to a developer environment we created a package, and we know we can build it with flox build, but how can we install and use it in a flox environment?

This is where flox publish comes into the picture, and you can use this to safely make precompiled signed and checksummed versions of your software easily available for customers to download (exactly as we make flox itself available for download in the flox/default environment!).

I did exactly that as an exercise for this project, publishing the result to the github:flox-examples/floxpkgs repository - a transcript of the process is included below:

[brantley@clubsoda:~/src/cloudflare-go]$ flox publish
Select package for flox publish
> cloudflare-go
  cloudflare-go-internal-tools

HINT: avoid selecting a package next time with:
$ flox publish -A cloudflare-go
build repository: https://github.com/flox-examples/cloudflare-go                                                                                                                                                                  
package name: cloudflare-go
channel repository: git@github.com:flox-examples/floxpkgs                                                                                                                                                                         
binary cache for upload: s3://flox-store-public                                                                                                                                                                                   
upload to: s3://flox-store-public
binary cache for download: https://cache.floxdev.com                                                                                                                                                                              
download from: https://cache.floxdev.com
HINT: avoid having to answer these questions next time with:
$ flox publish -A cloudflare-go --build-repo https://github.com/flox-examples/cloudflare-go --channel-repo git@github.com:flox-examples/floxpkgs --upload-to s3://flox-store-public --download-from https://cache.floxdev.com
Cloning git@github.com:flox-examples/floxpkgs ...
Cloning into '/tmp/tmp.ILZ3PPvN8p'...
remote: Enumerating objects: 446, done.
remote: Counting objects: 100% (129/129), done.
remote: Compressing objects: 100% (83/83), done.
remote: Total 446 (delta 45), reused 106 (delta 23), pack-reused 317
Receiving objects: 100% (446/446), 88.79 KiB | 502.00 KiB/s, done.
Resolving deltas: 100% (143/143), done.
Building cloudflare-go ...
publishing render to catalog ...
flox publish completed
[master 1dbcff6] published x86_64-linux/stable/cloudflare-go/0.57.1-r2085.json
 1 file changed, 12 insertions(+), 12 deletions(-)
Enumerating objects: 13, done.
Counting objects: 100% (13/13), done.
Delta compression using up to 8 threads
Compressing objects: 100% (5/5), done.
Writing objects: 100% (7/7), 919 bytes | 919.00 KiB/s, done.
Total 7 (delta 3), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (3/3), completed with 3 local objects.
To github.com:flox-examples/floxpkgs.git
   d496671..1dbcff6  master -> master

… and having done that, users/customers can find and install this package with the following:

[brantley@clubsoda:~/src/cloudflare-go]$ flox subscribe flox-examples
Enter URL for 'flox-examples' channel: github:flox-examples/floxpkgs/master

[brantley@clubsoda:~/src/cloudflare-go]$ flox search -c flox-examples cloudflare-go
flox-examples.cloudflare-go
  stable.flox-examples.cloudflare-go@0.57.1-r2085

[brantley@clubsoda:~/src/cloudflare-go]$ flox install -e cloudflare-test flox-examples.cloudflare-go
created generation 1

[brantley@clubsoda:~/src/cloudflare-go]$ flox activate -e cloudflare-test -- flarectl --version
flarectl version dev

I imagine that version “dev” is not what we’re looking for here, but I have no doubt that you will know where that value is coming from. :slight_smile:

Hope this helps!

My apologies that this whistle-stop tour was somewhat light on background detail, but hopefully useful because it’s tailored to your specific example. Please let us know how you get on!

3

Thanks for your extensive and informative answer @limeytexan! I’ve added notes below for your questions.

Despite running on the prerelease, I haven’t been very disciplined with keeping it up to date (as you correctly noticed!). I’m hoping incremental changes like this will force me to do it more regularly. Versions:

  • Non-prerelease: 0.0.7-r92
  • Prerelease: 0.0.7-r106

When interacting with flox, is the intention that you have flox-ception (flox operations from in a flox environment) and you only use the system flox to upgrade itself?

$ which flox
/nix/var/nix/profiles/default/bin/flox

$ flox activate -e flox/default -- which flox
/Users/jacob/.local/share/flox/environments/flox/default/bin/flox

This is excellent context! The primary driver here was my lack of experience with Nix packaging as a whole. My assumption was that it would be easier to get a developer environment up and then look at how the packaging side works but it’s good to know they work hand-in-hand for this use case.

internal/tools is a convention within Go that allows to include your local development tooling in the repository itself but exclude it from the built artifact. In our case, it is invoked as go generate -tags tools internal/tools.go. The intention is that this will go away with flox once it is generally available or at least be part of the build environment itself.

TIL! This is great to know both from the existing tests perspective but also when using the Nix build sandbox itself so thank you.

Heh, yes. In our case, it uses Go ldflags to inject the version at build time which is something I’m sure we can bake into the build pipeline soon enough.

No apology needed; this was a great explanation on the whys and how things are intended to work, so I appreciate it immensely.

One final clarification, I was reading https://beta.floxdev.com/docs/using-flox/managing-project-environments/#bootstrap-the-declarative-environment and I was wondering, is floxEnv then more suited to say, running a dedicated environment for different versions of a tool? Example would be if I needed to run 3 different versions of kubectl where I wouldn’t actually be intending to build a package, just run it isolation.

4

This is a great question that we’ve had a few times so I wanted to answer it separately so that we can include links to this response in documentation, etc.

This is something we’ve put on the back burner for the short term as we work to port the CLI from the original (bash) “MVP” to the new rust rewrite, but the eventual goal is for the “system-installed” flox to:

  • automatically pull the flox/default environment if it hasn’t been pulled already
  • at some configurable interval (which can be zero to disable the feature), perform a background git fetch for the flox/default profile and warn or prompt users to perform an upgrade
  • perform various “sanity checks” such as highlighting when the installed version of Nix is too old, and if necessary warn people to perform a “system upgrade” of flox
  • finally, finish by invoking exec path/to/flox/default/bin/flox @argv

With this approach users will always invoke flox from the flox/default environment (even if it is not in their PATH), which will always be at or newer than the version installed in the “system”, but that can be rolled back as necessary if someone wants to explicitly pop back to use a previous version.

5

Another excellent question. I’ll start by noting that particular document is due to be revised for flox version 0.0.8 and above, so please bear with us as we work to get that updated.

From flox version 0.0.8 and above flox environments are rendered using the same underlying implementation, and with that the differences between “project” and “named” flox environments are summarized in the following table:

type: named project
maintained in: “floxmeta” repository project repository
supports history/rollback: yes no
activate with: flox activate flox develop
performs Nix instantiation: no yes
used for: CI/testing & production development

Or in words:

  • flox environments are collections of already-built packages with logic for running hooks and setting variables & aliases.
  • Named flox environments are maintained in an “offline” repository entirely managed by the flox CLI, benefit from atomic upgrades and rollbacks, and are activated with flox activate. These are best used in production and testing environments where you only want to use packages, much like you would a container.
  • project-based flox environments are maintained alongside source code in project repositories and activated with flox develop. This command builds and activates the flox environment, and finishes by exec’ing nix develop to layer the builder’s own development environment on top. These are only used when developing a specific project, and help tremendously with the process of developer onboarding.

You can add the tooling you need to a named environment or to a project-based environment based on your needs. One rule of thumb is that a tool you need for all projects (like an editor) should go into a named environment, but a tool required for just one project (like a version of vscode optimized with a bunch of project-specific plugins) should go into that project’s environment.

Hope this helps!

2 Likes
6

Good to know. This is how I envisioned it would eventually be working but was missing the inbetween stage that we are now that was a little confusing at first.

This for me was the :bulb: moment – again, probably due to my lack of understanding on the packaging side given I was trying to use floxEnv more as a named environment, not a project as you’ve outlined above.

Major kudos for the extremely thorough explanations. They are awesome!

3 Likes
7

Ah, this is the reason I joined this discourse. A co-worker showed me flox briefly and I decided to try it out, and this is the same question I had.

So- I am not at all interested in writing nix derivations. Like you said, I wanted something like a dev container- a straightforward way to get the tools I need for my build to work. Maybe its a prejudice- I like nix as a result just fine, but spent enough time fiddling with the config language to say I never want to do that again.

So- if I can register a vote for a development environment that is just “a list of packages to install to my environment so that I can run the build/test commands like normal” I’d like to do so :slight_smile:

3 Likes
8

thanks for your feedback Paul!

Stay tuned… we hope to get something out on this front in the next 1-2 weeks… :slight_smile: