How to install a couple python packages?

I’d like to use gdal_calc to do geographic processing.

With plain-old nix-shell I can do this:

nix-shell -p "python3.withPackages(p: [p.numpy p.gdal])" --run " --help"`

or create an equivalent shell.nix file like this:

{ pkgs ? import <nixpkgs> {} }:

pkgs.mkShell {
  buildInputs = [
    (pkgs.python3.withPackages(p: [p.numpy p.gdal]))

How can I achieve the same with flox?


Hi, this would be one way to do it:

first set up an environment with:
flox install -e gdalEnv python3 nixpkgs#python3Packages.numpy nixpkgs#python3Packages.gdal
then run a command without entering that environment:
flox activate -e gdalEnv -- python

Effectively your environment because your shell.nix file, but you can activate it in any directory.
I think there’s a better way to do this which I will follow up on if I find it.

Hi @paulovieira - we are currently working on an “inline packaging” feature that will make it super easy to create arbitrary python “withPackages” environments, but in the meantime you can create your own “gdalPython” package that can be installed to an environment like any other. This is probably a good workflow to be aware of in any case and not to worry - with flox this is not nearly as hard as it sounds!

Using a custom package

I’ll start by first describing how to use a custom package, and then we’ll work backwards to explain how to create one.

I’ve taken the liberty to create a gdalPython package that meets your specifications in the github:flox-examples/floxpkgs repository, and here’s how you can use it:

brantley@Michaels-MBP flox-examples % flox subscribe flox-examples github:flox-examples/floxpkgs
brantley@Michaels-MBP flox-examples % flox install -e gdal flox-examples#gdalPython
created generation 1
brantley@Michaels-MBP flox-internal % flox activate -e gdal
flox [gdal default] brantley@Michaels-MBP flox-internal % 

From this shell you can then use any of the gdal programs, or a version of python equipped with all the modules you need:

flox [gdal default] brantley@Michaels-MBP flox-internal % gdalpython
Python 3.10.8 (main, Nov 13 2022, 14:51:22) [Clang 11.1.0 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> from osgeo import gdal
flox [gdal default] brantley@Michaels-MBP flox-internal % --help |& head -3
usage: [--help] --calc [expression ...] [-A [filename ...]]
                    [--A_band [n ...]] --outfile filename
                    [--NoDataValue value] [--hideNoData] [--type datatype]
flox [gdal default] brantley@Michaels-MBP flox-internal % 

Easy, eh? But how do you create your own custom package in your own package collection?

Creating your own floxpkgs collection

We’re thinking about better ways to do this in future, but at present the way to create your own floxpkgs collection is to publish something to it. It doesn’t matter what that is, so in this case I’ll publish the gdalPython package from above to a new limeytexan/floxpkgs repository:

flox [gdal default] brantley@Michaels-MBP flox-internal % flox publish flox-examples#gdalPython
package name: gdalPython
+ /nix/store/qvfslp3zix6zv2bl56lgbw9dld9zpw8q-flox-0.0.7-r76/libexec/flox/gh repo create --public --template
✓ Created repository limeytexan/floxpkgs on GitHub
channel repository:
HINT: avoid having to answer these questions next time with:
$ flox publish -A gdalPython --build-repo --channel-repo
Cloning ...
Cloning into '/var/folders/_s/s7vwf8td57xbnh5h2styht2r0000gn/T/tmp.OhiFhJAg71'...
remote: Enumerating objects: 7, done.
remote: Counting objects: 100% (7/7), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 7 (delta 0), reused 5 (delta 0), pack-reused 0
Receiving objects: 100% (7/7), done.
Building gdalPython ...
publishing render to catalog ...
flox publish completed
[main 65b3067] published aarch64-darwin/stable/gdalPython/3.10.8-env.json
 1 file changed, 111 insertions(+)
 create mode 100644 catalog/aarch64-darwin/stable/gdalPython/3.10.8-env.json
Enumerating objects: 9, done.
Counting objects: 100% (9/9), done.
Delta compression using up to 10 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (7/7), 1.93 KiB | 1.93 MiB/s, done.
Total 7 (delta 0), reused 0 (delta 0), pack-reused 0
   7f90744..65b3067  main -> main
flox [gdal default] brantley@Michaels-MBP flox-internal % 

Let’s step through all that happened above:

  1. flox publish asks for a) the URL from which the package can be built, and b) the URL where it should publish the build results. In this case we can build gdalPython from and I’m wanting to publish to my own floxpkgs repository, which it will automatically create for you in the process (from the floxpkgs-template).
  2. It then asks for the URLs where you might want to copy the built package to (and download from) as part of the publish process. This is entirely optional and meant just to speed up installations for others, so you can just hit <enter> to skip those fields for now.
  3. It then builds the package and writes all of the metadata required for others to reproduce it to your floxpkgs repository as provided above.

If you do all that yourself (creating the pauloviera/floxpkgs repository in the process), you will now have a new way to use that package!

flox subscribe pauloviera github:pauloviera/floxpkgs
flox install -e gdal pauloviera.gdalPython

But that’s still not quite the whole story because you need to be able to update that package over time, so let’s add that gdalPython package to your collection.

Adding a custom package to your collection

In this case you can just copy the example I created, but it’s a similar (and similarly straightforward) process to create packages for other languages. This should do the trick:

git clone
cd floxpkgs
mkdir pkgs/gdalPython
wget -O pkgs/gdalPython/default.nix
git add pkgs/gdalPython/default.nix
git commit -am "gdalPython: init"
git push

… and following those steps you will be able to install or publish directly from your floxpkgs clone. Try invoking flox publish from your clone to see what I mean.


If you’ve made it this far it wouldn’t surprise me if this wasn’t still a bit confusing, and in particular you may be wondering about the difference between pauloviera#gdalPython and pauloviera.gdalPython. This is an important distinction, let me try to clear that up:

  • pauloviera#gdalPython is the flake URL for building the gdalPython package in github:pauloviera/floxpkgs using the current version of the (stable) nixpkgs collection. When you reference this you are asking flox to build the package for you new every time. This can (and does) fail over time as nixpkgs moves out from under you, and can result in delays as you wait for builds to take place.
  • pauloviera.gdalPython (or to be more accurate, stable.pauloviera.gdalPython) is the floxpkgs triple providing all the information necessary to build or download the latest known-good build of that package. When you reference this you are asking flox to go back through the history of all successful builds and then repeat exactly what worked last time, using whatever version of the (stable) nixpkgs collection was active at the time. This will not fail, and can be very fast when you combine this with saving build artifacts in a binary cache.

Which one should you use? At this point while this is all very manual it depends entirely on your use case, but in future we will provide automation to continuously build and update your package collections over time, making the second of the options above generally faster, more convenient and more reliable than the first.

Hope this helps!

Thanks for asking about this example - it provided the opportunity to cover a ton of flox features and workflow, and hopefully will be of use to you and others!

Thanks Michael for the long explanation, it is also helpful for me.

I’m starting to understand how you “floxify” a repository:

  • you define a flake
  • that flake has multiple outputs in the pkgs directory, defined by their nix file
  • you can then reference any output with # (it seems like the proper full reference is with the name packages rather than pkgs. Could we be more consistent here?)

Thank you much for the helpful responses. I haven’t yet had opportunity to try it but I’m looking forward to it. I think this post could be used as a tutorial.