Unattended installation + deb package for aarch64 (Debian on RPi 4B+)

alexdelprete · February 27, 2023, 3:02pm

Hi,

I completed the migration from nix to flox on 8 servers, created a script that uninstalls nix multi-user and installs flox through the deb package.

I found the following issues:

during the installation, I install the first package to activate the environment, and the script halts requesting permission to collect metrics. Is there some options to bypass this?
I have a Raspberry 4B+ that serves my homelab as a Stratum 1 Time server, it runs on Debian Bullseye 64 bit: unfortunately after I uninstalled nix, I found out that the available deb linked in the docs is only for x86_64, so I stopped the migration halfway. Is there a deb for aarch64?
It would be nice to have an installation script like nix provides for automating the installation.
I use chezmoi to manage the configuration of the servers: I was wondering if the config files in $HOME/.config/flox can be templetized in order to automatically configure flox (the GH token, protocol, etc.)
After the migration from nix, I found out that the packages are much older vs the ones installed via nix-env. Is there a way to use the same channel I used with nix? From a quick look, the prerelease channel is still behind the channel used in nix.

Thanks for your work, I think this project will be a great success.

ysndr · February 28, 2023, 11:30am

Hej Alex

Thanks for giving flox a go and providing feedback!

About your questions:

When you first run flox it will show you a dialog for accepting or rejecting metrics. In an unattended script you can make sure that the first call to flox is
```
echo "y" |  flox config
```
This will persist your decision on that instance.

Alternatively, you can set an environment variable FLOX_DISABLE_METRICS=true to skip the metrics question for as long as that variable is set. Eg.
```
$ FLOX_DISABLE_METRICS=true flox install <pkg>
# or

$ export FLOX_DISABLE_METRICS=true
$ flox install <pkg>
```
We are working on making this configurable through a flox config file, which should be available with the next release.
We’ll add aarch64 installers to the download page soon, will update once we do.
What would you like a script like this to do?, genuinely curious. – detect the running system and download the right installer?
We are currently in the process of reworking our configuration management (see point 1.).
With that you’ll be able to template your machine based configuration.
you can install more recent packages using flox install unstable.<pkg>

Again, thank you for your feedback!

alexdelprete · February 28, 2023, 4:17pm

I love the flox concept, so thank you for the work. I’m sure it will become an excellent tool.

Great. If I set it to “false” it will skip the question anyway? I’d like to enable metrics to help you devs.

great. this would allow to automate the installation. a config file or an installation script with options would be perfect.

great news. btw: on the GH repo I found no release files, the only link I found was the one in the docs. Will the GH repo become more “active”? Can we open issues there or the forum will be the main tool for support?

Well, when I spin a new server (I use Proxmox), through chezmoi I configure the environment and I setup all the needed tools, right now I have flox installing all packages, but I need to install flox first, and the installation has to be automated and possibly configurable so that it can be adapted to the server (different OSs, CPU architectures, etc.). With nix it was a no-brainer, just one unattended script downloaded on the fly through curl and it installed nix multi-user on every possible platform. Hope we have something like this for flox. An apt custom repository would also be good as an alternative approach so we can install through apt-get once the custom repo is configured.

great news, can’t wait to try it.

I already use the unstable channel, but it’s still much older vs nixpkgs, here’s an example for chezmoi (2.28 on unstable vs 2.31 on nixpkgs):

❯ flox search chezmoi
chezmoi - Manage your dotfiles across multiple machines, securely
  stable.nixpkgs-flox.chezmoi@2.25.0
  stable.nixpkgs-flox.chezmoi@2.27.1
  stable.nixpkgs-flox.chezmoi@2.27.2
  staging.nixpkgs-flox.chezmoi@2.27.1
  staging.nixpkgs-flox.chezmoi@2.27.2
  staging.nixpkgs-flox.chezmoi@2.28.0
  unstable.nixpkgs-flox.chezmoi@2.27.1
  unstable.nixpkgs-flox.chezmoi@2.27.2
  unstable.nixpkgs-flox.chezmoi@2.28.0

❯ nix-env -qaP | grep chezmoi
nixpkgs.chezmoi   chezmoi-2.31.0

ysndr · February 28, 2023, 8:28pm

Yes, we currently only have an opt-out. Exciting to see interest in an opt-in, we’ll consider that for a future release!

We haven’t setup releases to github yet. Though we are considering that as there seems to be a interest for that.

We are committed to open sourcing flox so there will be more activity keeping that updated.
In terms of community engagement, we encourage you to use Github for PRs or bug reports.
For discussions and general questions we’d like to concentrate those here on discourse

That’s interesting, let me get back to you with this!

alexdelprete · February 28, 2023, 9:29pm

does the variable work also when installing the deb package? because that was the main reason why I asked, not when installing a package via flox install but when installing flox itself, here’s the part of my installation script that installs flox:

# Install flox
if ! command -v flox > /dev/null; then
    echo "Install flox via deb archive"
    tmp_deb="$(mktemp)"
    src_url="https://floxdev.com/downloads/debian-archive/flox.deb"
    wget -O $tmp_deb $src_url
    dpkg -i $tmp_deb
    rm -f $tmp_deb
    if command -v flox > /dev/null; then
        echo "flox installed correctly"
        echo "- activating default env."
        flox pull -e flox/default --force # to remove your local modifications
        flox install flox.flox # to install the latest stable flox to your default env
    else
        echo "flox was not installed correctly!"
        quit 1
    fi
else
    echo "flox already installed"
fi

ysndr · March 1, 2023, 10:15am

We’re happy that you’re keen to enable metrics.

What you could do too is to run

echo "y" |  flox config

as the first flox command on a new installation, as in the following snippet.
This is basically just answers the first occurence of the consent dialog with “yes”, and saves it to your config.
With this there is no need for an environment variable thereafter.

...
    if command -v flox > /dev/null; then
        echo "flox installed correctly"
        echo "- activating default env." 

        echo "y" |  flox config # this will answer the metrics consent with "yes" for you, no flags needed after that

        flox pull -e flox/default --force # to remove your local modifications
        flox install flox.flox # to install the latest stable flox to your default env
    else
...

alexdelprete · March 1, 2023, 3:35pm

problem is that on a clean server I install the deb file through dpkg -i, and while installing the deb, the metrics question is asked. So I wanted to know if setting that env variable works also for the deb package, when the flox command is not yet available.

mkenigs · March 1, 2023, 9:08pm

I’m not able to reproduce the installer prompting for metrics. Did I miss how you’re running into that? Here’s what I got:

$ sudo dpkg -i flox-0.1.0-r372-0.0.10-r190.deb 
[sudo] password for floxfan: 
Selecting previously unselected package flox.
(Reading database ... 142065 files and directories currently installed.)
Preparing to unpack flox-0.1.0-r372-0.0.10-r190.deb ...
Unpacking flox (0.1.0-r372-0.0.10-r190) ...
Setting up flox (0.1.0-r372-0.0.10-r190) ...
Processing triggers for man-db (2.9.4-2) ...

Also, re a custom apt repository, the deb actually sets that up (you can see it adds /etc/apt/sources.list.d/flox.list and /usr/share/keyrings/flox-archive-keyring.gpg). Would it be easier for you to set that up and then apt install flox? And is the issue that you don’t want to have to detect architecture in your script?

alexdelprete · March 2, 2023, 1:58am

I’m using the following script to install flox, maybe it’s so fast downloading (cache?) that when I ran it I wrongly attributed the metrics prompt to the deb installation step while it was either the flox pull or the flox install commands. Now I added the variable for both, but I will try to understand which of the two triggers it once I setup a new server.

install flox script

# Install flox
if ! command -v flox > /dev/null; then
    echo "Install flox via deb archive"
    tmp_deb="$(mktemp)"
    src_url="https://floxdev.com/downloads/debian-archive/flox.deb"
    wget -O $tmp_deb $src_url
    dpkg -i $tmp_deb
    rm -f $tmp_deb
    if command -v flox > /dev/null; then
        echo "- flox installed correctly"
        FLOX_DISABLE_METRICS=true flox pull -e flox/default --force
        FLOX_DISABLE_METRICS=true flox install flox.flox
        echo "- activated default environment"
    else
        echo "- flox was not installed correctly!"
        quit 1
    fi
else
    echo "flox already installed"
fi

For some other tools, in my chezmoi dotfiles, I download the source list and keyring, put them in the right places, then apt update and apt install the specific package. Through chezmoi I have the possibility to determine the proper architecture and use different URLs to the apt sources etc., no problem with that. Is there a public url to the apt sources list and keyring? If so, we can explore that type of installation too.

The only thing I’m missing now is how to install it for aarch64 and also why the packages are older than nix-pkgs I was using before. From what I understood, flox works on top of nix, and I wrongly assumed the pkg repositories were more or less on the same page, but I notice you have a more staged approach. It would be nice if one of the channels was in sync with nix-pkgs.

Thanks a lot for the support, it’s highly appreciated.

robinbrantley · March 2, 2023, 2:36pm

Hi @alexdelprete,

Thank you for raising this issue (point 5 above). You are correct in that flox sits on top of nix and therefore the pkg repositories should be more or less on the same page but with a staged approach. Looking into your query further, we found an issue with our automation that we have now fixed so unstable should now offer the latest available as expected:

$ flox search chezmoi
chezmoi - Manage your dotfiles across multiple machines, securely
  stable.nixpkgs-flox.chezmoi@2.27.1
  stable.nixpkgs-flox.chezmoi@2.27.2
  stable.nixpkgs-flox.chezmoi@2.29.3
  staging.nixpkgs-flox.chezmoi@2.29.3
  staging.nixpkgs-flox.chezmoi@2.29.4
  staging.nixpkgs-flox.chezmoi@2.30.0
  unstable.nixpkgs-flox.chezmoi@2.29.4
  unstable.nixpkgs-flox.chezmoi@2.30.0
  unstable.nixpkgs-flox.chezmoi@2.31.0

Regarding your Raspberry Pi query (point 2 above), it looks like we will have an installation ready for testing soon. Would you be interested in helping us test this out?

alexdelprete · March 2, 2023, 2:56pm

Robin, thanks a lot for the feedback and your kindness. This is a good news. Glad to help you uncovering these issues.

So now with a flox upgrade I should see new versions of packages…will give you a feedback very soon.

Absolutetly, will be more than glad to help you test it. I think support for arm64 (aarch64) would benefit many users. Plase let me know what I should do to test it, thanks.

alexdelprete · March 2, 2023, 3:01pm

@robinbrantley the upgrade worked perfectly, now I can see latest versions:

Before your fix:

Packages
    0   stable.flox.flox                     0.1.0-r372-0.0.10-r189
    1   unstable.nixpkgs-flox.bitwarden-cli  2022.11.0
    2   unstable.nixpkgs-flox.btop           1.2.13
    3   unstable.nixpkgs-flox.chezmoi        2.28.0
    4   unstable.nixpkgs-flox.du-dust        0.8.3
    5   unstable.nixpkgs-flox.fd             8.6.0
    6   unstable.nixpkgs-flox.fzf            0.35.1
    7   unstable.nixpkgs-flox.git            2.38.1
    8   unstable.nixpkgs-flox.gnupg          2.3.7
    9   unstable.nixpkgs-flox.keychain       2.8.5
    10  unstable.nixpkgs-flox.navi           2.20.1
    11  unstable.nixpkgs-flox.ncdu           2.2.1
    12  unstable.nixpkgs-flox.neovim         0.8.2
    13  unstable.nixpkgs-flox.pinentry       1.2.1
    14  unstable.nixpkgs-flox.tldr           1.5.0

Just upgraded, after the fix:

Packages
    0   stable.flox.flox                     0.1.0-r372-0.0.10-r189
    1   unstable.nixpkgs-flox.bitwarden-cli  2023.1.0
    2   unstable.nixpkgs-flox.btop           1.2.13
    3   unstable.nixpkgs-flox.chezmoi        2.31.0
    4   unstable.nixpkgs-flox.du-dust        0.8.4
    5   unstable.nixpkgs-flox.fd             8.7.0
    6   unstable.nixpkgs-flox.fzf            0.38.0
    7   unstable.nixpkgs-flox.git            2.39.2
    8   unstable.nixpkgs-flox.gnupg          2.4.0
    9   unstable.nixpkgs-flox.keychain       2.8.5
    10  unstable.nixpkgs-flox.navi           2.20.1
    11  unstable.nixpkgs-flox.ncdu           2.2.2
    12  unstable.nixpkgs-flox.neovim         0.8.3
    13  unstable.nixpkgs-flox.pinentry       1.2.1
    14  unstable.nixpkgs-flox.tldr           1.5.0

robinbrantley · March 2, 2023, 6:32pm

Absolutetly, will be more than glad to help you test it. I think support for arm64 (aarch64) would benefit many users. Plase let me know what I should do to test it, thanks.

The aarch64-linux installer images have just rolled off the assembly line and are now available for download at:

Please note we have yet to properly test these and while we are pretty confident installer will work we’re less confident that upgrades will work because this is our first time deploying images for multiple system types to the same debian archive / yum repository.

Thanks again for offering to help with testing and please do let us know how you get on.

alexdelprete · March 2, 2023, 8:30pm

Thanks Robin. Can you provide the links for all the architectures? I’m modifying install script to create the url based on the archs.

I would expect also https://floxdev.com/downloads/debian-archive/flox.amd64-linux.deb to be present but it doesn’t exist, I only find https://floxdev.com/downloads/debian-archive/flox.deb.

I tried to browse https://floxdev.com/downloads/debian-archive/ but it’s not possible unfortunately.

robinbrantley · March 2, 2023, 9:20pm

No worries, here are all 6 links, including that for MacOS:

alexdelprete · March 2, 2023, 10:18pm

The script worked, and I have flox installed correctly on the RPi 4B.

local/aarch64-linux.default
    Alias     default
    System    aarch64-linux
    Path      /root/.local/share/flox/environments/local/aarch64-linux.default
    Curr Gen  14

Packages
    0   stable.flox.flox                     0.1.0-r372-0.0.10-r189
    1   unstable.nixpkgs-flox.bitwarden-cli  2023.1.0
    2   unstable.nixpkgs-flox.btop           1.2.13
    3   unstable.nixpkgs-flox.chezmoi        2.31.0
    4   unstable.nixpkgs-flox.du-dust        0.8.4
    5   unstable.nixpkgs-flox.fd             8.7.0
    6   unstable.nixpkgs-flox.fzf            0.38.0
    7   unstable.nixpkgs-flox.git            2.39.2
    8   unstable.nixpkgs-flox.gnupg          2.4.0
    9   unstable.nixpkgs-flox.keychain       2.8.5
    10  unstable.nixpkgs-flox.navi           2.20.1
    11  unstable.nixpkgs-flox.ncdu           2.2.2
    12  unstable.nixpkgs-flox.neovim         0.8.3
    13  unstable.nixpkgs-flox.pinentry       1.2.1
    14  unstable.nixpkgs-flox.tldr           1.5.0

Probably I will change the script, and instead of using deb archives, I will copy flox.list and flox-archive-keyring.gpg in the proper folders, then use apt install flox (like @mkenigs suggested). It would be nice to have those two files available in https://floxdev.com/downloads/debian-archive/ so users can simply curl/wget them.

I’m not sure I understand what you mean regarding upgrades: after installation via deb archive, when I use flox upgrade it won’t also upgrade flox? If not, what is the proper procedure to upgrade it?

Thanks.

robinbrantley · March 3, 2023, 2:17pm

Probably I will change the script, and instead of using deb archives, I will copy flox.list and flox-archive-keyring.gpg in the proper folders, then use apt install flox (like @mkenigs suggested). It would be nice to have those two files available in https://floxdev.com/downloads/debian-archive/ so users can simply curl/wget them.

Thanks for the suggestion! I’ve added it to our list of potential enhancements for prioritization.

I’m not sure I understand what you mean regarding upgrades: after installation via deb archive, when I use flox upgrade it won’t also upgrade flox? If not, what is the proper procedure to upgrade it?

I was referring to the fact we have not yet tested our upgrade instructions work on a Raspberry Pi as we wanted to get the installation out to you ASAP. The procedures to upgrade are in the installation docs, with the specific commands for a Debian installation noted here for easy reference:

It would be great if you could let us know how you get on when you go to upgrade flox following the next release!

alexdelprete · March 3, 2023, 4:00pm

It would also be more consistent with the upgrade instructions (via apt).

One question: after the first installation (via apt or via deb archive), in order to upgrade flox, I thought it would be enough to flox upgrade so the flox.flox package would be upgraded. Isn’t that enough? What is the difference?

Sorry for the basic questions.

mkenigs · March 3, 2023, 8:59pm

The gpg key is available at https://floxdev.com/downloads/debian-archive/flox-archive-keyring.gpg. Unfortunately the flox.list file isn’t available at the moment, but since it’s one line, that would be pretty easy to workaround by just echoing that line to /etc/apt/sources.list.d/flox.list. Although we will likely change the URL in that file soon

This difference between installing with apt or a deb and installing with flox is something we’ve discussed a fair bit but aren’t completely sure what’s best, so I’m curious to hear your thoughts.

Hopefully I won’t bore you too much with the details…but flox consists of two parts:

components needed by the host operating system, e.g. systemd units. We use the host package manager whether dnf or apt to manage these components
a flox package. This can be managed with flox - flox install/upgrade flox.flox 99% of our development work will result in upgrades to flox.flox, so most of the time it will be completely up to date.

So that’s a bit confusing, and not something we want every user to have to understand. We could:

Encourage only the usage of the apt package
Special case the flox.flox package to get special treatment, so that it can install system files, or maybe invoke apt behind the scenes. In that case we’d probably have to introduce a flox-managed system level environment owned by root. But once again that’s introducing additional complexity
Something else?

What are your thoughts?

alexdelprete · March 3, 2023, 11:41pm

That’s why I think it’s better having a public URL: so you are free to change the contents without breaking automation scripts. Meanwhile I’ll write the one-line via echo, but I’d recommend having it published as the keyring file.

Personally, I prefer the keyring+source list download and then apt install, it’s easier to manage.

I highly appreciate this conversation, and the fact that you are asking for feedbacks. Thank you.

So, since we have 2 components (systemd+special flox package) to install and manage flox, I would go towards a single installation/management point, otherwise it could get confusing for users.

Of the two possible approaches, I think the apt one is better, because the flox special package is actually a sort of system-package, so apt would be a better solution to install/manage it, while for all the rest of packages you would use flox.

The second approach (flox package that calls apt behind the scenes) is also feasible, but I see it a little bit hacky. The important thing is using either apt or the flox package, but not both.

So, to summarize, in order to easily install flox I see these macro-steps (that can easily be scripted):

download flox apt source list in /etc/apt/sources.list.d/
download keyring in /usr/share/keyrings/
apt update && apt install flox install (install flox system package)
flox pull -e flox/default --force (I would also create the default environment in this step, before installing the first package)

Done. You’re ready to work with flox. For upgrades to flox system package, you would rely on the standard OS upgrade procedures via apt (or other tools for non debian OS), and that makes perfectly sense, because it’s more tied to the specific OS.

That is my $0.02, thanks for the brainstorming Matthew.

Topic		Replies	Views
Debian install fails if nix is already installed? Bug Reports	5	310	April 5, 2023
Alternative to .config/nixpkgs/config.nix:packageOverrides Help	2	404	October 30, 2022
Can't publish a package on aarch64 Developing with flox	6	273	January 13, 2023
Nix/Flox way to replace ansible Nix closed	1	253	July 12, 2023
Manage a project with an existing flake with flox Developing with flox	2	356	November 11, 2022

Unattended installation + deb package for aarch64 (Debian on RPi 4B+)

Related Topics