Feedback on the elixir Phoenix Framework guide

I’m talking about this page: https://beta.floxdev.com/docs/language-examples/elixir_phoenix_framework/

  1. typo in docs
git clone https://github.com/flox-examples/hello-phoenix.git
cd hello-phoenix
checkout base-phx

last line should be git checkout base-phx

  1. flox build fails
warning: creating lock file '/home/thomas/Workspace/github/hello-phoenix/flake.lock'
error: getting status of '/nix/store/5b62grwvcasvr19kb62ifdfig39gyg7a-source/deps.nix': No such file or directory
  1. Flox develop

There is a missing quote in the documentation in the pg_ctl start command. I expect there needs to be an extra " at the end.

[flox] λ   pg_ctl start -l $LOG_PATH -o "-p 5432 -c listen_addresses='*' -c unix_socket_directories=$PWD/postgres -c unix_socket_permissions=0700"
waiting for server to start.... stopped waiting
pg_ctl: could not start server
Examine the log output.
[flox] λ 

This could be a bit more helpful. I found the issue by looking around in the files and found out it failed because there was already a postgres running.

2022-12-05 10:41:06.235 CET [38777] LOG:  could not bind IPv4 address "0.0.0.0": Address already in use
2022-12-05 10:41:06.235 CET [38777] HINT:  Is another postmaster already running on port 5432? If not, wait a few seconds and retry.

I was not expecting there to be a problem with the running postgres. I guess I still need to learn more about nix to really understand what kind of things it isolates.

  1. Not sure where to go from here.

In this develop shell, I tried to run mix phx.server but it fails because the deps are not installed.
But running mix deps.get fails

If I inspect pkgs/hello-phoenix/default.nix it’s hinting towards mix2nix but I’m not sure if this should be run manually or if this something the file does.

Also mix2nix doesn’t seem to be available in any environment, so I guess I need to add it somewhere?

1 Like

I tried to get the main branch running from hello-phoenix, but that doesn’t work either:

flox develop -A phx-shell
warning: not writing modified lock file of flake 'git+file:///home/thomas/Workspace/github/hello-phoenix':
• Updated input 'floxpkgs/nixpkgs/nixpkgs':
    'github:flox/nixpkgs/f994293d1eb8812f032e8919e10a594567cf6ef7' (2022-10-25)
  → 'github:flox/nixpkgs/2788904d26dda6cfa1921c5abb7a2466ffe3cb8c' (2022-11-22)
error: unable to download 'https://api.github.com/repos/flox/capacitor/tarball/830a026092dbed052442af65f0a5f03a458076e5': HTTP error 404

       response body:

       {
         "message": "Not Found",
         "documentation_url": "https://docs.github.com/rest/reference/repos#download-a-repository-archive"
       }
(use '--show-trace' to show detailed location information)

Thank you @tcoopman for your reports, I’ll get back to you swiftly about the pheonix issues you experience!

1 Like

I’ve made a bit of progress:

I’ve been able to create a build:

  • I’ve added pkgs.mix2nix to the buildInputs of phx-shell/default.nix
  • In a develop shell I’ve run: mix2nix > deps.nix
  • I’ve uncommented the postBuild steps.

After this, flox build -A hello-phoenix works.

But the release doesn’t seem to actually work:

➜ result/bin/hello_phoenix start
cat: /nix/store/61y7dq9grvrvpgy30fqz1mxv61yhp2wx-hello-phoenix-0.0.0/releases/COOKIE: No such file or directory

For develop I tried to do something similar, but it doesn’t work:

mix do deps.loadpaths --no-deps-check, phx.server
Compiling 18 files (.ex)

== Compilation error in file lib/hello_phoenix_web/telemetry.ex ==
** (CompileError) lib/hello_phoenix_web/telemetry.ex:3: module Telemetry.Metrics is not loaded and could not be found. This may be happening because the module you are trying to load directly or indirectly depends on the current module

Your approach was correct right, and we should include a default deps.nix as well as the mix2nix package!

Regarding the phoenix build / runtime errors I’ll cc @Samuel_Rose into the discussion, who is our expert for the elixir stack.

Hi @tcoopman @ysndr Thanks for trying flox out!

I am going to give a few responses here to a couple of the issues you had right away. Then, I’ll follow up in a bit on the rest a little later today.
Regarding #2
We definitely need to update our docs to reflect that running mix2nix is needed, and we’lll do that today!

Issue #3 with flox develop we’ll fix the docs there again, thank you! We have a new branch that we’ll merge ASAP that runs the postgres services with a Procfile and hivemind so that when you exit the flox develop shell, the services will just automatically turn off.

I’ll follow up today to resolve these issues, and thanks again.

1 Like

@tcoopman you wrote

But the release doesn’t seem to actually work:

➜ result/bin/hello_phoenix start
cat: /nix/store/61y7dq9grvrvpgy30fqz1mxv61yhp2wx-hello-phoenix-0.0.0/releases/COOKIE: No such file or directory

…yes we need to document how to use the release. It’s use is mostly the same as standard mix release without flox involved. mix release — Mix v1.14.2

At minimum you need to somehow set the DATABASE_URL SECRET_KEY and RELEASE_COOKIE env vars. Or DATABASE_URL and SECRET_KEY can be moved to runtime.exs as discussed at Configuration and releases - The Elixir programming language and the phoenix framework example phoenix/runtime.exs at master · phoenixframework/phoenix · GitHub

However, we will document a “using the release” section this week, to give some guidance on using a release that has been built from the flox build and runtime perspective. I’ll be sure to update here when we add that section.

@tcoopman you wrote

mix do deps.loadpaths --no-deps-check, phx.server
Compiling 18 files (.ex)

== Compilation error in file lib/hello_phoenix_web/telemetry.ex ==
** (CompileError) lib/hello_phoenix_web/telemetry.ex:3: module Telemetry.Metrics is not loaded and could not be found. This may be happening because the module you are trying to load directly or indirectly depends on the current module

I don’t know which steps you followed to generate this outcome, but my guess is that after having run flox develop you may need to then run mix deps.get for the above command to work. I’ll check our docs to make sure we are making this clear

Basically, we want to orient people in how they can use flox in a development context (where you would run all the normal mix deps.get, iex -S mix phx.server etc commands just as you otherwise would), and flox will bring in an integrated development environment for you, and then when you are ready to release for production, you can flox build -A hello-phoenix and that package has all the settings to output the release, but built deterministically by flox, and then you can distribute it to other machines via flox install at minimum (and we will document other ways to do it too)

Thanks, I’ll try these things tomorrow afternoon!

Also it took me a while to find what hivemind is: GitHub - DarthSim/hivemind: Process manager for Procfile-based applications

I’ve been able to get most of it working, and I think I start to understand more about how flox works (at least the develop part).

Some more questions and remarks I have:

  1. If I wanted to develop would I open multiple terminals with (1) for the things that need to run (things that go in hivemind, like the database), (2) the development server (mix phx.server) and (3) the tests?
  2. I’m not sure I completely understand the isolation model, if you start something in a flox shell that keeps running (for example postgres) and then exit that shell, I’d assume that postgres would stop as well, but that’s not the case.
  3. Something I haven’t thought about yet, but that I would need to know if I continue using flox. How would this work together with fly.io (flyctl) that creates docker images to deploy.

An other update. I wanted to do some regular elixir development and couldn’t start my server because there was still a beam and elixir piece of code running from a previous flox develop shell.

Is this really the expected behavior?

Hey,

  1. Sure, flox doesn’t have any bearing on whether you run multiple terminals for development, just activate the shells you want in each one.
  2. You are correct, and while we haven’t implemented this yet, we have it planned for Q1’23.
  3. flyctl available in nixpkgs, so you can just flox install flyctl in any environment.

Is this really the expected behavior?

This is not expected, and I couldn’t reproduce this behavior. Could you provide some details on the app running or logs?

So for the building of an elixir app with flyctl, I would just install the necessary things inside the nixpkgs and change the flox build steps to deploy to flyctl?

Regarding the things that keep running, I’ve been able to reproduce:

  1. run hivemind
  2. kill the terminal without killing hivemind first

Now all the process that hivemind started are still running :frowning:

Expected behaviour: killing the terminal should kill all applications that hivemind runs as well.

Thank you for this additional info. We agree this isn’t ideal and I can confirm I have now logged an issue for it to be addressed. In the meantime, I’ll see if we can find a workaround for you.

1 Like

quick update - unfortunately we haven’t found a workaround for you for the short term but we are hoping to get this addressed in the next few weeks.

1 Like

I just also want to follow up: we have it on our early 2023 list to document deployment to fly.io I’ll be sure to ping this thread when we do so.

Following up on this:

What I have been doing in practice is to run all companion services in hivemind, for development. These can be services like postgres, redis, etc

However, for elixir itself, when in flox develop shell mode, I just run elixir or phoenix like usual (eg iex -S mix phx.server) you are still using all of the the underlying dependencies like postgres etc that are brought in by flox this way.

I only run the mix release version of the app in production environments, after merging and testing etc and that usually has a bit different set up and management than the dev env. Although when the release is built with flox build, it will be deterministic and use the same dependencies etc.

It sounds like possibly you are running the release of the app you are developing, and yes this can leave orphaned beam processes running on the machine. In general, in flox develop with hivemind and iex, you should not see services continuing to run if you exit the terminal they are running in. So, it would be good to know more about your configurations, and how you are running apps etc to help you better if you keep seeing those issues.

As far as I can see, if hivemind is killed, it doesn’t kill the processes it manages: Sub-processes aren't in the same process group · Issue #8 · DarthSim/hivemind · GitHub

@tcoopman just some follow up: I will say that in practice on an ubuntu machine, when I use hivemind to run these processes with a procfile that is formatted like hello-phoenix/Procfile at procfile-hivemind · flox-examples/hello-phoenix · GitHub when I ctrl+C on the running processes, they stop and are not orphaned.

We offer hivemind and Procfile as a suggestion, but when it comes to this area of using flox, it ultimately up to you as to how you decide to run these services in development or production. Any conventional approach per operating system should work as expected (systemd, initd, hivemind, foreman, custom bash scripts, etc).

It could be that your issue is indeed related to Sub-processes aren't in the same process group · Issue #8 · DarthSim/hivemind · GitHub however that appears to be beyond that scope of issue we can support and fix directly. We definitely thank you for sharing that information as it may help others. It is usually the case with all of these services, there is not much special about how you run elixir, postgres, or other services, other than changing the paths where they are run from.

All that said, if you run into specific problems running specific code, likely we can help you if you post those here along with the code you are trying to run, the specifics about the operating system. Sorry to report that I have not been able to recreate the problems you see with hivemind.