Multiple Environments with Shared Environment Variables across individual project components using Taskfiles

Trying to get to load your environment variables in your project, updating your environment variables across different parts of your application and switching between environments during your project can be cumbersome.

You might create some scripts in shell/bash or golang to solve this problem, which you then need to execute right before executing the actual commands you want to run for your app (e.g. starting or deploying), in order for these env variables to be loaded with the app. This seems very hacky and might require some expertise and might not work on every OS equally.

This is why I am writing this post, to show you how to circumvent this strategy with something much more simple and reproducible across all your projects that will not only solve this challenge but bring a lot more clarity to all your command-line commands for your project.

Let's start with what Taskfiles are. Task is a task runner / build tool that was created to replace Makefiles, which very often are very unreadable and not as clear. It makes them simple, still fast but without complexity. So you can basically just execute all your NodeJS / Deno / Go projects commands with Taskfile, rename them, run multiple with one, run them in parallel and even include environment variables.

Here is an example of a Taskfile, which includes multiple parts of your project including the database, deployment scripts, and for example your actual node server.

version: "3"

dotenv: ['.env', '{{.ENV}}/.env']

includes:
  db:
    taskfile: ./db/Taskfile.yml
    dir: ./db
  scripts:
    taskfile: ./scripts/Taskfile.yml
    dir: ./scripts
  server:
    taskfile: ./server/Taskfile.yml
    dir: ./server
    aliases:
      - "s"
tasks:

Each part of your project has its own Taskfile. If you supply a dir then all commands will be executed from that directory. If you supply aliases you can shortcut the command execution, here by just writing s instead of server. In short to execute a command you can just write task db:create or task server:start if these commands are respectively defined in the Taskfiles. The interesting thing, though, is that we can now have environment variables from the root dir for all these parts of the project. We can create production.env, staging.env and development.env. Respectively based on what ENV is in your shell session it will load .env and then if ENV=production you will also load production.env and overwrite the same key/value pairs that were in the .env file.

This modular structure gives us a way to structure env variables for multiple environments and load them easily. In the respective folders of our project we can just have simple Taskfiles like this:

version: "3"
vars:
  OUT: server

tasks:
  tailwind:build:
    cmds:
      - npx tailwindcss -i ./index.css -o ./public/index.css
  tailwind:watch:
    cmds:
      - npx tailwindcss -i ./index.css -o ./public/index.css --watch
  run:
    cmds:
      - go run ./src/main.go
  build:
    deps: [tailwind:build]
    cmds:
      - go build -o {{.OUT}} ./src
  test:
    cmds:
      - fmtFiles=$(gofmt -l .) && vetFiles=$(go vet ./) && [ -z "$fmtFiles" -a -z "$vetFiles" ]
  format:
    cmds:
      - gofmt -w .
  docker:build:
    cmds:
      - docker build -t $DOCKER_REGISTRY_URL/$DOCKER_REGISTRY_REPO/{{.OUT}} .
  docker:push:
    cmds:
      - docker push $DOCKER_REGISTRY_URL/$DOCKER_REGISTRY_REPO/{{.OUT}}

This way we have environment variables for all environments loaded and can still benefit from a very modular structure without any complex setup. You can install Taskfiles as described here.

Sources:


← Back to Homepage