Show HN: Dotenv, if it is a Unix utility(github.com) I like the idea of using dotenv files, but I dislike having to use different language-specific libraries to read them. To solve this, I created a small utility that lets you prefix any command with "dotenv" to load the ".env" file. This is how I imagine dotenv would work if it had started as a UNIX utility rather than a Node.js library. |
As an example of the difference, dotenv is useful for running programs inside Docker containers — which do not inherit your interactive shell's environment variables — whereas direnv isn't particularly useful there. Ditto for programs run via init systems like systemd or even classic SysV init. On the other hand, direnv is convenient for end-user env var config, since it's aware of your shell's working directory and updates the env vars based on it without needing to run extra commands.
`env -S "$(cat .env)" <cmd>`
Believe it or not that’s all you need.
> S, --split-string=S process and split S into separate arguments; used to pass multiple arguments on shebang lines
edit: forgot the quotes around shell substitution
sh -c '. .env; <cmd>'
sh -c '. .env; "$@"' -- command arg1 arg2 arg3.
sh -c '. .env; exec "$@"' -- command arg1 arg2 arg3.
execl("/bin/sh", "/bin/sh", ". .env; exec \"$@\"", "--", "command",
"arg1", "arg2", "arg3", (char *) 0);
The nice thing about utilities like env and dotenv is that they can be easily exec-ed:
execl("/usr/bin/dotenv", "/usr/bin/dotenv", "command", "arg", (char *) 0);
You want $(cat .env) quoted, as in "$(cat .env)" so that the content of the file is reliably passed as one argument.
-S will split on whitespace; but it respects quoting, so spaces can be protected. Basically .env has to be prepared with the features of -S in mind. Of which that thing has quite a few: escape sequences like \n, commenting, environment variable substitution.
https://lists.gnu.org/archive/html/coreutils/2021-10/msg0000...
It came up in the mailing also this March. I saw the posting in my inbox and proposed that a null-terminated format be handled, which is exactly like /proc/<pid>/env:
https://lists.gnu.org/archive/html/coreutils/2024-03/msg0014...
If that feature were available, this becomes
env -f .env command arg ... export $(cat .env | xargs)
The language runtime dotenv projects are banned in my engineering org.
source <(cat .env | xargs)
export $(cat .env | xargs)
unset $(cat .env | cut -d= -f1)
The last one unsets the environment variables that were set by the first command, ensuring they are not persisted beyond the current shell session.
If you are worried about forgetting to execute it, there are a couple of ways to work around it, depending on your case.
env $(cat .env) [program]Would love to hear more about why dotenv is banned at your org though.
I believe in convention over configuration. Most of our apps have hard-coded config, with a concise/short and finite number of things that can be overridden (like 3-4 parameters, tops). Secrets get injected.
I do subscribe to the idea of the 12 factor app, but there is a line that needs to be drawn between env config which is more dynamic and more persistent config that should be baked in to the release.
sh -c '. .env; echo $MY_VAR'
do the same thing? (I am not in front of a shell at the moment.)
I have this on my .bashrc:
alias loadenv='export $(xargs <.env)'
--
loadenv() {
set -a
source ./.env
set +a
} sh -ac '. ./.env; ./prog'
The xargs idea made me think of using bash as the parser :
bash -c "exec -c bash -c 'source $CONFIG/main.bash; env'"This outputs the env vars to stdout; for the OP's purpose, the output could be sourced :
envFile=$(mktemp /tmp/env.XXXXXX);
bash -c "exec -c bash -c 'source $CONFIG/main.bash; env'" > $envFile;
env $(cat $envFile) sh -c 'echo $API_HOST'
sh -c "env -i sh -c '. $CONFIG/main.sh; env'" > $envFile
envup() {
local file=$([ -z "$1" ] && echo ".env" || echo ".env.$1")
if [ -f $file ]; then
set -a
source $file
set +a
else
echo "No $file file found" 1>&2
return 1
fiYou can also specify `envup development` to load .env.development files should you want. Obviously this will pollute the current shell but for me it is fine.
It's a shame that running modern software requires carefully packaging a virtual environment and then injecting a bunch of ugly global env vars.
I still think Docker shouldn't exist. Programs should simply bundle their dependencies. Running a program should be as simple as download, unzip, run. No complex hierarchical container management needed.
Alas I am not King.
The .env being evaluated as a shell script means that it's in a widely used language, with a widely known syntax. You can look at it and know what it's going to do.
The .env being a data format to some uncommon utility; that's anyone's guess.
For instance, suppose we want a newline character in an environment variable. Does the given "env file" format support that? How?
There is one de-facto standard format: the /proc/<pid>/environ kernel output format on Linux. The variables are null-terminated strings, so it is effectively a binary format. It represents any variable value without requiring quoting mechanisms.
So your suggestion is to munge these for local development. And you're okay with that barrier? That's terrible dx, and it adds surface area for bugs.
... except I'm thinking this may `set +a` if the environment already had `set -a`, which maybe could cause problems? I wonder if it would make sense to record the existing status of "-a" (allexports) an set it / unset it as necessary.
loadenv() {
case "$-" in
*a*) source ./.env ;;
*) set -a; source ./.env; set +a ;;
esac
}
awww. i don't think it's OK in any way to download libc6/msvcrt as many times as I download __any__ software. even more, is there a strong difference between dependency and runtime environment? if sensible people does not bundle the whole python distribution to a "stuff.py" then why bundle libopenssl.so to a webserver application?
IMO, a saner approach would be just not to confuse dependencies: appX depends on libY 1.9; appZ depends on libY 2.0; people are quick to declare that appX and appZ are incompatibe as they can not run on the same system due to "conflicting dependencies". but who said you have to seach libY in /usr/lib*/libY.so? if you need different versions of a lib, just install them in separate dirs and make your apps find the right one (eg. by setting RPATH or versioned .so filenames).
Programs should rely on the global runtime environment as little as possible
> if sensible people does not bundle the whole python distribution to a "stuff.py"
Unfortunately Python deployment is such a such an unmitigated disaster that it's a leading cause of Docker images.
Deploying a portable copy of Python is about 9 megabytes compressed. This is significantly preferable to multi-gigabyte Docker images.
> people are quick to declare that appX and appZ are incompatibe as they can not run on the same system due to "conflicting dependencies". but who said you have to seach libY in /usr/lib*/libY.so? if you need different versions of a lib, just install them in separate dirs and make your apps find the right one (eg. by setting RPATH or versioned .so filenames).
You make a strong and compelling argument as to why programs should bundle their dependencies and not rely on the system environment.
Users should not have to perform any witchcraft to launch a program. Download and run. No further steps should be necessary.
From your description, direnv is implicit and noisy, whereas dotenv seems to be (unless you embed it in a script) explicit and quiet.
env $(cat .env) my-cmd-wanting-dotenv
ETA: the main difference between `env` and `dotenv` seems to be that `env` gets its arguments from the command line, whereas `dotenv` gets its arguments from a file. I think that's a fair difference, but I might also think that perhaps `env` should expand its offering to include some kind of `-f filename` option so that it can focus on the notion of "a configurable sub-environment for a command" and we can avoid subtle distinctions.
(. .env; my-cmd)Another advantage of env is that you can type `man env` and learn something useful; sourcing and subshells via syntax is a little bit harder.
Finally, I think the major point of this branch of the discussion is to explicitly decorate a command with a special environment. Starting up a subshell isn't the same thing. It might have the same effect, but you can see that you're creating a subshell, running a builtin in the subshell, and then running a command in the subshell. It is something of a difference between declarative (dotenv/env) and imperative (sourcing in a subshell) approaches, and inherits all the pros and cons of the imperative approach.
If it works for you, I make no recommendation against it.
My assumption overall in this is that most people have just one .env per project (or perhaps in sub-folders per environment, e.g prod, staging, local), but these don't nest. With nested .env files, the mental overhead they bring remove (IMO) most of the benefits, if not more.