hit -> [return] ~ .
to break a jammed up ssh session. A lot of people don't seem to know that one.
* Respect the user's default foreground and background color. Don't change them without good reason.
* If you use colors, make them legible regardless of what the default background and foreground colors are, and regardless of the terminal's color map.
* Don't use color as the only indication of something. The user's terminal might not display it, and it probably won't be preserved in copy&paste into notes.
* Use emoji only judiciously, if at all. Similar with gratuitous non-ASCII characters. It doesn't display everywhere, it doesn't paste well everywhere, and emoji can be a bit much when copy&pasted into some notes.
* In a scrolling (non-full-screen) stdout-ish output, don't delete important information that you showed temporarily. For example, hiding warnings or filenames compiled, to display a green checkmark for done. For another example, clearing the screen of Web app build information (including package security warnings!), to display a message that it's running in dev mode, is also not wanted. People might want to see that information, or copy&paste it into notes.
* If you went full angry fruit salad with your command line program, because it's your baby, and you're having fun hamming it up, that's fine, but please provide an easy preference setting for people to opt out of that. Your program is probably only one of many things on user's workstation display, where other programs might be using color and visuals more meaningfully, so animated throbbing red explosions for the code reformatter is a bit much.
That reminds me of an incredibly annoying bug I encountered a few years ago involving Docker. One of the scripts being run was outputting emoji to STDOUT, and this was causing the interactive terminal to crash and thus the container to exit. (This issue [0] has error-strings and simple repro tests.)
I'm not sure if the root cause ever got fixed, but I (and many others) ended up making PRs for various open-source projects, to grudgingly implement workarounds that compromised their original artistic vision. :p
In any case, the character itself is unrelated to the bug. As someone pointed out in the bug thread, certain Cyrillic strings will also cause it.
This is kind of impossible. One can potentially provide for customized colors, though.
A downside of this is that it is quite restrictive, there are only like 8 colors.
EDIT: I missed the “regardless of the color map” bit, that is a bit unreasonable. Either you trust the terminal emulator or don’t. I think trying to have it both ways is too much.
[1]: https://gist.github.com/JBlond/2fea43a3049b38287e5e9cefc87b2...
There is some effort to standardize this: <https://bixense.com/clicolors/>
This deserves to be highlighted more. Nowadays some authors keep assuming users use a patched font and build their tools with that in mind, thus hampering ease of use significantly.
The user may be colour-blind (30% of the population). The user may be completely blind and relying on a screen reader.
And some of us choose to disable color by default. (Yes, I'm old-fashioned.)
It does include the blog post's rules on exiting on Ctrl-C, accepting `-` for stdin, disabling color in pipes, and much more.
This is really important. I'd like to expand on this.
Standard output is for the data the program was asked to produce, no more and no less. If user asked for some JSON data, standard output should contain that exact JSON object and absolutely nothing else.
Standard "error" is actually a misnomer. It should have been called the standard user stream. Anything meant for the user to read on the terminal is supposed to go there. Error messages are of course included in that set but so are status messages and verbose output.
This ensures the output of programs can be piped into other programs seamlessly. Non-output data still gets sent to the terminal or redirected somewhere else.
Would have been great if programs were able to easily create new terminal-connected file descriptors for specific purposes. They could document those numbers in their manuals just like they document exit codes. Then users would get "ports" for every output. Could cut down on parsing significantly.
For compatibility, they could all redirect to either standard output or standard error by default... I think I'm gonna experiment with this a bit.
- If this is your first time hearing about the readline/emacs keybindings like Ctrl-E and Ctrl-W, you'll be pleased to know that most macOS input sources use these keybindings. If you're on macOS, feel free to try Ctrl-E, Ctrl-W, or Ctrl-U in your browser's address bar right now
- If you're using a command line program that doesn't support _any_ line editing (e.g. no readline keybindings, and no other keybindings), you can install the `rlwrap` program and launch the REPL under rlwrap. For example Standard ML of New Jersey has a REPL but no line editing functionality, but you can recover that via `rlwrap smlnj`
- "don’t use more than 16 colours" — I would go so far as to say "don't use more than 8 colors, or at least make your colors configurable." Many popular color schemes, including Solaraized and the default Base 16 color scheme, use the "bright" colors to hold various shades of gray. What you think is "bright green" might actually be the same shade of gray that normal text is colored.
Most browsers I've used close the current tab when you press Ctrl-W. Actually, the terminal emulator I use, Alacritty, also does this, and most file explorers that have tabs also do. Iirc even windows explorer does this now, but it's been a while since I've actually used windows.
This can't be right. I use ctrl+w all the time, and occasionally use Alacritty. I'd notice if this shortcut closed my terminal window (it's extremely annoying when I use a web-based ssh, because I have this shortcut deep in my muscle memory).
- Press Ctrl-D, like normal
- Get confused when nothing happens
- Remember that it doesn't work in GHCi, so run `:q` instead
- Get an error message about "lexical error at character '\EOT'", due to Ctrl-D inserting an invisible char at the start of the input
- Try `:q` again, without any invisible prefix
- GHCi successfully quits
Vim works really weird when you start typing random capital letters after thinking you were moving somewhere.
Is there some setting that changes it?
Every other REPL I've used handles this fine, e.g. for quitting Python, Nix repl, SSH sessions, and even the shell itself. Weird.
func main() {
if _, err := os.ReadFile("~/.bashrc"); err != nil {
log.Fatal(err)
}
fmt.Println("ok")
}
$ echo ~/~/~
/home/jrockway/~/~
I mention it because while "yourprogram ~/path/to/file" always works, having a repl that asks for a filename might not work. I've seen a lot of software where this DOES work, so I think it counts as a "most TUI programs do this" thing.
There are conventions, but following all the conventions in a CLI is a lot easier than designing a good GUI. So they tend to be higher quality as a result.
I spend a lot of time thinking how to bring this property to GUIs, but my best answers are still "lots of effort" or "lower your expectations".
The Application Framework defaults which underly most native macOS application build a lot of common keyboard controls in. Use the default menu bar classes with the default basic commands, and Command O, N, Q, X, C,V and probably others come for free, you just implement the code that you need to for those functions. Use a standard text field and you automatically get Command/Option/Fn Left, Right, Up, Down for navigation. It’s more notabke when a macOS application doesn’t follow convention (e.g. InteliJ uses Shift-Opt up/down to move lines rather than expand the selection by paragraph) than when one does.
Windows does decently well on this front, but ctrl as a default modifier can hurt terminal based app usage and there are a number of UI frameworks even within the OS that appear to get different defaults.
And in the Linux world, I think the only way you could do this would be for someone to design (and a distro to standardize on and port apps to) a full on application framework. The window managers don’t want to be in the business of dictating the behavior of stuff in windows. The GUI toolkits don’t want to be in the business of defining os wide defaults and the DEs and distros don’t want to be in the business of if porting or dictating UI frameworks. And realistically there’s no one “on high” that could make the sort of dictation that for example “hence forth copy and paste will be Meta-C and Meta-V”
mysql(1) only links to editline instead of readline, where Ctrl-W by default deletes everything to the start of the line, not just the last word. It drove me mad in the period where I had to use it; you just see your entire nice query disappear. :-)
On Unixes of course, don't know what VMS/MVS/.... did.
This one is still nowhere near universal enough to count in the original lisr sadly.
1. Don’t assume a terminal type. Look at `TERM` and use termcap/terminfo or a library built atop them for anything beyond line-oriented plain text output, or least assume a plain teletype unless you specifically recognize the user’s terminal type.
2. Don’t assume the presence of a terminal at all. Check `isatty()` before doing anything fancy, and be sure to work without a terminal so your tool can be used in pipelines and called by other programs via `exec()`.
3. Follow the common conventions in your arguments and output structure. For example, if your tool takes an open-ended set of arguments, support specifying a response file with `@path/to/file` to avoid argument limits. If your tool supports record-oriented output, support a `-0` argument to `NUL`-separate the output for use with `xargs` in pipelines.
4. Use the standard `<sysexits.h>` exit codes. They exist for a reason and they make use of your tool within pipelines and programs more straightforward because they make it easier to trace why a failure occurred.
5. Include both in-binary `--help`/usage information and a man page. Often a user will just need a quick refresher on argument syntax, which is what the built-in help text is for; the man page should be a comprehensive reference with examples. It should *never* defer to a web page or GNU `info`—it’s fine if those exist too and are pointed out, but they should not be the primary user reference.
Lots of Linux-oriented tools have one or more of these failure modes, and behave poorly on real terminals that aren’t VT100 derivatives or are awkward to use in anything but an interactive setting.
Presumably, you would ideally output text in whatever encoding is specified by the LANG environment variable, but this seems like something that only comes with full i18n/l10n support, since it also specifies the actual language to use.
Are there any actual established guidelines on this?
Play your terminal like a piano . Your livelihood depends on it.
This is akshully not correct. Control-D makes the read(2) return with the data currently in the input buffer. If there's no data in the buffer, that results in a 0-length read, which is how EOF is signaled.
Try this: run cat, type foo, press control-D. "foo" will be echoed, without any newline.
In particular, those can change between OSes: On all linuxes, programs already get a pre-parsed array of command line args on start, so parsing and quoting behavior depend on the shell, not individual programs.
On Windows, programs get passed the entire command line invocation as a string and have to do the parsing themselves. While all "well-behaved" programs let the libc do this, it's perfectly possible for a program to break the rules here.
On UNIX, expanding globs (*.txt) is the shell’s job.
On Windows, expanding globs is the program’s job.
I used to have a bunch of four-line Python programs to, essentially, run `flac --best --replay-gain *.wav`.
Strictly speaking that does not explain where globbing happens, but it does help understanding where they come from.
We aren't in the '80s. Use true color if you want to, modern terminals should support it (built in Linux tty is a weird outlier that should have supported true color years ago).
But that also depends on the context. For example if something implements its own TUI with a lot of elements - it makes more sense to use more colors than the barebones set.
Most programs that do care about colors, check what terminal capabilities are before using them.
I commonly run "ssh remote-system" in a tmux window, then attach to a tmux session on the remote system.
If you nest tmux sessions like this, you have to type the prefix character (Ctrl-B by default; I use Ctrl-Space) twice for the nested session to see it.
I just checked to see if Ctrl-F and Ctrl-B work, and found that the former kills one word forward and the latter acts like Ctrl-W ought to?
If memory serves, this behavior depends on the OS. On Windows I believe the norm there is to type "<Ctrl-Z><Enter>"
For example, if an application you're using has left things in cooked mode, and you press ctrl+c, the application will never "see" that keystroke; something higher up in the input chain (I believe Linux's TTY driver) will see it, swallow it, and send SIGINT to the application.
Applications can also put the tty into "raw mode", where this won't happen; in that case the app is responsible for implementing those "expected" keystroke behaviors, if it makes sense for the application to do so.
Raw means send characters immediately, literally. Cooked imply some some of the flags are on, specifically (from memory) icanon, echo, among the `lflags`. Plus others.
Further down among `cchars` you'll see werase = ^w and kill = ^u. Here kill means kill (erase) the line, not send a signal.
https://learn.microsoft.com/en-us/powershell/module/microsof...
What if programs could define any number of output streams? By default they could all coalesce into the terminal but other programs could connect to each one separately if they needed. Like an audio mixer of sorts: by default you get the mixed audio but there are ways to access each individual voice if needed.
Once you get used to having all that, going back to other shells is pretty hard.
Of course, the real problem is there's no standard, the standards that do exist are ignored, and each new command-line tool generates a new standard.
Can you provide a concrete example of such an application?
That's what it is....a log. Of errors, warnings, informational messages, whatever to complement the primary output.
BONUS: stdlog fits nicely :)
Tilde expansion is the first operation in word expansion:
> The expansions that are performed for a given word shall be performed in the following order: 1) Tilde expansion, parameter expansion, command substitution, and arithmetic expansion shall be performed, beginning to end. 2) Field splitting shall be performed on the portions of the fields generated by step 1. 3) Pathname expansion shall be performed, unless set -f is in effect. 4) Quote removal, if performed, shall always be performed last.
See https://pubs.opengroup.org/onlinepubs/9799919799/utilities/V...
To see the whole shell specification, goto the main page at https://pubs.opengroup.org/onlinepubs/9799919799/. Select "Shell & Utilities" from the top-left frame, then "Shell Command Language" from the bottom-left frame.
In POSIX shells '~name' expands to the home directory of user 'name'
$ echo ~bin
/bin
All for the rare case that someone passes a file path starting with tilde and wants it to be expanded. IMO, when you provide a file interactively you do so through a shell and tilde expansion is handled by the shell and otherwise just provide the actual file path and do not rely on the tool doing additional expansions.
PS: I mixed up TUI and CLI program a bit in my head. For an interactively used TUI program it might be beneficial to implement tilde expansion (but then it should be as complete as in Shells). A CLI program should not do magic stuff like tilde expansion.
mv *.m *.c
Going further, as part of shell expansion, expanding the tilde could also be useful for filename expansion in the standard library.
What if there is a file named "~"? What about "~username"? How do you escape/quote it? What if $HOME is not set, or set to something different than the actual user home directory? What about Windows? Also, a lesser known fact is that typing "~." after newline in ssh will force close the connection from the client side and there is nothing the server can do about it, so don't make it part of your workflow.
Entering "~vicky/doc.txt" or "~Ricky/doc.txt" over SSH will actually only write "icky/doc.txt", and entering "~Chris/doc.txt" will have SSH print out it's internal command-line's usage instructions in the middle of the TUI. Screenshot from Vim: https://imgur.com/a/NCt0G9r
./myprogram --config=~/.config/myprogram
./myprogram --config ~/.config/myprogram
Interestingly, bash does do tilde expansion, but not tab completion, after the first equal sign, and after every colon, in variable-style arguments:
$ /bin/echo --foo=~ bar=~:~root
--foo=~ bar=/home/jtm:/root
[1] https://github.com/scop/bash-completion/blob/8a8880db78e9b04...
https://pubs.opengroup.org/onlinepubs/9799919799/functions/w...
I agree, but these days I think you can mostly get away with assuming VT100-compatible behavior.
> 4. Use the standard `<sysexits.h>` exit codes. They exist for a reason and they make use of your tool within pipelines and programs more straightforward because they make it easier to trace why a failure occurred.
I just took a look at this header file. (It defines exit codes starting at 64.) I'm not sure I've ever seen a program that uses these codes. Many programs for UNIX-like systems just use exit(1) for generic errors, or maybe something to distinguish between data errors and usage errors such as unrecognized command-line options. I mostly use Linux; maybe it's more common on BSD-based systems (it appeared "somewhere after 4.3BSD"). For example curl defines nearly 100 distinct error codes; none of them are based on <sysexit.h>.
> 5. Include both in-binary `--help`/usage information and a man page. Often a user will just need a quick refresher on argument syntax, which is what the built-in help text is for; the man page should be a comprehensive reference with examples. It should never defer to a web page or GNU `info`—it’s fine if those exist too and are pointed out, but they should not be the primary user reference.
In practice, GNU `info` tends to be the primary reference for GNU programs. The man page is often missing a lot of information.
Error logging can be set to either spit out the error and continue, or to stop processing, which is very useful.
Warning and error messages come out in their own colours automatically, which is nice.
And sure, you can just have them all come out to the same place. But having the options to configure them is very handy.
It seems like the only way to satisfy your ask is to not use color at all.
So I think it's not on every user to somehow design optimal color palette settings on their computer that work in all combinations (if they even can), but rather on the developers of software not to say "Hey, I bet every yellow would be legible on white" or "Yolo, I bet yellow is legible on every background, so I'm just going to set this foreground to yellow and not set background at all."
printf '\e[48;5;196m\e[93m test \e[!p\n'So no, I do not think that what you wrote is correct. From what I can tell, the line-by-line mode is “extproc”.
As a side note, some shells have started implementing a shorthand for `foo 2>&1 | grep` which is `foo |& grep`.
(To be fair, that does seem like a pretty rare use case.)
foo 2>&1 >&- | grep ...
Alternatively, if you only want to grep to see the stderr, but you still want to see stdout, you can swap stderr and stdout like this:
foo 3>&2 2>&1 1>&3 | grep ...
foo > >(grep stdout) 2> >(grep stderr)You mean this:
`foo 3>&1 1>&2 2>&3`
The above swaps stdout and stderr, saving + restoring the original stdout through descriptor 3. Shell redirection expressions map directly to dup2 syscalls, except the operands are reversed: 3>&1 is dup2(1, 3). Pipes map fairly simply to fork + exec. At its core the shell is a rather thin wrapper around the core Unix syscalls fork, exec, dup2, open, and close. If you look at the original shell implementation, the command parser more-or-less executes these syscalls as it goes along, left to right.
I really like stdlog. Standard log stream is a pretty awesome name. Short and terse, the word "log" doesn't even need abbreviation and it's correct since it's a superset of error and info streams and also generic enough to cover other unforeseen categories.
I'll use it from now on!
> Unless your app doesn't generate [stdout] output,then grepping the [visible] output will not work if it's stderr and it will be confusing
With the first "output" meaning "specifically stdout", and the second meaning "what you see on the terminal".
As in, you run a command, see it printed a lot of stuff, so then you do it again but piping it to `grep` or `less`, and realize none of that output was actually going to stdout.
The output of a package manager is the packages. The text it writes to the terminal is not part of the output, it's just the program's log. I haven't thought about what's appropriate in this situation. By unix standards package managers shouldn't even be printing anything to begin with.
If the user passes invalid options and inputs, then the program should produce no output and any error or help messages should be written to the error stream. It's important that these messages end up on the terminal when one gets the command invocation wrong while trying to pipe data into another program. If they're written to standard output, the program on the other end of the pipe will slurp all the error messages up and try to parse them as though they were valid inputs.
Nothing wrong with that, but it was a bit confusing the first time I saw that behavior.
min N with -icanon, set N characters minimum for a completed read
Also on macOS the documentation for cbreak is different, it lists a bunch of settings including -icanon, it's Linux that makes it literally an alias for -icanon.
This hasn’t been my experience - while I’m not familiar with a wide breadth of terminal emulators, all the ones I’ve used have a default black background with the ANSI colors being very bright, making them clearly visible. I would again say that if a terminal emulator has some of the standard ANSI colors set to not be visible on their default background, that is the terminal emulator’s problem, as it is clearly undesirable.
And of course, once a terminal program starts changing the background color then it can’t make any assumptions about which of the user’s colors will be visible - which is why, as you say, the background color should not be changed without a very good reason. If the bg is set, it should be very easy to switch it to either a “dark mode” or “light mode” to make colors visible.
But some assumptions must be made in order to make any use of color, and “the 6 standard ANSI colors (red, green, yellow, magenta, cyan, blue) are visible on the user’s background” seems like it has to be the safest assumption.
I am in support of terminal programs respecting a universal configuration to disable color: https://no-color.org/
The way I see it is the version information and help text belong on standard output if the user passes --version and --help since that's what the user explicitly asked for. When the command is invoked incorrectly, the help text should go to standard error while standard output should be empty.
I agree about the exit code. People like to parse error messages and that's always wrong. Everything should be done via exit codes instead.