Literate Devops with Emacs Org-Mode (2014)(howardism.org) |
Literate Devops with Emacs Org-Mode (2014)(howardism.org) |
https://github.com/Wilfred/ag.el
Read find-grep exclusion patterns from a .findignore file: https://emacs.stackexchange.com/a/19038/6682
I've tried a few mobile org-mode clients and been universally disappointed. I'm increasingly convinced that the right path forward is to give Emacs an Android port (it's not that hard) and let the mobile org just be the mainline org, perhaps with a layer of android.view.View widgets (orchestrated from lisp) as an interface.
[1] https://news.utexas.edu/2017/06/26/the-mere-presence-of-your...
I find that for a true productivity tool, I'm fine relying on my desktop & laptop. I'm actually dialling back what I use my phone for, simply because I can't trust either of the major mobile OSes to respect me & my privacy. It would be awesome to have all of org available on a phone though; you're not wrong at all there.
I would kill for an emacs which makes sense on mobile, but I've not given enough thought to it to imagine what would make sense. Emacs without the key chords would be still powerful but also odd.
Magit is my go to tool, I only use command line to quickly switch branches sometimes or for non-standard things.
Things I love about Magit:
- CHUNKING .. staging chunks or unstaging chunks is so easy (my favorite feature)
- General overview of information
- diffs, rebases, etc
- It's really fast when you learn the key chords
Other thing I use for Git:
- Command line (if I'm in command line and need to change a branch, might as well do it there)
- Tig (Log is fine in Magit too, just old habbits)
For example to stage things you can just tap ‘s’ while running your cursor down a list of changed lines/files. To commit, it’s just ‘c c’, and you’re in an editor, in a markdown-like mode, with hard wrapping lines at a good default.
Amending previous commits is also very easy, and there are some quality of life abstractions involving branches which I’ve found very useful.
But, ultimately, it’s all the same stuff. If you have a good system with Gitx at the moment (and not much of an interest in Emacs), I wouldn’t worry about it.
I was going to do my blog entirely with org-mode+org-publish, but it spits out a mess of HTML and also a small bit of js for some reason, so I'm just writing html directly myself now.
I don't really find writing html myself all that annoying or anything, but it'd be nice if I could get org to generate really clean/simple HTML for me that I could then write a small script to parse and wrap in divs I've defined for styling.
If tweaking those is not enough for you, you can write your own HTML export. You wouldn't have to parse Org syntax: you'd receive the Org markup already parsed into a nested list structure. If you don't use many Org features, writing an Org backend that supports only those probably isn't that much work. And you'd have ox-html.el to base your own backend on.
On my site (https://two-wrongs.com/) there is very little left of the original HTML. I'll see tomorrow if I can dig up the Lisp code I wrote for it.
The HTML export back-end includes a compact default style in each exported HTML file. To override the default style with another style, use these keywords in the Org file. They will replace the global defaults the HTML exporter uses.
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="style1.css" /> #+HTML_HEAD_EXTRA: <link rel="alternate stylesheet" type="text/css" href="style2.css" />
To just turn off the default style, customize org-html-head-include-default-style variable, or use this option line in the Org file.
#+OPTIONS: html-style:nil
After a bit of googling, I also found an email[1] on the mailing list asking for more or less exactly that, but things don't seem to have gotten far.
I've been thinking about adapting the existing HTML export for such an XML format, but I really haven't had the time for it. :-/
[1]: https://lists.gnu.org/archive/html/emacs-orgmode/2009-08/msg...
Few examples:
1. ox-hugo documentation site from one file[1], source[2]
2. My blog[3]
3. ox-hugo test site (300+ test pages and counting)[4]
[1]: https://ox-hugo.scripter.co/ [2]: https://github.com/kaushalmodi/ox-hugo/raw/master/doc/ox-hug... [3]: https://scripter.co/ [4]: https://ox-hugo.scripter.co/test/
The JS is for interacting with tables, if I'm not wrong.
[1]: https://www.reddit.com/r/emacs/comments/809xwx/how_to_get_pl...
Do you want to do something much more complicated/simple than this?
M-x magit
M-x profiler-report
You can accomplish a surprising amount of optimisation after seeing what it is that takes time.
My main problem with it is that, unlike with executing commands in the shell, with org you don't get any progress, and emacs seems to hang while running the command.
For example, running apt update tends to freeze emacs without any progress reporting for a few minutes.
I'm hoping that I'm doing something wrong and that someone can chime in with a suggestion, because I really like org-mode, and I really like the idea of doing devops using it.
Check out: https://github.com/astahlman/ob-async if you want to change that.
I still struggle to see how people actually use this with devops on a day to day basis.
I think that some type of integration with tmux would work well - have a running tmux session, and then just queue commands from emacs to that session, instead of running directly through emacs.
It's mostly about making a literate program, executable on the spot, sort of like a Jupyter notebook, only for shell commands (or nearly anything, frankly).
I agree it's a good idea. I do similar things when writing PowerShell scripts (can run them line by line using PowerShell ISE).
EDIT: If you're going to downvote me, it'd be better to explain why.
Is the fact that you got the idea exactly backwards: is not about putting comments on your scripts. Is about writing an article, intended to be read by humans, that is also executable.
Imagine it this way: Suppose you are reading a blog post on how to install, configure vagrant and install drupal on it. The article is pretty clear an well written and you enjoy it. And you also know that is correct because every single one of the instructions shown was executed while you were reading the article and every output was given back to you.
That is literate programming, is closer to a jupyter notebook than to 'comments on scripts'.
You can keep the notebook in the repo if you do two (or three) things:
1. The scripts read passwords from a key store instead of the notebook source. I wrote a Python package for doing this easily in Python cells; for bash, I just use subcommands.
2. A git pre-commit hook clears the notebook output cells.
3. (Optional) A git pre-commit or pre-push hook searches for text that looks like private credentials.
Surely that's a matter of perspective? Imagine you have a text document containing one command and one description of that command. Is this a literate programming document or not?
Furthermore, does the mode change depending on which tool is used to open the source file? For example, if I open a Bash script in a text editor then it's a document, and if I open it in Bash then it's a script, but if I could open it in a tool that allows me to both read and execute the script at my leisure does it then become a literate programming document?
Aside from this, to use your Jupyter notebook example, is it necessary for the results of execution to be captured in-line in order to follow the literate programming methodology?
[1] https://www.gnu.org/software/emacs/manual/html_node/emacs/Ou...
org-mode is a major-mode and covers a little more than that^^
I'm sure most of the individual features are available in vim and other editors, but it's tying everything together in a neat package that's the main draw of org-mode.
Notmuch is like the best of the newsgroup and email models. It has a great threaded view, tagging and filtering are easy, but it still makes everything very easy to find when you need it.
Also in combination with w3m it does a really good job at rendering HTML emails in plain text. Even those with tables and other layout tend to be rendered very accurately within the constraints of a fixed font plain text view.
There are a few different android apps for the phone (I think they are Dropbox based) though I don't use any.
I would not be where I am without Orgzly. Yes, I wish it was more complete. But given the very limited manpower behind it, it is excellent.
It's also better than any other similar app I've tried, even non-Org ones. So it's the best note-taking/todo/planner app for Android, and then the fact that it uses Org files behind the scenes is just a great bonus.
If you use org-mode without the need to extract such specific information (and are only interested in generating a document), pandoc might of course be a good way to go.
Edit: ...or maybe there even is a hack to identify and extract things like timestamps anyway. I didn't look into it that much.
A general-purpose markup language like XML would be an ideal intermediate export format; XSLT can basically emit anything.
Android will kill applications pretty indiscriminately, so your Termux session persistence isn't good.
Terminal performance is poor, especially on scrollback, for some reason.
I've bumped into numerous SSH issues.
My general preference would be for a hybrid tablet/laptop system, running Linux, rather than Android (or iOS). We may eventually get there.
The other thing tablet formats are good for is reading electronic document formats, with an 8" or better tablet size. Though a large eInk device might be better there (again, ideally with shell capabilities).
I've had pretty good luck with SSH on Termux - it's great to be able to transfer things from my computer to my phone via rsync (I have some apps which do SAMBA, but they tend to be flaky; likewise physically plugging the phone in and transferring with adb isn't very convenient).
I'd tried to make that happen over the course of 2.5 years, a full year of which was with Termux. And as mentioned: the best damned thing to happen to me on Android.
Termux is really nice, and for the 700 - 800 or so packages it has, there's a tremendous amount of utility. But you're still stuck with limitations.
1. The Termux-dies-randomly problem is, AFAICT, inherent to Android and its memory management model. Unless Termux can somehow indicate that it is not a terminable process, that's not going to go away.
2. The terminal-performance issue appears to be specific to Termux, as other full-screen console tools (ConnectBot SSH client) do not have this problem. Since it manifests on both local and remote connections w/ Termux (e.g., scrolling backwards within a manpage on a remote system), it seems to be the local terminal code on Termux itself. Of all my complaints, this seems the one most readily fixed. But it has not been. (I have reported it. Several other issues I've noted have been rapidly addressed.)
3. The SSH lag and hang issues I saw seemed to cover all Android SSH clients, and were either Android, its networking stack, or quite possibly my Very Shitty WiFi Network (since somewhat improved). But in all Not Particularly Heartening. Wireless sessions from a laptop didn't have the same lag issues.
4. Package selection. Unless you switch to a full Android/Debian system or something like it, you're limited in what's available. Commendable effort, and a Hard Problem. Just ... well, it leaves me short.
5. The Android storage model. Termux is Just Another Android app, and has Very Limited Access to storage. Which fucking sucks as there's a 128 GB MicroSD card on the device, but almost completely unusable. Rooting your Android might get around this, but if you're stuck with an unrooted (or unrootable) device, Thou Remainst Fucked.
6. Shell only. There's no X.org or other graphical interaction. (Other than Android native, which leaves you with that platform's bastardised and pitiful apps selection, on which I've written elsewhere.)
Mind: I really wanted to like Termux. I kind of wanted to like Android. I have a Samsung Tab A and Linksys Bluetooth folio case/keyboard. In terms of a portable form factor, it's really fairly close to my ideal. But in terms of capabilities, performance, behaviour, applications, surveillance, and vendor shitfuckingheadedness, from Samsung, Google, and Linksys, and absolute and complete clusterfuck. I want to run as far as I possibly can from any of them. I absolutely recommend AGAINST any of these vendors.
The simple truth is that I want and need something that they are unable, or far more likely, unwilling to provide. Neither my financial nor time budget for exploring alternatives is large, and the profound level of frustration I've had in this (and earlier) Android excursions has left some keep and I'm sure long-lasting scars.
I've tried and used many other systems, few of which strike me as desireable. I don't care for Microsoft's products (though the MS Surface tablet, again, strikes me as well-considered hardware). Apple's tablets and laptops are shiny, but ultimately annoying. I'm typing this on a retina iMac, which has a luscious display but an annoying OS. I've been partial to Thinkpads for nearly two decades, but find Lenovo's recent shenanigans massively off-putting. I'm also no longer generally satisfied with the traditional clamshell laptop design, yet don't see what might yet supplant it. Portrait reading for documents is a very compelling feature, but what I see are either tablets (with no or poorly-considered keyboard and mouse options), or hybrid devices with hinging mechanisms that leave me in a cold sweat. (Maybe they Really Do Just Work and Will Prove Rugged in Actual Use, but I Have My Doubts.)
Presumably the only real solution is using desktop Linux on a device, rather than Android.
Is the GDP Pocket [ https://medium.com/@tomac/qpd-pocket-7-the-return-of-the-hac... ] included under your category of sweat-inducing devices? It looks handy, and even seems to have some sort of pointing/TrackPoint-like nub. I haven't used or even seen one in real life, but I've heard good things.
I'm not convinced there's anything that will replace an X-series ThinkPad. Non-phone devices that fit in our pocket and run Linux are rather appealling, and I'd love to have one, but I still don't imagine it could really oust my ThinkPad for most purposes. I need at least the X-series sized screen for most of what of I do.
The keyboard, a complete piece of crap, and with abysmal customer and warranty non-support, was from Logitech, who deserve absolutely all opprobrium that can be directed their way.
(Linksys make my old router -- a (very) tried-and-trued WRT54G model. Since retired, after well over a decade of excellent service.)