Ask HN: Do you keep a developer diary?

83 points by simopaa 9 years ago | 59 comments

I recently was struggling with an design/architecture issue all day, and when I was finished I thought about writing down some thoughts about the what/how/why aspects of my solution for future reference. I then stumbled on the term 'developer diary' for this kind of thing (for ex. https://www.smartics.eu/confluence/display/PDAC1/Developer+Diaries), that apparently some software developers hold in high regard.

I will be trying to keep my own (though a bit more light/free form than that in the link) in order to bring a bit more structure to my day-to-day.

Do you have any experience with developer diaries? Have they helped you become a better developer, and if so, what kind of effects did you notice?

samuell 9 years ago |

I researched basically all the solutions (wikis, tiddlywiki, evernote etc etc), but they all had some limitation either in offline access, shareability, portability, openness or something else.

What finally made it into a working solution is writing plain markdown files, one per day, automatically synced to bitbucket via tiny bash scripts that initiate a new file for the day (aliased to 'j' for minimal hurdle to start writing), opens them in an editor, and committs to the repo after closing the editor.

It is available here: https://github.com/samuell/mdnote

The nice thing with markdown, if you make a level-1 header for each new file, is that you can concatenate the file into a long file, and run pandoc -i allnotes.md -o allnotes.epub, and get a formatted and searchable notebook for offline use, when you want to read it, with the Level-1 headings denoting chapters - so, one chapter per day!

(I have a script for this too of course: https://github.com/samuell/mdnote/blob/master/as_epub.sh).

aldanor 9 years ago | |

The latest TiddlyWiki is actually pretty good for this. I remember using it about a decade ago -- and I've recently started using it for various dev-related stuff, e.g. documenting how to configure this or that; it's surprisingly nice.

kabdib 9 years ago |

Keeping a diary is pivotal to how I work. I used to write things down in notebooks, but you can't grep dead trees, especially handwritten ones. So I keep a simple flat text file with a new timestamp every day, and nearly everything I do makes it into this file, at some level. I can answer all kinds of questions, from when someone joined or left a team, to how I installed that driver last year, to which servers I installed three years ago and the problems I encountered configuring them. Pretty much anything unusual, or middlin' difficult, or just thoughts on engineering problems. It's candid, I swear and call things as I see them. These notes are an extension of my brain.

I start the day with:

    % note (which just runs Emacs on my notesfile)

and

    M-x new-day

which just appends the current date to the file and positions the cursor, all ready for typing. Dirt simple.

I used to have macros and whatnot for carrying forward lists of TODO items and important tasks, but that was overkill and keeping things simple seems to be the key to success. Likewise, apps like OneNote and so forth just got in the way and wanted to own a piece of me.

ams6110 9 years ago | |

Have you tried org-mode? I would assume you have, as an emacs user, but it can be as simple or as complex as you want to make it. It gives you things like timestamped entries, agenda lists, etc.

weaksauce 9 years ago | |

Just one file?

I have a man page repository that is just a simple folder that gets tab completion (basically ls fed into the complete function). aliased to `mm` as in myman(pages). that gets opened up into emacs though so it's basically a repository of files that you can grep through.

Care to share your emacs/shell setup for what you are doing?

bmalehorn 9 years ago | | |

I have a similar setup to GP: Emacs + one file + an "new-day" command.

Everything goes in one big "scratch.txt" file, with headers like:

    ###### Saturday, May 20 2017

...at the start of each day. These are generated by a function I run every day:

    (defun new-day ()
      (interactive)
      (insert (format-time-string "###### %A, %B %d %Y\n")))

"scratch.txt" is always an open buffer in Emacs. You can search through the file with isearch, helm, or whatever you use. I use the "desktop" library to save and restore all open files when I close Emacs.

That's really it, just one big unstructured text file.

I don't put a lot of prose in this diary. Only todo items, code snippets, and errors / log output.

Our setups have slightly different goals: yours is to create daily nuggets of documentation, while mine is to archive whatever thoughts and code were running through my head each day.

kabdib 9 years ago | | |

Just one file, backed up regularly. I think the simple is key; one of the sibling replies here has a bit of lisp similar to what I use, and that's it.

The problem with applying a bunch of structure is that you start to screw with the structure, maybe try to turn things into JSON or whatever, pivot by date or technology or location, and a few hours later you've wasted a bunch of time mucking with stuff that you'll probably waste more time mucking with later. Just slamming text into a file and having a minimal syntax for dating things is, I've found, sufficient. I want notes and a little support for continuity ("what the heck was I juggling when I left on Friday?"), not a blog or a platform for deathless prose.

It'd be nice if there was a rich text system similar to Emacs that didn't force you do write markup or handcuff you to a proprietary data format. Truthfully I haven't looked that hard (multi-platform is probably difficult to achieve). I have notes files strung back over twenty years, and simple text is a hard habit to break.

kaushalmodi 9 years ago | |

Same thought as /u/ams6110.. if using emacs, give M-x org-capture a try. It does the time-stamping automatically, and you get stuff like org-agenda, TODO, expirting the notes to other formats, and other org goodies.

simonw 9 years ago |

I use Evernote for this - every time I figure out how to do something I drop it in a note. I create 4-5 notes a day. It's incredibly useful - I find myself referring back to some notes months after the fact.

My team also do standups on Slack, and I've got into the habit of writing detailed Slack stand up bullet points with links to commits, documents I've created and so on - almost a semi-public developer diary. I don't check back to these very often myself, but they do spark some useful conversations.

I've also gotten into the habit of writing extremely details commit messages - often multiple paragraphs for a few lines of code change - that record the background and reasoning that lead me to the change. I do most of my code reading through the GitHub web interface, frequently referring to blame and history, so the more context I can find for historical commits the better.

codecamper 9 years ago | |

I hope Evernote is better now. I lost data when it was synchronizing once. It was 5 years ago though.

jammycakes 9 years ago |

I started keeping a comprehensive developer diary (I actually refer to it as lab notes) back in December, and it's made a considerable difference to how I think about what I'm coding.

The way I write it is similar to test-driven development — I write down each step in what I'm doing before I do it. When I run a command, I copy the command itself into my notes, then once it's run I copy the important parts of the output (e.g. any error messages).

It really comes into its own when I want to pick up on a task that I had put to one side a few days previously, or when I run into a problem that I think I'd encountered before. It also makes it much easier to avoid the trap of trying the same thing several times before you realise that you're going round and round in circles.

It's also useful for writing documentation. You can just copy and paste what you've written in your lab notes into your commit summaries, Jira tickets, e-mails, spec, help files, whatever, then tidy up as appropriate.

joepvd 9 years ago | |

Is there a reason why you made the structure of the notes depend on time? Aren't notes that are about the same topic getting very far apart from each other?

I think searching works well when you are as month or maybe three into the project. How does this work like after five years?

bmalehorn 9 years ago | | |

Also not GP, but I also have notes per-day.

If you organize by topic, you have to decide which topic to put information in. This means you have to pause and think for 10 seconds about your organization system before writing anything. 10 seconds is enough to lose your train of thought.

If you organize by time, you don't have to decide where to put it. It always goes in "today". You have to pause for 0 seconds.

pjmorris 9 years ago | | |

Not GP, but I have kept a 'logbook' file per year for ten years, with similar contents. I'm sure there's a better way than relying on OS X Spotlight to drum up things from five years ago, but it works well enough for me.

kaushalmodi 9 years ago | | |

I do time-stamping based note taking too using org-capture in emacs, and I also use tags to put my notes in some broad category. Tag auto-completion using counsel package does not need me to remember the exact tag string if it's used previously.

llccbb 9 years ago | |

Have you every considered using Jupyter Notebooks[0] or is the execution of code not so important?

[0]http://jupyter.org/

domk 9 years ago |

I use org-mode in Emacs to keep daily notes, with the majority in one big file with an entry per day. The content is quite diverse - a mix of TODOs for the day (especially when I need to get through multiple smaller things that day), things to do or think about later, small notes to self about the code I'm working on, and sometimes quite extensive notes when thinking through the design/architecture of something new.

The last technique especially helps me a lot. When thinking through a hard problem, type out your thoughts in a semi-structured way, never deleting anything. Makes it easier to deal with problems that can be too large to hold in your head at once, to think about the aspects of something step by step, or to retrace your thought process and see why you made certain decisions earlier and maybe explore a different route when you get stuck. When finished, it can be used as a foundation for a design doc I share with the rest of the team.

smoyer 9 years ago |

I use the Bullet Journal [0] method (with ink on paper) for all my personal and professional journaling. One notebook per year is about perfect. The only thing missing is aggregation (across years) and search - but I'm working on those "features".

[0] http://bulletjournal.com/

Bilters 9 years ago | |

I just browsed over their site, and I found it a very interesting concept. Thanks for the tip!

5_minutes 9 years ago |

I've been thinking about a little app to keep "logs" about the sites I am working on. Something that lives in the menu bar, and pops up with a short key. A litle pull down to select the project, so I can quickly dump the changelog, and automatic time stamp to the entry. All this in markdown.

I can't believe nobody made this already!

zubat 9 years ago |

I keep a minimal form of journal for all life events. I briefly describe a thing I did or a thing that happened, and how much energy it took me to go through it on a 0 to 4 scale, where 0 is breathing and 4 is major life crisis. If I think there was a lesson I write the lesson too.

This creates a feedback loop for stress reduction where I aim to mitigate both the likelihood of having a crisis and train a calm response by learning how to deal with smaller stuff so that the day is full of 1's. I chuck it all in a text file once a day, and return to it because my todo is also in there so I always have a backlog. I don't distinguish between work and hone tasks although I might if I were going in to an office.

Long form diaries I don't really engage with because they'd sap relatively more time - I would get caught in trying to make it a storytelling experience.

uvatbc 9 years ago |

I keep daily status logs to remind myself of the work that I have done because that's the only way I can put a stop to the nagging anxiety that I haven't done enough.

A nice side effect is that weekly status meetings are easy.

A simple markdown formatted file is more than enough.

captn3m0 9 years ago | |

I keep something similar to a .plan file which I serve at /plan/ on my machine so others at work can take a look.

I have headings as: "Working on", "Will be soon working on", and "Worked on" to mark my ongoing, todo, and done tasks.

simonw 9 years ago |

Hyperlink from this post: https://www.smartics.eu/confluence/display/PDAC1/Developer+D...

sawyerjhood 9 years ago |

I've always struggled with journaling in my personal life as well professionally. Recently I created a small messenger bot for myself that I can then use to send messages about what I did / learned, categorize them by hashtags, and then generate a list filtered by categories and sorted by timestamp. I don't think this solution is for everybody, but it works especially well for me because I use FB Messenger as my main chat app and I tend to want to be able to write anytime I have a thought, but generally read my notes infrequently.

SFJulie 9 years ago |

I keep a diary: with following info day : arriving : departure : time of lunch // travel time.

Why ?

I noticed after 43h in a moving window of 5 days my focus, irritability is up to the roof and I make way too much mistakes that can lead to a dramatic mistake.

My work contract is 37.5 so everytime I reach too much overhead I talk to my manager ask him if he will need me one day to make extra efforts in case of emergency?

He says yes, I say I will abide but if we are working under normal load, than he needs me to rest and give me my time back.

He then picks on me, saying it is my responsibility to not stay to long at work, I make him notice it is because of the company management I have to stay late and the deadline.

He grumbles says it is not the right time.

I go to see the CEO say the same thing

They give me my family time back. I stay quite rested (add 2h30m / day (when the rail company has exceptionally no problem) of commuting with the worsts train lanes of Europa (1M commuters/days)) so it actually feels like commuting is working. Teleworking is denied for lack of control.

Oh! and lunch time is not counted as worked hours except for the other managers. (so I stay away from my family 10hours per day, I see my wife 5 hours a day).

That is the only diary I am keeping.

If I get overworked/burntout, it costs nothing to the company, but I will certainly get fired sooner or later and have a hard time finding a job with a sort of depression running.

allover 9 years ago | |

Sounds like you've mentally set up an adversarial relationship between yourself and 'them' (managers). This is only going to continue to drain you.

Ideally, look for a job with a smaller commute or better attitude to remote work, and get out now. If you can't do that and need to stay in this job for a while, think about changing your mindset to empathise with your managers, even if you dislike them. Be helpful, think about what they are 'actually trying to achieve', which might not necessarily be what they asked for. Remember this is only a stepping stone til your next, better job.

If you're never going to like working 8-10 hours a day you need to figure out the necessary stepping stones to move into working for yourself.

But whatever you do, don't spend years in a me-vs-them mental battle with management at your current role. I made this mistake and deeply regret the lost years (though I did teach myself JavaScript in that time so now I'm massively employable, fuck that employer, har).

ufmace 9 years ago |

Sure do, I have at every job I've worked for several years.

I mostly used plain text files for it, and I never found a web solution that did what I wanted. So I wrote my own a few months ago during a time between jobs: https://www.masongup.com/DailyNotes

Kinda similar to what some other people on this thread seem to do. I always used plain text files for simplicity. IMO, thinking about formatting and making things pretty just gets in the way. The point is to get all of the details of everything you did that day down for future reference. That's what my site does too, just plain text notes, no formatting codes supported, stuffed into a DB and searchable.

Works pretty good for what I need. I've been meaning to add some more features to make it easier for other people to use, but haven't gotten around to it yet. Feel free to use it if you want, it does have import from and export to text files.

ytjohn 9 years ago |

Nothing shared like this. But I have kept daily logs, first as text file. Currently I do this using quiver (a multi-noteobok editor). I create a notebook called worklog and each day I put in a short list of planned actvities and then I copy/paste a lot of details below as I move through the day. Code snippets, log snippets, bits of conversation. Along with my own notes of what I tried and observed.

Currently, I'm sort of split between three tools. Omnifocus (a GTD manager) for tracking all the tasks, quiver for notes through the day, and then I use [emergent task planner], a paper based task planner for the days tasks. As much as omnifocus/gtd helps me track things, I found using paper to block out each hour of the workday ahead of time really helps me focus. If I find myself losing focus, I can refer to that and (usually) get myself back on track.

drakonka 9 years ago |

I used to keep a very active dev blog, with near-daily posts about what I did/learned each day in my game dev projects. However, that was when I was freelancing and had a lot of time to spend on personal projects daily. So there was a lot to talk about and so much new activity each day that the posts really helped organize my thoughts and hammer in what I had just learned - it was in practice a public developer diary.

After I moved to a full time role in my chosen industry, I had less time to focus on personal projects and less motivation to keep blogging regularly. I try to keep up with it but blog posts are few and far between now so I would no longer call it a diary.

MaulingMonkey 9 years ago |

Yes and no. I tend to write long, rambling details along the same lines in my commit descriptions and JIRA tasks (after the shorter summary that people will actually bother to read). I've even got tools to pull all my commits from multiple projects for easier review of the previous week/month.

The only standalone artifact I keep, though, is an excel spreadsheet tracking rough office hours, project, caffeine consumption, and choice of lunch. This helps me reassure myself that I'm not slacking off (especially when I'm considering a short day), a top level rough timeline, keep my caffeine intake in check, etc.

herve76 9 years ago |

I do keep a dev diary. It helps me a lot in organizing and coding. I encourage it. For me writing down my current affairs and coding endeavours, keep me in sync with my plans. My journal is a simple md file that I edit with Atom.

wishinghand 9 years ago |

I don't keep a diary or journal but I do keep a public repo of little things I discover that I won't need everyday, but will definitely need again someday.

https://github.com/wishinghand/Today-I-Learned

They're not deep or very in depth, but I leave them public in case someone needs what I discovered.

I got the idea from someone who posted their version on Hacker News years three or four years ago. Theirs was more journal like in its disclipline- the author added one a day.

dpc_pw 9 years ago |

Relevant: https://github.com/vimwiki/vimwiki (I highly recommend; does have journal support as well)

Arun2009 9 years ago |

I keep a private blog for my side-project, with key links and resources in pages specifically set for them. For example, I might have a page on market research with links to posts on specific topics. But probably I should be more systematic about it, since a lot of my documentation related to the project is also in miscellaneous text files (committed in the /doc folder of the project), and also a few Google docs (for sharing with others).

I find it really handy to figure out how your thought process about the project has grown over time.

navalsaini 9 years ago |

I do so on medium. I am sharing the link for the sake of it - https://medium.com/online-learning-ideas . Though I very recently started keeping one and hardly has anything.

Earlier I have used google docs for the purpose. Though those documents were often more structured than the link I shared above.

pasbesoin 9 years ago |

It can also be useful to keep a diary of your work experience. Not the proprietary information, but the assignments, personal interactions, politics.

Something that is yours and walks out the door with you, every day.

When the system or someone starts to screw you, such a record can be useful and actually viewed with some validity. Better yet, can be meeting minutes and internal communications that document the situation.

There's a reason Comey is known for writing memos about problematic situations.

mythrwy 9 years ago |

I do not keep a developer diary.

What I do keep is a large file of projects or snippets of projects, sometimes with extensive notes.

I refer back to these frequently to see how I did something or look at code samples.

But, as time goes on this has become a bit of maze to wade through. I spend time trying to remember what project it was etc. etc. Grep helps a bit :). It would be better to organize this somehow. Not sure I'm interested in a full blown diary though. DRY.

gkya 9 years ago |

I don't have any such experience but, shouldn't the VCS log serve that purpose? Maybe in conjunction with a NOTES file?

joepvd 9 years ago | |

There are different targets for notes in VCS and personal notes. In VerySpecificBug759 you do not want to read about how to get the code to build at all. The context of discovery has a lot of (necessary!) detours, cul-de-sacs, and mis-interpretations.

The context of discovery is often not relevant for the context of fix.

vlfig 9 years ago |

If the main purpose is to document decisions on a project, not your journey as an engineer, consider using Architecture Decision Records — ADRs. Simple, to the point, live-with-the-code. https://github.com/npryce/adr-tools

vfulco 9 years ago |

Simplenote is turning out to be an excellent resource. Light enough to be a pleasure to use and with great search functionality, automatic archiving, etc. A lot of thought put into it. https://simplenote.com/

It's available for every platform.

pagutierrezn 9 years ago |

I use Trello Yes I know it's not for diaries. I have one column/month. And specific columns for special projects/issues. When a note doesn't fit in a project it goes to the current month column.

You can create checklists. You can share anything from your mobile with trello so I keep track of everything I read.

lelandbatey 9 years ago |

I do, though I guess I arrived at the idea more informally. I use it chiefly as a checklist, where at the top of each "page" I list the things I want to grapple with that day. Then the rest of the document is where I commit my thoughts and questions about the subject.

car 9 years ago |

Using a developer diary is essential to my work. It's very similar to keeping a laboratory notebook.

Initially I kept notes in Evernote, but eventually switched to Quiver (http://happenapps.com/).

wickedOne 9 years ago |

i've taken notes for a while, but in the end considered it to be a waste of time (other than that it helped me remember thing more easy at first).

like any other diary, you won't re-read it and if you run into an issue of which you know you've faced it before, chances are that your written down thoughts are that unstructured that solving the issue for a second time probably takes less time than trying to find the solution in your notes.

so if writing down the issues you're facing and their solutions makes it easier for you to remember, go for it, other than that such a diary is of no use to my opinion

afarrell 9 years ago |

I have a paper scratchpad which I just use for swap space, a bullet journal i use to keep track of what I'm doing, and a dropbox papr doc that I use for reflection and recording the problems I'm working througg.

sehugg 9 years ago |

I take notes in text files as I'm learning stuff. I've published a couple of books from these notes (and also using the example code I write.)

RobBlackwell 9 years ago |

I write notes in markdown, one file per day, and use Pandoc to create HTML , PDF and mobi versions monthly for reading back later on the train.

mooreds 9 years ago |

I do this with my blog. Not with every decision, but often enough that I have referred back to it occasionally.

Share the wealth! :)

NoCanDo 9 years ago |

No. Have used in the past, didn't really help me and didn't really see the point in continuing it.

mohitsingh 9 years ago |

Just a markdown file with emojis and special characters is enough for me.

sreenadh 9 years ago |

We all need to document our knowledge, I keep forgetting stuff.

I use tiddlywiki.

VladimirGolovin 9 years ago |

Yes, I keep diaries for all my projects. I use Workflowy.

robre 9 years ago |

I use my developer diary (or journal) on a daily basis. It helps me to track what I have done, what I deem important at this time, and what I want to do next. It is also a tool to get my thoughts in shape through writing.

What is important to me:

1. Page per Day and Event I have a page in a wiki per day and one for each event that happend on that day. A event may be an idea, an issue to solve later, a link to an information, etc. By having each event as a single page with an URL, this piece of information is easy to share via messenger or email or can be mentioned from other places. Within the wiki I create a new page for the day in the morning and add child pages for each event I assume may be of interest in the future.

2. Document vs. Record Since our wiki stores information more reliably than my brain, I need to write it down. When I discovered the difference between a document (needs to be updated) and a record (needs not to be updated), the organization of information in our wiki got much easier. Records come in form of events and are part of the journal. Information that is central and important on the long run (most is not), is part of the wiki in form of a topic space.

3. Multiple Views with Tags I also tag most of the events. Since each tag also has its own page where all pages are listed where this tag occurs, I get multiple views on the journal. One form of tag is the "subject" that maps the event to a project/product I'm working on. Others (and there are only a few) simply categorize an event as an "issue" or an "idea". Some also have a tag related to a subject I'm currently working on. When I come back to the - for instance - "issue page", I can see, what I wanted to have fixed not today, but by "the end of the week". If I think to remember to have had a great idea last week - but fail to remember what it was - I head to the "idea page" (to see that it was usually not that spectacular as I thought in first place). The tagging may cost some seconds, but it makes it easier for me to track fields of interest (by text search) and find related information (by browsing).

4. Structure for Speed As others have mentioned, it is also important for me to get the information into the journal quickly. Therefore too much given structure may be an enemy. None the less I use a template/form to put information (such as the description or links to related resources) in separate, always the same (name and location on a page) sections. For me this is actually much easier than to start with a blank page.

5. Copy-Paste for Speed I also need to paste screenshots, files, and URLs quickly to a page. Therefore I choose a wiki that supports this over markup files in a repository (which are great for other use cases with other requirements as others already have mentioned on this page).

Not everything runs as smoothly as I want it to be. So there is always room for improvements ...

BTW: I'm one of the authors of the mentioned article about developer diaries (https://www.smartics.eu/confluence/display/PDAC1/Developer+D...).