Ask HN: Hackable Personal Database?

11 points by the-kenny 1 year ago | 9 comments

I have been feeling an itch to build a personal "everything" database for some time now. Use cases:

- Knowledge Management / Journal / Zettelkasten

- Link Archive / pinboard.in replacement

- Cycling Route Archive / Planner

- ...

The crucial thing is: I don't want it to be text/file based like Obsidian and similar. I want a machine-readable database.

I'm thinking about an entity-key-value representation, with 'notes/content' being one attribute, 'cycling.route/gpx' another one, containing the GPX of the route, etc.

This would allow tooling to be a first-class citizen: Entities with 'web.bookmark/url' properties could archive the content into 'web.bookmark/snapshot' in the background, GPX routes could be matched with a 'ride' entity, etc.

A generic editor would allow editing of any kind of attribute, while use-case specific editors would allow edits to (for example) the GPX route.

Anything out there?

CrypticShift 1 year ago |

Grist [1] desktop [2] is a (kind of) local "wrapper app" over an sqlite database. It is "As extendible as it is flexible" [3]

The problem is I feel the tabular DB format is "too raw" for some usecases. I prefer Outliner/DB combos (like the upcoming logseq DB version) or maybe one of the local Notion alternatives.

[1] https://github.com/gristlabs/grist-core [2] https://github.com/gristlabs/grist-desktop [3] https://www.getgrist.com/product/#:~:text=As%20extendible%20...

evanjrowley 1 year ago | |

This looks amazing. I wish there was a native encryption option so that if the DB were ever leaked, all the info in it would have some reasonable level of protection.

evanjrowley 1 year ago |

I'm looking for the same. Open source, self hosted, and end-to-end encrypted.

I haven't found anything that really does all of this, but it seems like Cryptpad[0] and Peergos[1] are viable platforms upon which something like this could be built.

Except for the lack of encryption, Atomic Data[2] also seems like a really cool platform to build this on.

[0] https://cryptpad.org/

[1] https://peergos.org/

[2] https://github.com/atomicdata-dev/atomic-server

PaulHoule 1 year ago |

It's a topic I have on my mind.

Right now I have a system I call FRAxInUS which uses a document database back end and contains a commenting system, tag system, personal webcrawler, and has some visualization tools. It started out reusing a lot of code from my YOShInOn RSS reader and one of these days I want to make YOShInOn a FRAxInUS application.

In one instance of this I have a collection of 250,000 or so images or so which I am organizing. In the other one I loaded about 400 notes from somebody's Evernote and I am tagging them. I'm planning to introduce "projects" to the system so I could go to a URL like

/projects/b12f2df5-0eda-436a-9776-04221488f744

and this goes to a "tenant" in the system which might have its own db or might be in a shared db with records marked with the project id and also activates a set of Flask blueprints and sets template settings to personalize it. It starts out with ingesting images, HTML, and plain text but the long term plan is to implement structured records, for instance, to have a semiautomated process to suck out references to people in the notes and make a database of people.

Look up my email in profile and I'd be glad to talk about it.

bpev 1 year ago |

I've actually been building something like this kinda on the side for the past year or so, and use it daily for notes + map bookmarks, and occasionally for workout logs + cookbook recipes. Basically a local sqlite db of notes that can define note content types and plug-ins for using different visual treatments.

Although it's not really ready to show people yet. But would be interested to hear some more about people's use-cases for this kind of thingy. Would also be willing to jump on a call or smthn sometime and share my experience if u email me.

Rochus 1 year ago |

Have a look at https://github.com/rochus-keller/crossline/. I use it for the mentioned purpose (besides Cycling/GPX) since more than a decade. Originally I used Netmanage Ecco, which was discontinued and didn't support all features I needed (i.e. cross-references, more than one outline, etc.). It has an integrated database using the SQLite backend on top of which I implemented a kind of object database (extended entity-value). It's written in C++ and "hackable".

jjice 1 year ago |

This may not be the solution you're looking for, but I've found that I've really liked plain text files with some machine readable parts.

One thing I've done is a new line staring with a colon represents metadata, like

    :tags customer,product2

And I've built tools around tag searching. Same thing goes with dates and other misc metadata I may want to use.

It may not be what you're looking for, but I've found it to be a good middle ground for myself.

runjake 1 year ago |

I'd start with SQLite3 and go from there. Everything and anything has hooks for it.

tacostakohashi 1 year ago |

Sounds a bit like a bunch of files in some directories.