Ask HN: Resources for designing custom file formats

3 points by m0ther 6 years ago | 1 comment

I am working on a custom OO micro-database, which needs to persist to files.

I am spending a lot of time solving problems utilizing files that are novel to me; but I am certain have been solved many times over by others. I'm having a hard time finding resources describing patterns and practices for custom file types (especially where data is changing often, the items being stored can vary in size, and I want to rewrite the smallest portion of the file possible).

Are there resources out there that you know of? Old resources would be great. The language doesn't matter. Any hints would help me save time.

The data is too volatile for something like JSON or XML to make sense, and the more I can optimize for performance the better. I will be storing more data than I want to keep in RAM at any one time, so just loading everything into RAM is not an option for this.

Thank you for any response :D

PaulHoule 6 years ago |

The authors of SQLLite are better than anyone else at writing persistence code that works despite the real-world imperfections of file systems. Either do what they do or just use SQLLite!