One thing people often do, and it does help to a certain extent is to write/blog about the books you read. Summarize the essentials for your readers, it'll serve them as inspiration for new reading material and you as a perfect refresh of what you actually read and worked through.

Stop trying to read fast like a moron.

As you see bits and pieces of text that you want to remember, you should be internalizing those words, thinking about their true meaning, asking yourself if you agree/disagree, and thinking about how you would approach the same thing in a different way.

The key to memory is meaningful association.