How to write good prompts: using spaced repetition to create understanding(andymatuschak.org) |
How to write good prompts: using spaced repetition to create understanding(andymatuschak.org) |
Good authors take care with their prose and use nuance to get their point across, which is often lost in the translation to a few bullet points. This subtlety is often the meat of the text, and the high-level points are the bones that hold it all together: essential, but not where the real value is.
My Anki deck is mostly cloze deletions automatically extracted from ebooks, where I've imported the entire text ~20 lines at a time. I've deliberately set up the review schedule so that it's hard for me to memorize particular phrases: 1 week for the first review and 4 weeks for the second, with a 1 day delay for misses. This forces me to remember what role the passage plays in the wider narrative (argument for non-fiction) to successfully predict the blanked-out word.
With this setup, I'm noticing new connections between different sections of the same book, and between different books, every time a card comes up for review: My understanding of the world has changed since I last saw it, and so I notice different subtleties than I did the first time around.
Instead of viewing them as a chore, I'm now actually excited to do my Anki reviews every morning-- where they used to make me mentally exhausted, they're now a warm-up that gets my brain started for the day.
Though this engagement itself aids memorization...
I'm not a fan of spaced repetition, instead preferring to understand material and find connections (associations) integrating it within my existing knowledge. This makes it usable, not just recallable. It's holographic, in that I can also recover it from other things I know, like the shape of the missing jigsaw piece, or a theorem or proof.
However, I've come to realize that the very act of engagement to understand is, in itself, aiding memorization.
The skill most people use formal spaced-repetition methods for is factual recall, to be sure, but the underlying principles can be applied to any human endeavour. My Anki deck, for example, is designed to exercise reading comprehension. For cooking, I don't try to memorize recipes like the article demonstrates; I do, however, try to revisit and make the dishes in my repertoire semi-regularly to reinforce the operational cues that are impossible to put into words. It's similar for all of my other hobbies: I naturally tend to have a period of intense engagement with them at first that trails off as my confidence builds. I seem to revisit most of them on the same sort of exponential schedule without really trying, as if my subconscious understands when it's time to have a refresher.
This is very interesting! Can you say more about the task you perform when you see these prompts? How do you decide how to "grade" your response? Presumably it's not just whether or not you can remember the deleted word(s)?
Also: are you blanking out individual words? Phrases? Sentences? An example would be interesting if you're willing to share!
-----
For reference, here's the next new card for one of the books I'm reading:
Bleak House
Charles Dickens
CHAPTER III
---------------------------------------------------------------------
coach gave me a terrible start.
It said, "What the de-vil are you crying for?"
I was so frightened that I lost my voice and could only answer in a
whisper, "Me, sir?" For of course I knew it must have been the
gentleman in the quantity of wrappings, though he was still looking
out of his window.
"Yes, you," he said, turning round.
"I didn't know I was crying, sir," I faltered.
"But you are!" said the gentleman. "Look here!" He came quite
opposite to me from the other corner of the coach, brushed one of his
[...] furry cuffs across my eyes (but without hurting me), and showed
me that it was wet.
"There! Now you know you are," he said. "Don't you?"
"Yes, sir," I said.
"And what are you crying for?" said the gentleman, "Don't you want to
go there?"
My response, having never seen the passage before, was "coat's"; the correct word is "large". The substitution doesn't meaningfully alter the passage: The only thing we know about the gentleman at this point is that he's a stranger whom Esther has just met in the coach, wearing a large coat (cloak?). This is the first time he speaks in the book.
I selected "again" because this is a new card; if this had been a review, I might have deemed it acceptable. The bar for similarity varies depending on the book the passage came from, my mood on any particular day, and how much I care about the contents of the passage. The Rime of the Ancient Mariner, being poetry, doesn't get any latitude: I must remember the word exactly. Some of the tedious battle descriptions in Le Morte d'Arthur get a pass no matter how badly they go wrong.
If a card is causing me trouble, there are a few corrective actions I can take: In some instances I'll edit the card to blank out a more meaningful word than the one randomly selected, which is an easier and more engaging review task. If I can't figure out the blanked word from context, I'll look it up in a dictionary; this happens most often with my second-language texts. And in extreme cases, I have enough coverage of each book that permanently dropping card or two won't cause problems.
The idea for this came from the paper[1] that introduced the concept of Cloze deletions. They were originally envisioned as a readability score. Instead of calculating statistics about the words used, like traditional methods, Taylor proposed to blank out every Nth word and measure how many could be guessed correctly by a reader. Readability and comprehension are two sides of the same coin. They both rely on observing the relationship between text and reader. If used to evaluate the text, this is called readability; if to evaluate the reader, comprehension.
[1] https://www.gwern.net/docs/psychology/writing/1953-taylor.pd...
I read a couple pages of the friendly article, waiting and waiting for the missing vocab word to be filled in.
I reluctantly opened the link that was an introduction to spaced repetition.
I opened the Wikipedia article for spaced repetition.
I searched on DuckDuckGo for "Anki prompt" and "Anki what is a prompt"
I opened the Wikipedia article for Anki.
What the hell is a prompt? Is it the question on the flashcard? Does Anki show you pop-ups periodically? There's no onboarding ramp for someone who doesn't know what spaced repetition is to read this article about 'prompts'
I skimmed through to understand what it's about or at least to know what a "prompt"/"spaced repetition" is. I still don't know, but I bet it's something very trivial.
This might help:
> This guide assumes basic familiarity with spaced repetition systems.
The words "prompt"/"spaced repetition" are basic jargon for such systems.
But to make this comment at least a bit helpful, too: An example of a "prompt" in a spaced repetition system is the front of a flashcard. It's "spaced repetition" because you look at them multiple times, spaced out over time.
Clicking through to the next article, "spaced repetition" itself seems to be a flash card based learning technique.
I wish more people would define the uncommon abbreviations and terms they use for the convenience of their readers.
Creating and generating good questions and answers is a skill. Too vague, you don't know if your answer is incorrect or not. If your answer require too much, you're never going to get the answer right, and so forth.
I was very wrong.
Ironically the solution already exists but no one uses it because of the editorial overhead: hyperlinking text to their definitions. Also, there are many ‘types’ of linking and they all look the same (blue underlined text by default). I’d love a more subtle “context” link to be natively supported. Maybe one day the semantic web will save us all...
SRS simply takes advantage of how spaced repetition works.
Maybe it is time to stop digging.
- obscuring language
- missing references and/or imprecise citations
BTW your link does not explain what a prompt is.
Your objection is a little like complaining that a computer science textbook aimed at post-grads doesn't bother to explain what a variable is. Some background knowledge is assumed.
There are many examples of prompts and answers in the main article, with blue backgrounds, and prompts about the article's content with red backgrounds.
If you have a blog post describing your process, I'm sure I, and many others, would love to read it.
A "normal" length book, like a genre fiction or popular science book, generally produces about 1000 cards like the one above. At a pace of 1 new card per day, that's about a three year commitment. It's a lot easier to have more books going than to increase the pace on any single book; the brain likes variety. I've got about 20 books going at once, which is around an hour of reading every day, including both reviews and new material. Overall, that works out to an average pace of 1 "standard" book every two months.
As for why, it started as an experiment to fix several problems I was having at the same time: My preexisting Anki deck was running dry, but still contained items that I wanted to keep reviewing; I needed a source of low-effort cards to keep the review habit going. I also had a long list of books that I should read someday but that day never seemed to be getting any closer; I decided to force the issue.
And finally, I had been unable to figure out how to make flashcards for literature at all. What series of questions / prompts can you write that captures the essence of something like this:
It is an ancient Mariner,
And he stoppeth one of three.
'By thy long grey beard and glittering eye,
Now wherefore stopp'st thou me?
The Bridegroom's doors are opened wide,
And I am next of kin;
The guests are met, the feast is set:
May'st hear the merry din.'
He holds him with his skinny hand,
'There was a ship,' quoth he.
'Hold off! unhand me, grey-beard loon!'
Eftsoons his hand dropt he.
He holds him with his glittering eye—
The Wedding-Guest stood still,
And listens like a three years' child:
The Mariner hath his will.
The Wedding-Guest sat on a stone:
He cannot choose but hear;
And thus spake on that ancient man,
The bright-eyed Mariner.
(...)
If I may, how do you ensure you're only getting one or two cards per book? Do you create a deck per book, or is all in one deck and there's some trickery in the review settings you do?
Additionally, it's really interesting to me that there is a non-linear approach for the reviewing, what's the impact on your enjoyment of books? And, does this approach give you better retainment of details?
Finally, is the length of text set, and how did you determine what it should be?
Sorry for the interrogation