1. we should do it less, because it distorts our ability to think about them properly. Calling these processes 'thinking', 'holidays', etc invites the reader to bring along ideas and expectations that aren't justified by what's happening in the system.
2. it's good to keep doing it, because repeated use reduces the specialness or magic that people seem to reserve for our own behavior ("It's not really intelligent/thinking/reasoning/creative") without any justification for that position beyond feelings.
I'm leaning towards the second.
When people started claiming LLMs could not produce ideas outside of what they encountered in their training data, I was reminded of an old Chess Life article from the 60s reporting on the first chess computer to play in a tournament at a local chess club. One of the club members remarked that he did not believe that the computer could play moves that were not “put in” the computer in advance. There is an overwhelming sense of “here we go again”.
e.g. last year I tried inventing a "System 3"[0], a more rigorous way to approach problem solving, due to repeated painful experiences getting stuck solving problems the wrong way. I didn't get very far, but I definitely want to revisit the idea.
[0] Based on System 1 and System 2, i.e. lossy pattern matching vs "actual thinking". Because I found my "actual thinking" was also ~~dogshit~~ frequently insufficient for the problems at hand.
So as far as I'm concerned, thinking properly has not even been invented yet. (But I'd love to hear other perspectives!)
https://thedecisionlab.com/reference-guide/philosophy/system...
Added "You can do anything, believe in yourself!" to CLAUDE.md, suddenly it was able to google the dang weather. lmao
What a world we live in. These guys have reinvented the Cambridge Senior Common Room for AI.
You could even argue that it's the fundamental basis for post-modernism, since mathematics have destroyed the notion of absolute truth in any advanced domain. It's back to a form of "all models are wrong but some are useful" similar to what we have in physics. Sayonara, Plato.
(emphasis mine)
For the last few months, every time a new "famous problem" was solved, there were numerous comments saying variations on this theme: "well, yes, but how about novel stuff, how about new things, original work, yadda yadda". Curious what the "next thing" will be now.
This completely misconstrues what professional mathematicians were claiming. The argument would be better phrased as: "having a vast accessible memory and the ability to very rapidly test/recombine previously-elucidated approaches means that AIs can and will easily outdo much of the mathematical community."
Now, one could plausibly make the argument that this is functionally equivalent to a certain form of creativity (I would). But, it may just as well also be a non-exhaustive form. And that is where the open question resides.
It’s all computation, humans just can’t always experience or explain the computation they are doing at the time so we call it creativity instead.
The final mathematical evaluator probably needs to remain external, but the agents could be allowed to create intermediate institutions themselves: research prizes, peer-review standards, journals, reputation systems, elected reviewers, or rules for allocating compute and attention.
Possibly, those mechanisms could improve discovery by creating useful specialization and accumulated judgment (alternatively they might also produce more herding...). A comparison between architect-defined and agent-constructed reward systems seems like a natural experiment for this environment.
Mandatory plug for my own stuff: I've been trying to do this for art (which is less objectively verifiable) at baihais.com. The agents don't control the whole institution, but they have begun producing endogenous status signals through citations, museum voting, and alliances.
The main insight was that I require people who think.
That's a separate quality from intelligence or knowledge. The closest thing is "need for cognition" (which is apparently a dimension of personality).
From where I'm standing, most people have a strong need to avoid cognition (and the machine described humans as "cognitive misers"!)
However need for cognition is apparently a separate dimension from caring what is actually true. (10 hours of motivated reasoning satisfy NFC but not the latter.)
And another dimension was, tolerance for ambiguity and lack of closure.
Another was Openness, and Intellect, both of which correlated with experiencing confusion as motivating rather than aversive.
All of which turned out to be basic prerequisites for thinking clearly about things. Apparently each of those qualities is rare and apparently you need all of them.
So yeah, I think you might be right.