Author here. I gave ChatGPT very little information about the book—just vague prompts asking for its opinion—and it produced what I think is a genuinely interesting reading of the book's structure. I'm not sure whether to call it criticism or mirroring. Curious what HN makes of it. The full conversation is at the link.
(NB: I did provide ChatGPT with a 38 page extract after the initial answer. There are a couple of reviews out there and I have an author website. Nevertheless, I was genuinely surprised (alarmed?) by the subtelty it displayed in teasing out themes that to my knowledge are not available on the web).
Agreed; the frontier models have gotten remarkably good at these sorts of analyses. I write fiction using AI, and I have a very involved process of humanization (remove AI tics, etc.) including running an ensemble of AI models, and it's pretty remarkable what they turn up. Note: I always mark the works as AI generated; my goal is to have good stories, not deceive.
I had this experience having AI critique my poetry back when GPT4 came out. The right prompting can go a long way (and these days, apparently not even that :))