Social learning: Collaborative learning with large language models(blog.research.google) |
Social learning: Collaborative learning with large language models(blog.research.google) |
try it with a mindlessly boring topic you can barely get through a paragraph of, not something you're actively trying to read
Edit: misunderstood the paper on first skim, they don’t actually want smaller students, they might actually want bigger ones. So this is more a privacy thing
They said their focus was PII but I imagine Social Learning is also very useful when dealing with copyrighted content too:
- (current level) use your own model to generate a bunch of solutions to a task
- (teacher level) use RAG, web search or a larger model to solve the same task. But always use multiple sources, never drawing from a single copyrighted source example, we want to integrate information.
- (grading & teaching feedback) analyze the issues of your own model and synthesize a training example to fit the issues found - it might lack some facts, or not have some skills
This would be fair use because it studies the shortcomings of the student model compared to the more empowered teacher models. It can also check for regurgitation of the copyrighted content while formulating completely new text to fit the needs of the student.
I prefer to name the method Machine Study rather than Social Learning, but the social part is there. When LLMs get a legitimate way to process copyrighted content it would help their development.
This method will ensure the new model never sees the original copyrighted content, only the teacher vs. student analysis outputs related to that content.
I'm sure it's been or is being researched, but since I'm new to LLMs, my immediate thought was having code-aware models where one tries to find and fix vulnerabilities in a codebase while another tries to exploit them where the ultimate goal is to secure a codebase.
Social learning exist to transcend the limited processing power and limited training data exposure of individual agents through multimodal transfer of their own individual models (distilled down from an individual's entire worldview, sense-of-self, perspectives, skills, semantic memory etc)
LLMs already exploit the propositional transfer of human models over language, and they abuse their massive compute capacity to compress them all in a giant model to simulate-them-all. For sure internally it does have some notion of distribution - as it at least has to distribute the compute at train time - but this is not an agent level distribution - not to confuse with the weaker metaphor of an "agent" used in model architectures -, and the end product presents itself as a singular "agent" with all of the processing power and all the training data that is infinitely copyable.
> "A teacher model provides instructions or few-shot examples to a student model without sharing its private data."
So the real concern is not utilizing social learning to transcend compute and training data limitations, it is about creating inferior models that can be distributed back into the world without giving up all of the secret sauce.
For sure this could work, one could create inferior "agents" from stronger "agents", but we cannot create an even stronger "agent" through the dialogue of two strong "agent"s, because everything to be shared is already perfectly encoded in the model&architecture and perfectly copyable. Therefore this is not social learning at all.
To abuse back their anthropomorphization, they are trying to create a deliberately stupid kid to send out to the world so that the kid doesn't tell all the things mommy and daddy already knows and could have perfectly taught. Because one can make more money from selling/renting a bundle of differently stupid agents than a singular state-of-the-art one I guess?
