How good could GTP-4 get if copyright was not an issue? If we could train a GTP on every book in the Library of Congress, every stock photo, every piece of art, etc. how smart could it get? What if it was trained on all intercepted data? Why would this NOT be happening right now? |
But even ignoring the above, it's not clear that training on more data will dramatically improve the quality of the model. You want high quality data to get high quality model. I'm not sure if much more high quality data exists in the world than what they already have access to.
It's possible that to make it smarter we should train it on video - predicting the next video frame might be a more challenging task, requiring different world models leading to deeper understanding of reality.
GPT4 technical paper doesn't seem to disclose it (or I didn't dive deep enough): https://arxiv.org/abs/2303.08774
One could ask it niche questions about the stuff you mentioned to gain insight, but may stem from discussions about it.
I think we're going to find out soon.
Some countries are creating fairly strict regulation limiting AI, but others are laying out clear guidelines encouraging innovation.
Japan for example has ruled that copyright does not apply to AI training, so as I understand your idea could be carried out there.
I suspect the jurisdictions with the most innovation friendly approaches will attract talent and capital and wind up with the strongest models.