Man, I find SOTA deep learning somewhat hilarious. We scale models to absurd proportions, burning through a shitload of resources just to achieve (slightly above) human intelligence. The human brain has a few billion neurons and uses as much power as a light bulb.
However, a neuron is much more than a single parameter. The brain is estimated to have from 10^14 to 5x10^14 synapses.
If I had a magic button I would not only pause AI development but set it back 10 years. Sadly I have no influence on events and those who do, don't care about the future of humankind or actively wish us dead.
huge parameter models with many small but efficient layers can work quickly on low resource hardware
similar to how neurons experience chemical spiking to activate small portions of the brain at once