Not all models need to run on the device - of course. BUT there are a few that make a lot of sense. It all depends on speed and what you want to offload to the device. Example, a more quickly responsive (even when offline) image classification (eg is this poison ivy or not) model is best on device. Or in an AR-kit app. Or privacy, like not wanting to ship a medical image across the pipe and so on.

Skafos actually leverages CoreML and the models do run on device. It's a live update to device local models in the background, or even hot swap while in the foreground.