The official base Ubuntu image is really not particularly large, I'm assuming you were using an image with additional things installed? But regardless, once pulled, it is cached locally. Be sure to reference the image by specific tag/sha rather than latest to ensure you're not doing excessive pulls.
Overlayfs 1 had many issues. Overlay 2 was buggy for a long time and we needed to patch in aufs in production for stability and lock around image pulls to prevent kernel deadlocks. But at this point, overlay2 is extremely stable in the 4.x kernel series in production. Though, there is still the copy-up quirk with hard links but that affects very few applications in the wild.
If you don't like overlay filesystems, you can use the devicemapper storage driver, and if you set it to direct-lvm mode it should be pretty equivalent to VM based volumes. When it comes to mounting in shared directories from the host though, I trust docker's bind mounts much more than VM based filesystem solutions.
Anyway, if you work alone or on a small team, VMs certainly suffice and the appeal of docker may be limited, but much of the convenience of docker comes from the ecosystem and immutability of images. I've always found it frustrating that such an ecosystem was built around docker when it could have been done with VMs all along. If your company is building docker image artifacts of your software as part of the CI system and that software has many dependencies, executing a production build becomes as easy as "docker run X" and reduces the need for developers to standardize their workstations on one linux distribution. That said, while I find this incredibly useful, it's rare that I do proper development with local docker images unless I'm briefly touching something with painful dependencies (like getting the frontend stack working to make a quick UI change as a backend developer)