I tried to switch from aufs (which performs poorly with writes) to the "recommended" filesystem overlay2 (and later overlay) but the container wasn't building anymore.

It seems that using overlay or overlay2 file systems corrupts data between intermediate containers during build.

I found the solution: https://github.com/docker/for-linux/issues/72