Plenty of people do start their own custom engines. The problem is, a general purpose game engine of any relevant quality is comparable in complexity to a modern web browser, most game developers don't have the time or talent to handle that, and once they get past a gee-whiz renderer and maybe loading textures, they wind up in the weeds and never ship anything again.

Don't get me wrong - I've done it. I've got the corpses of several "game engines" including the ersatz one I'm building around a roguelike that I haven't and may never finish, but it isn't a satisfactory general solution for most people.

The problem is, it's a massively difficult undertaking to make a custom 3D game engine. Rendering alone is a huge topic without getting into the specifics of interfacing with GPU drivers. Window management is incredibly complex, especially if you want to support multiple platforms, or multiple modes of input. And then once you have all of that figured out, you still have to make some system to integrate the state of your game with all of those systems, and then finally, add your game logic on top. All while still building out those lower level systems as you go.

This is way easier to do in 2D than it is in 3D, largely because most people have a better grasp of 2D geometry than 3D, but even just making something OG Mario Bros. needs a fairly substantial game engine. Sure you can code golf it into something tiny, but then you don't have an engine, you have a game program. Engines need a certain level of flexibility to explore.

And sure, there are libraries for doing all of those things, but wiring them up together often ends up with piles of adapter code between the libraries to let them all use each other's data structures, at which point natural groupings of libraries form, and you're effectively back to having a game engine, just by defacto standard instead of intentional assembly.