1) Investors and their money. It's much easier to get angel/vc money in a tech hub. Investors either are there already or frequently travel through the tech hub. For companies with investor money, investors generally prefer to be able to come into the office and see everything. Most of these investors are based near tech hubs.
2) Partnerships. If you're in a tech hub, it's easier to find companies to partner with, even by happenstance. It's also easier to sell yourself if your from SF, NYC, Boston than if you're from Boise, unfair as that is. Lastly, it's easier to convince a company to acquire you if they already have an office in that metropolitan area.
3) Talent Pool. This is really about two things: tech workers and seasoned execs. Generally speaking the talent pool is much higher in the tech hubs. The average engineer has been to multiple meetups, best practices have filtered down from the FAANG alumni into the general engineering population. Anecdotally, the people we turn down at my current job would have been "rockstars" at my previous jobs in CT and Westchester (the NYC suburbs). Execs are more likely to have been part of a high growth startup or at least have exposure to tech best practices or tech in general.
Outside of tech hubs, you're really committing to hiring junior engineers and training them, hiring consultants to fill key exec or management roles, hiring engineers from consultation companies (many of the hires that are willing to travel are international) and if you want truly great hires your going to be flying in a lot of candidates and many will still decide that they don't want to move anyway (they essentially need to decide to move away from their family, friends and work connections and rebuild that. It's a hard decision and many people back out late in the process)
A good technology analogy is that starting a company in a coastal tech hub, is like being able to use AWS cloud services. It can be cheaper to build your own data center, but being able to spin up server clusters at will and outsource database management, etc increases a company's velocity significantly. You can build a tech company outside of a tech hub (see ExactTarget), but it's harder to scale up past a certain size and exiting is much harder. Getting venture capital is harder and may be on worse terms. Finding good engineers in your stack nearby is more difficult (anecdotally it's a frequent occurrence to find companies using custom frameworks and/or best practices from 3-5 years ago, including the regional "success" stories)