If I could do it all over again:
1. I'd try everything in my power to avoid taking on any debt, much less "significant debt." Go to a State school, go to a school near home so you can stay with mom & dad, work a part-time job during school, etc. Actually, I did all of these things, and did manage to avoid taking on a large amount of debt ($3k was all I ever borrowed).
2. Given (1) above, I'd drop out of school at the drop of a hat... in practical terms, that probably would mean "if I could find one other person crazy enough to work on the startup with me" then I'd drop out and do the startup. If we could hack nights and weekends for a while, get a prototype out and demonstrate some traction first, so much the better.
3. If (big IF) funding were available (outside investors, a paying customer, etc.) then it would be a no-brainer to me to quit and do the startup.
Looking back, I did drop out, just not for a "startup" per se, and I've never really regretted it. The only reason I'd go back and finish my degree at this point is just so I could say I did it (I actually do prefer to finish things I start).