A First Course in Linear Optimization (2013)(umich.app.box.com) |
A First Course in Linear Optimization (2013)(umich.app.box.com) |
A great way to get started is to play around with the solver feature in Excel. Many software engineers may be loath to use this tool, but Excel actually provides a great GUI with which to do simple linear and integer programming problems.
Take any undergrad algorithms class: 99% of it is combinatorial. There's an equal (probably larger) parallel universe of numerical thinking and numerical algorithms which is either not taught, or taught as a secondary class under the names of 'numerical analysis', 'scientific computing', and so on. None of stacks, queues, trees, and graphs are going to help you out when you have to discretize a differential equation or solve the resulting linear system with correctly enforcing the boundary conditions. You could ace an algorithms class and not have a clue how to begin tackling those kinds of problems.
This is especially relevant today when machine learning is heavily dependent on computational linear algebra and other numerics, and a typical CS student is not trained for that.
And what about a typical CS prof? No, I won't name any names, not today.
LP and NLP do seem to be treated as a niche subject, something that would be taught in an Industrial Engineering (or barring that, a math) class rather than as a core CS topic.
Interesting that you mentioned excel - I agree that this is a good way to learn and play around with LP, but normally I'd have a good python library to point to. I used a python library a while back to do some homework problems, and I wouldn't' be surprised if it has improved considerably since then. Still, interesting that LP and NLP aren't part of scikit-learn (http://scikit-learn.org), which covers classification, regression, clustering, dimensionality reduction, model selection, and preprocessing, but not optimization per se, even though non-linear optimization is a key bit of mathematics behind many of these algorithms (regression that uses a cost function almost certainly uses some kind of NLP, right?). It really is notable that LP is not part of the main packages, front and center.
If you go on kaggle or other data science sites, linear and non-linear optimization don't seem to get much attention, either. Wonder why that is. Anyway, interesting topic.
However, for Python I would actually recommend the PuLP package (http://pythonhosted.org/PuLP/), which has pretty good documentation and let's you use various solvers (CBC, GLPK, CPLEX, Gurobi, etc.) on the backend. This package definitely has it's advantages over Excel, especially for large or complex LP problems where I have found Excel to be much slower and/or incapable of finding as food of a solution. However, for someone learning about LP (linear programming) or integer programming, it's unlikely that they will be doing anything that can't be done efficiently enough in Excel, particularly with an add-on such as OpenSolver. I think that Excel is just easier to use than any of the Python packages that I've come across for this (hopefully that will change though!). But once someone knows what they're doing, learning how to use something such as PuLP can be very valuable in my opinion.