Visibility and Monitoring for Machine Learning Models

Visibility and Monitoring for Machine Learning Models(blog.launchdarkly.com)

64 points by heitortsergent 8 years ago | 6 comments

mav3r1ck 8 years ago |

“The tricky thing, though, is in order to get good at machine learning, you need to be able to do deploys as fast as humanly possible and repeatedly as humanly possible. Deploying a machine learning model isn’t like deploying a regular code patch or something like that, even if you have a continuous deployment system.” -Josh

Sounds a lot more like a DevOps problem then a Machine Learning problem to me+. But really, in general, this is something any one doing any sort of software deployment should be doing to begin with. If I encountered a continuous deployment system that doesn't already do this, then I usually take the time to get it as close as possible. Still haven't gotten anywhere close to Netflix's level, maybe some day.

+ This is the buzziest comment I have probably ever made.

infinitone 8 years ago |

Apart from the occasional pun or allegorical comment... frankly that presentation ended very abruptly with very little actual substance. Or maybe i was just expecting more?

mkagenius 8 years ago |

Not sure I get the context or how slack operates -- but wouldn't most of the big companies have functional tests already in place to catch erratic ML models in production. If they are deploying ML model for the first time for some new features, wouldn't they require to test the functionality and SLAs..

joe_the_user 8 years ago |

This looks very interesting in terms of addressing machine learning as application, which is to say that ordinary software has both the property "does it's job" and "can be looked-at, tested and so-forth to see how it does it's job" with machine learning systems as nominally defined having only the first property.

Edit: I originally asked for transcript, it is just video.

wellactually 8 years ago | |

The transcript is below the video, on the same page.

joe_the_user 8 years ago | | |

Just saw that. I'm so used to no transcript, I didn't look.