How to Prevent Machine Learning Models from Failing in Practice?
Have you seen machine learning solutions fall flat in practice?
Well, I have. Several times. I get occasional panic calls from teams about their 98% accurate models generating questionable predictions once released to actual users.
Did they build a bad model? Maybe.
But the real issue is that the majority of these teams skipped a step.
And that step is testing. Not just any type of testing, but post-development testing (PDT).
What is Post-Development Testing (PDT)?
Post-development testing in the context of machine learning is an experimentation period where you take a model from development and test it on real data, and often with real users. And this happens before