Machine learning isn’t for everyone
Recently, a few people forwarded me press releases for Amazon’s latest major software-as-a-service in the “cloud”. The web page for Amazon Machine Learning begins with
Amazon Machine Learning is a service that makes it easy for developers of all skill levels to use machine learning technology.
As a software developer, that sounds great. I’d probably use it if I had an application that was integrated with AWS and required some off-the-shelf machine learning. To me, deployment and management of applications is the least interesting and most frustrating part of working with software, and I am immediately drawn to anything that simplifies these or eliminates them entirely.
But, as a mathematician and data scientist, I can’t help but shake my head and roll my eyes every time a software provider claims to bring machine learning to the masses by making it “easy” or---here’s where I cringe---to ”automate” it. Machine learning software is a tool, and, as with any tool---in order to be effective---you have to know how to use it properly.
Having a set of wrenches does not make you a good auto mechanic.
Perhaps this example will help illustrate the issue for software developers: imagine that an academic statistician with years of experience coding in Matlab or R is trying to create his first web app. Have you ever met someone like this? If you knew me in 2012, feel free to say “yes”. Anyway, this statistician can employ every tool and framework designed to make web development “simple” or “easy”, but there are mainly two possibilities for the results:
The interface looks like crap - the statistician tried to build a minimal app from scratch and ended up with a white background, a few buttons, a text box, and a graph. If there are any style sheets, they merely change the font sizes.
The HTML and code look like crap - the statistician got a template from somewhere and used every JavaScript and CSS package known to man to improve the way the page looks. The code itself consists of a bunch of copy pasta that can’t be modified in one place without being modified in three others.
In other words, the application may be good in one way but fail completely in another. A well-designed application has both an efficient architecture and a good user experience. Sacrificing one of these is generally frowned upon. The point is: without a thorough understanding of both the underlying concepts of application development and practical experience with relevant tools, an ambitious person is bound to make a critical mistake, even if the initial result looks fine. There is a reason that good application developers have years of experience and education.
It is the same with machine learning software. If you don’t understand the underlying concepts of machine learning, you are bound to make a critical mistake when trying to use it. The underlying concepts of machine learning are, essentially, mathematics and statistics. You don’t need a degree in either of these to use machine learning successfully, but if you don’t have much relevant experience or education, it is best to work with someone who does.
Someone using machine learning for the first time---much like our first-time web developer above---typically finds himself in one of two common situations:
Most or all of the results are heavily biased - the algorithm might depend heavily on certain aspects of the data, causing all resulting classifications or predictions to be skewed in one direction or another.
Most results are good, but edge cases fail completely - data points or cases that are out of the ordinary cause classifications or predictions that are obscenely far from what is realistic.
Both of these are bad. What’s worse is that it is often hard to recognize when things go wrong. Getting results out does not mean these results are correct. Furthermore, if you don’t understand the underlying concepts and algorithms, how will you detect and diagnose any problems?
Don’t let your machine learning application be the analytic equivalent of a horrible web app. Consult a statistician.
