AutoML vs. Data Scientists

A little respect is due—in both directions.

Can Automated Machine Learning (AutoML) beat serious data scientists in producing accurate predictions? People who understand data science and programming deeply, when armed with all the tools and computer resources and time they need, are nearly always able to produce better models than generalized, self-programming AutoML systems. There are two questions to ask before you try solving problems with specialists:

How much time and money do you have?
Hand-crafting machine learning models means projects will take from days to months to complete. Expert data scientists are costly. The investment in their work must exceed the value of their predictions by the expected rate of return to make projects viable. Custom work is best suited to very-high-value outcomes where generalized tools do not produce the required accuracy, and where there is time to build and test the solutions before the questions change.

How good must your models be?
Not all predictions have life-and-death consequences. Marketing decisions, for instance, can benefit from improving accuracy from “coin-flip” to 70%. Waiting to achieve 73% with fewer false positives/negatives rarely pays compared to acting on good-enough data. When detecting serious diseases, that same 3% in accuracy with lowest false results could determine who survives. Surgical precision is essential for surgery, but you wouldn’t bake an apple pie that way.

The takeaway: We love data scientists and employ them. Their knowledge of mathematics, statistics, algorithms, and code is irreplaceable—for the appropriate tasks. Use their rare talents for your most critical work. For most ordinary business decisions, AutoML results are remarkably accurate and available in minutes at very low cost.