Blog Post

Time for Data Science to Benefit Sysadmins

January 14, 2015 Author: Amir Halfon

Last week we predicted that 2015 would be the year that analytics finds its way into IT Operations. We claimed that with the mass adoption of next generation infrastructure and the complexity that comes with it, the only way IT will be able to realize efficiencies of scale is through the power of analytics, which will become essential to ensure the SLAs that the business requires.

Donnie Berkholz echoes that notion in this excellent blog post, claiming that “As scale and complexity increase with companies moving to the cloud, to microservice architectures, and to transient containers, monitoring needs to go back to school for its Ph.D. to cope with this new generation of IT”. We couldn’t agree more!

He specifically calls out statistics and machine learning as the way to cope with the vast amounts of data generated by machines in this new era, but ends up recommending that sysadmins take courses in data science. This is where we beg to differ. We don’t think sysadmins need to become data scientists any more than marketers or accountants or sales people should become data scientists. They should be able to reap the benefits of machine learning and statistics by using modern tools that incorporate these mechanisms. These tools should encapsulate the data science, exposing the results via visualization and alerting that surface the important trends in the data.

Unfortunately, this kind of analytical capability isn’t something you can just bolt onto existing monitoring tools, as it requires a modern platform that treats data science and statistics as first class citizens. It also requires managing the data as a time-series of events. (This is akin to managing and analyzing trade data in a financial trading system, where quotes are managed as event streams that can be sliced and diced in different ways). And it needs to access ALL your data, so that algorithms can learn from the full historical data set.

In short, for IT Operations Analytics to become a reality, a new generation of tools (rather than sysadmins) needs to emerge, tools which are built around analytics from the ground up. That’s our mission at ScalingData; to learn more about what we’re doing to empower sysadmins, contact us at