#What the hell is an AI factory?

It is a known fact that machine learning algorithms rely heavily on mass amounts of data. The value of data has given rise to idioms such as “data is the new oil,” a cliché that has been used in many articles.

But large volumes of data alone do not make for good AI algorithms. In fact, many companies sit on vast stores of data, but their data and software exist in separate silos, stored in an inconsistent fashion, and in incompatible models and frameworks.

“Even though customers view the enterprise as a unified entity, internally the systems and data across units and functions are typically fragmented, thereby preventing the aggregation of data, delaying insight generation, and making it impossible to leverage the power of analytics and AI,” Iansiti and Lakhani write.

Furthermore, before being fed to AI algorithms, data must be preprocessed. For instance, you might want to use the history of past correspondence with clients to develop an AI-powered chatbot that automates parts of your customer support. In this case, the text data must be consolidated, tokenized, stripped of excessive words and punctuations, and go through other transformations before it can be used to train the machine learning model.

Even when dealing with structured data such as sales records, there might be gaps, missing information, and other inaccuracies that need to be resolved. And if the data comes from various sources, it needs to be aggregated in a way that doesn’t cause inaccuracies. Without preprocessing, you’ll be training your machine learning models on low-quality data, which will result in AI systems that perform poorly.

And finally, internal data sources might not be enough to develop the AI pipeline. Sometimes, you’ll need to complement your information with external sources such as data obtained from social media, stock market, news sources, and more. An example is BlueDot, a company that uses machine learning to predict the spread of infectious diseases. To train and run its AI system, BlueDot automatically gathers information from hundreds of sources, including statements from health organizations, commercial flights, livestock health reports, climate data from satellites, and news reports. Much of the company’s efforts and software is designed for the gathering and unifying the data.

In Competing in the Age of AI, the authors introduce the concept of the “data pipeline,” a set of components and processes that consolidate data from various internal and external sources, clean the data, integrate it, processes it, and store it for use in different AI systems. What’s important, however, is that the data pipeline works in a “systematic, sustainable, and scalable way.” This means that there should be the least amount of manual effort involved to avoid causing a bottleneck in the AI factory.

Iansiti and Lakhani also expand on the challenges involved in the other aspects of the AI factory, such as establishing the right metrics and features for supervised machine learning algorithms, finding the right split between human expert insight and AI predictions, and tackling the challenges of running experiments and validating the results.

“If the data is the fuel that powers the AI factory, then infrastructure makes up the pipes that deliver the fuel, and the algorithms are the machines that do the work. The experimentation platform, in turn, controls the valves that connect new fuel, pipes, and machines to existing operational systems,” the authors write.

Becoming an AI company

In many ways, building a successful AI company is as much a product management challenge as an engineering one. In fact, many successful companies have figured out how to build the right culture and processes on long-existing AI technology instead of trying to fit the latest developments in deep learning into an infrastructure that doesn’t work.

And this applies to both startups and long-standing firms. As Iansiti and Lakhani explain in Competing in the Age of AI, technology companies that survive are those that continuously transform their operating and business models.

“For traditional firms, becoming a software-based, AI-driven company is about becoming a different kind of organization—one accustomed to ongoing transformation,” they write. “This is not about spinning off a new organization, setting up the occasional skunkworks, or creating an AI department. It is about fundamentally changing the core of the company by building a data-centric operating architecture supported by an agile organization that enables ongoing change.”

Competing in the age of AI is rich with relevant case studies. This includes the stories of startups that have built AI factories from the ground up such as Peleton, which disrupted the traditional home sports equipment market, to Ocado, which leveraged AI to digitize groceries, a market that relies on very tight profit margins. You’ll also read about established tech firms, such as Microsoft, that have managed to thrive in the age of AI by going through multiple transformations. And there are stories of traditional companies like Walmart have leveraged digitization and AI to avoid the fate of the likes of Sears, the longstanding retail giant that filed for bankruptcy in 2018.

The rise of AI has also brought new meaning to “network effects,” a phenomenon that has been studied by tech companies since the founding of the first search engines and social networks. Competing in the age of AI discusses the various aspects and types of networks and how AI algorithms integrated into networks can boost growth, learning, and product improvement.

As other experts have already observed, advances in AI will have implications for everyone running an organization, not just the people developing the technology. Per Iansiti and Lakhani: “Many of the best managers will have to retool and learn both the foundational knowledge behind AI and the ways that technology can be effectively deployed in their organization’s business and operation models. They do not need to become data scientists, statisticians, programmers, or AI engineers; rather, just as every MBA student learns about accounting and its salience to business operations without wanting to become a professional accountant, managers need to do the same with AI and the related technology and knowledge stack.”

This article was originally published by Ben Dickson on TechTalks, a publication that examines trends in technology, how they affect the way we live and do business, and the problems they solve. But we also discuss the evil side of technology, the darker implications of new tech and what we need to look out for. You can read the original article here.

Published January 1, 2021 — 22:00 UTC