How To Prevent AI Project Failure

In the last five years alone, so much has changed in our ability to build AI systems. Years ago, companies required teams to build very specialized parts of ML models: One team was dedicated to scraping the data, another was parsing the data, a third was understanding grammatical context, and so on.

Now, an organization has all the tools it needs to build a system that can categorize feedback and learn from examples to generate responses — at a fraction of the cost and time it would have taken five years ago.

Yet with all this advancement, we’re still seeing up to 92% of AI projects fail to yield business results.

This negative correlation between AI project complexity and success is a major obstacle forward-thinking companies must overcome if they want to see positive results with AI. Continue reading to learn more about top AI challenges shared by companies today — and how to solve them.

The Correlation Between AI Project Complexity and Risk

The advanced algorithms and machine learning models that we’re designing today have enormous applications across industries. With GPT-3 alone, we can produce summaries of complex content, teach models to accurately label customer feedback, and structure information from open text. The data used to train today’s ML models can be sourced from simple spreadsheets to more complex audio recordings or video footage.

But as data and project goals increase in complexity, it becomes much harder to manage risk.

When risk becomes unmanageable (and unpredictable in some cases), we see an increase in AI project failure for businesses. As AI continues to advance, so too must the companies and teams that are interested in adopting it.

How To Solve Top AI Challenges and Prevent AI Project Failure

Project complexity is just one factor that impacts AI success rates. Companies that struggle to see measurable results from their AI investments often also share the following challenges.

• Data and AI literacy is low among the workforce.
• The path to business value is often unclear.
• Data science project management is iterative — you can’t set it and forget it.
• Managing AI risk is more comprehensive than most companies assume.

Fortunately, there are a number of ways companies can position themselves to combat these challenges and experience success with machine learning models. Here are a few solutions.

1. Cultivate Data and AI Literacy

Fewer than 25% of the workforce would consider themselves data literate. Defined as the ability to assess, understand, and utilize data, data literacy is a skill that directly enables individuals to work with tools like machine learning models.

Cultivating data and AI literacy within your organization will significantly improve AI adoption rates and employee trust in AI-based initiatives. Here are just a few strategies to achieve greater AI literacy.

• Launch an internal AI summit (in-person or virtual) that includes a series of informational speakers, vendor demos, and hands-on workshops.
• Create working groups across business and data science teams to discuss AI use cases.
• Follow and learn from industry thought leaders like Cassie Kozyrkov, who produced the Making Friends with Machine Learning series, or Andrew Ng and his course AI for Everyone. Their playful style of explaining complex concepts in simple ways can either be used directly as training or inspire ways for you to adapt new content specific to your industry or organization to help stakeholders ‘get it’.

2. Clearly Define Your Business Value

Oftentimes, companies will have the right data, design an adequate model, and identify the level of accuracy the model can achieve, but the team does not spend enough time considering how humans might interact with that model. Thus, the company does not have a complete picture of the expected ROI for the project.

For example, a model designed to predict hospital readmissions might be able to accurately identify 70% of potential cases, but without also factoring in the effectiveness of the provider’s outreach initiatives, we cannot determine the true success rate of the project.

When developing your AI strategy, be sure to account for how the AI’s recommendations will be interpreted and used by your team. How can you ensure your team trusts and effectively uses the information? What is a reasonable success rate after factoring in all information?

3. Understand the Journey to AI Is Iterative

AI strategy and design can often be broken down into two processes:

1. Design. Where you are working to build a statistically valid model that can solve your problem. This process often requires experimentation with data and redefined requirements based on revealed constraints.
2. Develop. Where you are developing the solution and translating it into the hands of the end user(s).

One of the most important phases of AI design is building resilience. You will likely encounter instances where data in the real world doesn’t match the training data used to build the model. Or, you may realize decision makers or other end users don’t trust the model enough to use it. Working through these challenges to design a resilient, trustworthy model will result in higher success rates compared to companies that ignore the complexity of the AI process.

4. Mitigate Unintended Bias and Risk

As AI solutions have increased in complexity, we’ve seen an equal increase in companies (even large enterprises) facing the unintended consequences of unchecked AI. Because models are probabilistic and advancing at a rapid pace, many teams today struggle to even imagine ways in which they could go wrong.

Regardless, there are steps that can be taken to enhance your organization’s risk mitigation and bias prevention efforts.

• Involve diverse humans in your feedback loop.
• Test your AI against unexpected situations.
• Understand the financial, physical, and emotional costs of undetected bias in your solution.

Algorithms and machine learning models will only continue to evolve over time and regulations like the EU AI Act will only become more stringent. Organizations and data scientists alike must develop processes to better understand AI, and mitigate the potential risks it poses to all stakeholders.

About the Author on AI Project Failure: Cal Al-Dhubaib is a globally recognized data scientist and AI strategist in trustworthy artificial intelligence, as well as the founder and CEO of Cleveland-based AI consultancy firm, Pandata.

Originally posted on OpenDataScience.com

Read more data science articles on OpenDataScience.com, including tutorials and guides from beginner to advanced levels! Subscribe to our weekly newsletter here and receive the latest news every Thursday. You can also get data science training on-demand wherever you are with our Ai+ Training platform. Subscribe to our fast-growing Medium Publication too, the ODSC Journal, and inquire about becoming a writer.