How To Succeed in AI With Algorithmic and Human Guardrails
As a data scientist and entrepreneur, I’ve built a niche in the intersection of data science and effective communication with business leaders. Through this, I’ve recognized that organizations that see the most consistent success with AI have builders and users capable of understanding the potential and limits of AI, and how both will impact the organization.
In fact, technical clarity and building understanding across the entire team during AI design is one of the top ways to increase your chances of AI project success and build trust between your team and the model.
One technical concept that gets tossed around a lot is guardrails. At a recent conference, I kept hearing attendees and speakers using guardrails as a buzzword, but in many cases, it felt like the parable of the blind men describing an elephant. Technically they’re describing the same thing, but few people fully understand what goes into creating effective guardrails for AI design.
This article bridges the various conversations around guardrails into one holistic vision.
Get your ODSC Europe 2024 pass today!
In-Person and Virtual Conference
September 5th to 6th, 2024 — London
Featuring 200 hours of content, 90 thought leaders and experts, and 40+ workshops and training sessions, Europe 2024 will keep you up-to-date with the latest topics and tools in everything from machine learning to generative AI and more.
Understanding Guardrails: Algorithmic and Human
Guardrails are the set of filters, rules, and tools that sit between inputs, the model, and outputs to ensure a model aligns with your expectations of value and correctness. They are necessary checkpoints that guide the development and implementation of AI systems and help ensure data security, regulatory compliance, brand reputation, and ethical AI usage.
While many organizations (especially those with data science teams) may think of algorithmic guardrails like output filters, human or process guardrails like red teaming and ongoing monitoring are equally important in shaping the value and credibility of AI models — especially when it comes to generative AI.
>> Related Resource: How to Use Guardrails to Design Safe and Trustworthy AI
Algorithmic Guardrails
Algorithmic guardrails can be set at each point in the AI design and development process: model evaluation during training, for prompts and inputs, and for outputs. You can loosely picture them in the diagram below.
Algorithmic guardrails could include any of the following:
- Filters to detect unexpected inputs to avoid risks such as prompt poisoning or to prevent a model from operating in a situation significantly different from its training.
- Quality and correctness (outputs and inputs) to avoid model hallucination or toxic output and ensure consistent formats.
- Privacy controls (inputs and outputs) to ensure the model is only consuming information within its scope of allowable work and to prevent leaks of sensitive information.
- Fairness measures (outputs) to ensure that AI algorithms do not discriminate against individuals based on factors such as race, gender, or ethnicity.
- Transparency tools (outputs) such as explainability for complex AI-driven decisions.
- Model observability tools (inputs and outputs) to detect when input patterns or model results are consistently deviating from expectations.
Human Guardrails
In the same way that algorithmic guardrails can (and should) be applied before and after a model is launched, so too should human guardrails.
Prior to Launch
Pair algorithmic guardrails with a lot of stress testing and fine-tuning (read: red teaming), and we can look at the model and functionally determine what makes it behave in a weird way. Red teaming is a human-led activity assisted by tooling to uncover vulnerabilities in generative models.
Toolkits are rapidly emerging to assist in generative AI red-teaming, such as Fiddler.ai’s open-source Fiddler Auditor and Microsoft’s new PyRIT toolkit.
Equally important to the tools is the composition of the team. Microsoft’s AI Red Team, comprised of security, adversarial machine learning, and responsible AI experts, has gained massive efficiencies when supplementing existing Red Team practices with the tool.
Despite its benefits, Microsoft makes one thing clear: “PyRIT is not a replacement for manual red teaming of generative AI systems.” In other words, you need both automated processes and manual testing, especially for the highest-risk applications.
To do this right, companies must look at building diverse teams that are dedicated to stress testing. These tasks are too important to be a side job and the costs of treating it as such will be detrimental. And whether the red team should be internal or external should depend on the project’s level of risk.
Post-Launch
Ongoing monitoring and human-led interventions after a model is launched are highly important and even more so for high-risk environments. These activities can include:
- Live dedicated help-desk support to intervene if a model misbehaves.
- Ongoing monitoring of live results sampled and ranking to identify potentially problematic results or inputs.
- Periodic audits and evaluations of model responses and outputs to identify problematic behavior.
- Appropriate controls to roll back a problematic version of a model or take an application offline.
These processes can be informed, focused, or triggered by abnormalities from algorithmic monitoring. In the below chart, you can see how these guardrails and stress-testing efforts combine for one project.
At the end of the day, guardrails for AI models are one-part technical and one part human. While many organizations are learning to effectively establish guardrails today, emerging regulations like the EU AI Act will soon require these sorts of activities and controls for AI applications deemed to be high-risk.
It’s never been a better time to start building this organizational muscle: Invest in human experts to audit, oversee, and orchestrate change to mitigate risk and increase the potential for AI success.
ODSC West 2024 tickets available now!
In-Person & Virtual Data Science Conference
October 29th-31st, 2024 — Burlingame, CA
Join us for 300+ hours of expert-led content, featuring hands-on, immersive training sessions, workshops, tutorials, and talks on cutting-edge AI tools and techniques, including our first-ever track devoted to AI Robotics!
About the Author: Cal Al-Dhubaib is a globally recognized data scientist and AI strategist in trustworthy artificial intelligence, as well as the Head of AI and Data Science at Further, a data, cloud, and AI company focused on helping make sense of raw data.
Originally posted on OpenDataScience.com
Read more data science articles on OpenDataScience.com, including tutorials and guides from beginner to advanced levels! Subscribe to our weekly newsletter here and receive the latest news every Thursday. You can also get data science training on-demand wherever you are with our Ai+ Training platform. Interested in attending an ODSC event? Learn more about our upcoming events here.
