John Snow Labs Introduces Automated Responsible AI Testing Capabilities
John Snow Labs, the leading AI for healthcare company, has launched a no-code tool to test and evaluate the safety and efficacy of custom language models. The new Automated Responsible AI Testing Capabilities in the Generative AI Lab empower non-technical domain experts to define, run, and share test suites for AI model bias, fairness, robustness, and accuracy.
This tool is based on John Snow Labs’ open-source LangTest library, featuring over 100 test types that assess various aspects of Responsible AI, including bias, security, toxicity, and political leaning.
Leveraging Generative AI, LangTest automatically generates test cases, enabling the creation of comprehensive test suites in minutes rather than weeks. Tailored specifically for custom AI models, LangTest addresses areas not covered by general-purpose benchmarks and leaderboards.
Recent US legislation has heightened the urgency for such comprehensive testing solutions for Large Language Models. The ACA Section 1557 Final Rule, effective June 2024, prohibits discrimination in medical AI algorithms based on race, color, national origin, gender, age, or disability.
Additionally, the HTI-1 Final Rule mandates transparency in medical decision support systems, requiring companies to demonstrate their model training and testing processes. The American Bar Association Guidelines also necessitate extensive internal and third-party audits before deploying AI, responding to lawsuits against companies using AI for job candidate matching.
Many domain experts lack the technical expertise to conduct thorough AI testing, while data scientists often do not possess the domain-specific knowledge required to build comprehensive models.
The Generative AI Lab bridges this gap, enabling domain experts to create, edit, and understand AI model tests independently. The tool supports best practices such as versioning, sharing, and automated execution of tests for every new model iteration.
“There has long been a gap between how AI models should be tested and how they often are. The new Generative AI Lab helps by making it far easier for teams to deliver AI models that are safe, effective, fair, and transparent,” stated David Talby, CTO of John Snow Labs.
Originally posted on OpenDataScience.com
Read more data science articles on OpenDataScience.com, including tutorials and guides from beginner to advanced levels! Subscribe to our weekly newsletter here and receive the latest news every Thursday. You can also get data science training on-demand wherever you are with our Ai+ Training platform. Interested in attending an ODSC event? Learn more about our upcoming events here.
