Open-source Best Practices in AI
Article by Violeta Misheva, Vice-Chair at the FBPML| Senior Data Scientist, and Daniel Vale, Vice-Chair at the FBPML | Legal Counsel: AI & Data Science. They are both speaking at ODSC East 2022. Be sure to check out their talk, “Open-source Best Practices in Responsible AI,” there!
There are many examples of the undesirable and detrimental consequences that can stem from the fast and reckless adoption of AI (for an overview see [1, 2]). Many of them have received wide media attention and appropriate outrage. The good news is that this has spurred a number of initiatives across focus areas (example: facial recognition technology), industries (example: high-risk industries such as healthcare, finance, and banking), and countries that seek to address the undesirable and detrimental consequences of AI adoption. The bad news is that most of the proposed guidelines and principles remain theoretical, with little guidance on how to practically apply them.
Mission of the Foundation
Motivated by this, we created The Foundation for Best Practices in Machine Learning (ML). Our mission is to
Champion ethical and responsible ML through open-source Best Practices and free public knowledge.
The way we propose to decrease the unwanted and unfair consequences of ML (complete prevention is perhaps not realistic) relies on three main pillars:
- Context: every case is different and the solution needs to carefully consider the specific situation.
- Prudent MLOps (Machine Learning Operations) and Product Management: enable and conduct thorough management of the ML model lifecycle and the product lifecycle.
- Deep organizational support: the organization cannot burden the development team with the sole responsibility of ethical product development. Instead, it should provide them with tools, policies, and resources.
Our open-source Best Practices
Best Practices (BP) are at the core of our Foundation (You can download them from our website [3]). They are a pair of documents:
- one about organizational issues, and
- one about technical issues.
The BP are not limited to an industry or a specific team within an organization. They are suitable for different audiences with varying levels of technical expertise (data scientists, engineers, developers but also legal and compliance professionals, project managers). They are also suitable for all types of organizations, regardless of the maturity, domain, size, or potential social impact of the company.
Both documents are based on the same categorization of subjects (see Figure 1 below).
Figure 1. Topics in the BPs
The how of the BP
The Technical BP focuses on the entire product. It not only includes the data or the model but also encompasses the design, integration, and overall application of the ML solution to the real world. Its audience is both technical and non-technical stakeholders.
For each of the subjects in Figure 1, the items in the Technical BP are sorted into the lifecycle phases (Product Definitions, Exploration, Development, and Production).
Figure 2. Technical BP structure
The Organisation Best Practices are scoped for the entire organization. It advises how to effectively support product teams within an organization. This support is clustered around the core subjects illustrated in Figure 1. These are approached through Policies. Management and governance aspects that are overarching receive attention as well.
Figure 3. The Organizational BP scope
Conclusion
Our work is far from complete. ML is here to stay and its effects will continue to permeate every aspect of our lives. It is up to us to ensure that automation of processes and decisions does not propagate existing societal inequalities.
We address these issues at our upcoming talk at the ODSC East. In our talk, you can expect more details about all our open-source efforts, the BPs, as well as our team, future plans, and endeavors. Please come join us!
References:
[1] Partnership on AI, AI incidents database: https://incidentdatabase.ai/?lang=en
[2] Dao D, Awful AI, https://github.com/daviddao/awful-ai
[3] The Foundation for Best Practices in ML: https://www.fbpml.org/
Read more data science articles on OpenDataScience.com, including tutorials and guides from beginner to advanced levels! Subscribe to our weekly newsletter here and receive the latest news every Thursday. You can also get data science training on-demand wherever you are with our Ai+ Training platform. Subscribe to our fast-growing Medium Publication too, the ODSC Journal, and inquire about becoming a writer.
