Pedro Domingos on the Quest for the Ultimate Machine Learning Algorithm

As newer fields emerge within data science and the research is still hard to grasp, sometimes it’s best to talk to the experts and pioneers of the field. Recently, we spoke with Pedro Domingos, Professor of computer science at the University of Washington, AI researcher, and author of “The Master Algorithm” book. In the interview, we talked about the quest for the “ultimate machine learning algorithm.” You can listen to the full Lightning Interview here, and read the transcript for two interesting questions with Pedro Domingos below.

How close are we to a “Holy Grail,” aka the Ultimate Machine Learning Algorithm?

We are definitely close than when I wrote the book. It’s been quite amazing and gratifying to observe. First of all, there are these companies like DeepMind whose explicit goal is to come up with a master algorithm.

They want to come up with a learning algorithm that will be able to do everything from science to playing games to you name it. Then there are also these efforts from companies like Google prominently, but also Meta, OpenAI, and DeepMind, to have a learning system that will learn from all different kinds of data.

Gato from DeepMind is a good example of this. It’s a Transformer — a very popular architecture these days — but it doesn’t just learn from texts or learn to say things. It learns from text, images, and robot actions, and then it learns what to do. So if that were to succeed, it would be the master algorithm.

Again, the other companies have similar efforts now, but I don’t think those things are the master algorithm yet because of a number of limitations and issues that we can touch on, but clearly, finding the master algorithm has gone from being sort of a crazy idea that a few people have to what a lot of these companies with a lot of resources are gunning for. They’re gunning for it for very good reasons because whoever gets that first will have a very bright future.

Do you think it will emerge from existing ideas or from research and ideas to come?

First, we need to combine those five major paradigms (discussed earlier in the interview), one way or another, and there are a lot of different approaches. We have a good idea of how to combine all five and that’ll mean scaling up will be a major issue that will need to be solved. My intuition says that’s not going to be enough. There are people at one end of the spectrum who say that paradigm is all you need.

There are some people in deep learning today who say you can do anything with backpropagation. I have this ongoing discussion with one person who says gradient descent is the only thing you need for deep learning. Most people are not that optimistic that a single paradigm is dominating, and members of that paradigm who say that they’re going to run this all the way to human-level AI. Maybe that’ll happen, maybe not, but I think we need to combine all of them.

Like other fields, such as physics where we know what the main forces are, once we unify them all your full account will be there. The standard model captures three, then you add gravity, and you’re fine. But in AI I don’t think that will be done. There are really important ideas that we need to have the master algorithm that has not been discovered yet. In fact, some of my research right now explores exactly that.

It’s important to know that many parts of AI and machine learning came from some pre-existing fields, such as philosophy and logic, physics, neuroscience, connections from evolution, vision statistics from psychology, etc. I feel this maturity developed from its own ideas, not just porting over ideas from other fields, and I think that’s yet to happen in machine learning. So the next 10 years are going to be more exciting.

How to learn more about machine learning

If you want to be like the legendary Pedro Domingos and define the future of machine learning, then it’s time to make sure your current machine learning knowledge is up-to-date to build a solid foundation. At ODSC East 2023, there will be a number of sessions as part of the machine & deep learning track that will cover the tools, strategies, platforms, and use cases you need to know to excel in the field. Some sessions include:

• An Introduction to Data Wrangling with SQL
• Resilient Machine Learning
• Machine Learning with XGBoost
• Idiomatic Pandas
• Introduction to Large-scale Analytics with PySpark
• Programming with Data: Python and Pandas
• Introduction to Machine Learning
• Mathematics for Data Science
• Using Data Science to Better Evaluate American Football Players
• How to build stunning Data Science Web applications in Python — Taipy Tutorial
• Towards the Next Generation of Artificial Intelligence with its Applications in Practice
• Introduction to AutoML: Hyperparameter Optimization and Neural Architecture Search
• A Practical Tutorial on Building Machine Learning Demos with Gradio
• Uncovering Behavioral Segments by Applying Unsupervised Learning to Location Data
• Beyond Credit Scoring: Hybrid Scorecard Models for Accuracy and Interpretability
• Advanced Gradient Boosting (I): Fundamentals, Interpretability, and Categorical Structure
• Advanced Gradient Boosting (II): Calibration, Probabilistic Regression and Conformal Prediction
• Getting Started with Hyperparameter Optimisation
• Generating Content-based Recommendations for Millions of Merchants and Products
• Machine Learning Models for Quantitative Finance and Trading

Originally posted on OpenDataScience.com

Read more data science articles on OpenDataScience.com, including tutorials and guides from beginner to advanced levels! Subscribe to our weekly newsletter here and receive the latest news every Thursday. You can also get data science training on-demand wherever you are with our Ai+ Training platform. Subscribe to our fast-growing Medium Publication too, the ODSC Journal, and inquire about becoming a writer.