Meta Has Introduced I-JEPA — a Human-Like AI Image Generation Model

June has been a busy month for Meta. Earlier this month, the tech giant introduced I-JEPA, the first AI model based on Yann LeCun’s vision of a more human-like AI. This comes after Meta’s Chief AI Scientist Yann LeCun proposed a new architecture intended to overcome key limitations of advanced AI systems.

According to Meta, his vision was to create machines that can learn internal models of how the world works so that they can learn much more quickly, plan ways to accomplish complex tasks, and readily adapt to unfamiliar situations.

This is where I-JEPA comes to play. As Reuters puts it, the AI “uses background knowledge about the world to fill in missing pieces of images, rather than looking at nearby pixels like other generative AI models.” To put it simply, it takes an incomplete photo, uses the provided information, and fills in the gaps.

Meta went on to explain “This model, the Image Joint Embedding Predictive Architecture (I-JEPA), learns by creating an internal model of the outside world, which compares abstract representations of images (rather than comparing the pixels themselves). I-JEPA delivers strong performance on multiple computer vision tasks, and it’s much more computationally efficient than other widely used computer vision models.”

The company went on to highlight I-JEPA’s training efficiency. The team promises that a 632-million parameter visual transformer model can be trained using 16 A100 GPUs in under 72 hours, which they claim is two to 10 times faster than other methods while delivering fewer errors.

In short, Meta is asserting that their training method using 16 A100 GPUs can train a specific model in under 72 hours. Next, Meta touched on the importance of background knowledge within human intelligence. “Our work on I-JEPA (and Joint Embedding Predictive Architecture (JEPA) models more generally) is grounded in the fact that humans learn an enormous amount of background knowledge about the world just by passively observing it.”

They went on, “It has been hypothesized that this common-sense information is key to enable intelligent behavior such as the sample-efficient acquisition of new concepts, grounding, and planning,”. Meta also stated that this new model has the ability to “avoid the limitations of generative approaches” by “predicting representations at a high level of abstraction rather than predicting pixel values directly.”

Currently, I-JEPA is still in development, but the company is emphasizing the importance of sharing components of its AI models with researchers. In their view, the move will help in aiding innovation within AI.

Originally posted on OpenDataScience.com

Read more data science articles on OpenDataScience.com, including tutorials and guides from beginner to advanced levels! Subscribe to our weekly newsletter here and receive the latest news every Thursday. You can also get data science training on-demand wherever you are with our Ai+ Training platform. Subscribe to our fast-growing Medium Publication too, the ODSC Journal, and inquire about becoming a writer.