7 Data Science & AI Trends That Will Define 2024

About a year ago this time, I posted an opinion piece on what I thought were going to be big trends in AI in 2023. Of those seven ideas, I think I got all of them right. Generative AI became the thing, hiring and firing was all over the place, “data scientist” isn’t the sexiest job anymore (who even knows what it is now?), and responsible AI is now at the front of everyone’s mind.

What I didn’t predict was how huge AI would be this year. I didn’t expect everyone to be using ChatGPT, for South Park to parody it, and for HubSpot’s Inbound conference in September to be more about AI than it was about marketing. Sure, AI is now in the hands of the everyday computer user, but I didn’t expect it to be used to the point of controversy and at times, disdain.

Well, I’m going to try again anyway. After attending multiple conferences this year, speaking with dozens if not hundreds of data scientists over the year, writing dozens of articles, and reading too many blogs and news pieces, here are my thoughts on what trends will define the AI landscape in 2024.

Democratization of Gen AI

As mentioned above, everyone is using AI. My uncle is flooding his Facebook feed with AI-generated landscapes daily (my fault for getting him hooked on it), talk shows are making fun of it, and I’m even using it often as an idea generator for when I get stumped. ChatGPT is now an expected tool, even listed on countless job descriptions, regardless of what the job is. AI-generated images are flooding Reddit and DeviantArt, among other sites. Just as the internet was a force to be reckoned with 20 years ago, it’s now AI’s time to shine, and I believe it won’t be as much of a surprise to see AI-generated content in 2024. It’ll just be the new norm. More and more tools will become open-source, meaning fewer paywalls or subscriptions will be required to use AI.

Increased Public Scrutiny

…however, with increased use, comes increased pressure by the public. Comments flood every new fanart posted online asking if it’s AI or not, sites are putting up disclaimers asking to not be scraped for LLM development, and everyone wants to know where their data is being fed. From celebrities warning others of AI-generated deepfakes, to the former Google CEO claiming current AI guardrails aren’t enough, alarms are being raised everywhere about the potential dangers and repercussions of AI. I imagine 2024 will see even more discussions surrounding the ethical & safe use of AI, and now that AI isn’t just a “sexy” term as everyone’s using it, it’ll face increased scrutiny from individuals up to major corporations.

More AI Legislation

Because of the increased presence of AI everywhere, governments around the world are implementing legislation surrounding AI. In November, the president of the United States signed an executive order designed to address concerns about AI, while also aiming to not hamper its development. Over the summer, China started requiring generative AI operators to require a license to create content and to be transparent about it being AI-generated. Moving forward, I expect to see more nationwide AI legislation seen around the world, but also more local plans, such as how NYC created its own AI action plan this fall.

Responsible AI Moves to the Forefront

Now that everyone knows of AI and everyone’s using it, more people are asking questions regarding their data, artists are curious about whether their art will be used in training data, and so on. Just two years ago, AI was a bit of a luxury or a niche that the general public didn’t care much about, or at least no more than any other use of data online. Now, with more and more LLMs popping up, people want to know if their data is being used to train them, with even the least tech-savvy people being curious about it. Because of this, data scientists and AI engineers will now have to be more transparent than ever about where they’re getting their data and what they’re doing with it.

Advancements in Multimodal AI

AI is an arms race like we’ve never seen before in technology. As soon as OpenAI reveals its latest version of GPT, Google brings out a new tool ala Bard and Gemini. We’re at the point of the race where simple image or text generators alone won’t suffice, and it’s the multimodal AIs that are standing out among others. These jack-of-all-trades AIs allow people to input whatever form of data they want, and get a wider variety of options back, meaning they won’t need to have different apps for different purposes. We’ll likely see more and more of these in the next year and beyond as human nature always gravitates towards simplicity and more comprehensive tools.

Domain-Specific and Small LLMs

It’s weird to say small large language model, but that’s what we’re already seeing a lot of. While broader ones like GPT, Llama, and PaLM 2 get a lot done, they’re quite broad and lack the nuance needed for specific tasks in certain industries. Domain-specific LLMs, however, tend to focus more on what the particular industry needs, including unique jargon, context, and applications. These types of LLMs are particularly useful where you can’t afford to have a huge margin of effort, such as in healthcare and law.

Quantum AI

Lastly, as technology improves, so will the potential of AI. Quantum computing can be used to develop artificial intelligence algorithms, which can lead to faster computing times and more powerful algorithms, both leading to more interesting applications and discoveries if used in research. This field is still fairly early in development, but if the arms race of multimodal AI continues at this pace, I imagine that quantum AI will be the next goal in these races.

What’s Next?

Well…who knows? I think I got lucky by guessing everything right last year, but that’s when the goal was clear that generative AI was going to be the big thing. 2024 is a little more uncertain, as there are more variables in play, and with that, more potential problems. But, with these problems and developments come innovation in governance, reproducibility, and technological advancements. Either way, I’m excited about what 2024 has in store for the realm of artificial intelligence.

With that being said, now I need to do the obligatory promo bits. I’m proud of my and my team’s work here at ODSC, and we’re aiming to make 2024 a big year.

Coming up soon is ODSC East 2024 April 23rd-25th in my home city of Boston, MA. As one of our bigger events, we’ve listened to feedback from our 2023 conferences and we’re going to take a step back from so much generative AI representation and highlight the basics a bit more, as well as our historically popular areas such as data visualization and machine learning fundamentals. We’re still going to have plenty about LLMs and generative AI of course.

We’re also bringing back the Data Engineering Summit, which will be co-located with ODSC East. This year it was only virtual, but following its success, we’ve decided to make it an in-person event as well. We also had great success with our devoted AI X Business & Innovation Summit at ODSC West, so now we’re bringing it to ODSC East — also in person. This will be a good event if you’re interested in AI but you’re not a data scientist yourself, but you want to learn about the industry, how to use/implement AI in your business or organization, and what you need to know about hiring and working with data scientists.

As of writing, tickets to ODSC East are 70% off door prices. This means you will literally save hundreds of dollars by registering now. I know that it’s a bit far away still and that we haven’t announced much going on just yet, but if you look at our past conferences, you’ll get an idea of what to expect and what you can take away from the event. It’ll be good, and I will be there as well.

Originally posted on OpenDataScience.com

Read more data science articles on OpenDataScience.com, including tutorials and guides from beginner to advanced levels! Subscribe to our weekly newsletter here and receive the latest news every Thursday. You can also get data science training on-demand wherever you are with our Ai+ Training platform. Interested in attending an ODSC event? Learn more about our upcoming events here.