Are We That Simple? Predicting Human Behavior With AI

Modern server room, corridor in data centre with Supercomputer racks, neon lights and conditioners. 3D rendering illustration

getty

A large part of what we’re doing with large language models involves looking at human behavior. That might get lost in some conversations about AI, but it’s really central to a lot of the work that’s being done now. With projects that reckon seriously with the predictive power of LLMS, we can see how predicting human behavior represents what these technologies can do well.

But to do that, you have to cut through the noise, and get to the signal.

How does that work?

Some Modern Efforts

Various projects are aimed at this concept right now. For example, there’s the Centaur model, built at the Helmholtz Institute for Human-Centered AI. This used Meta’s Llama 3.1 and a natural language data set featuring 10 million choices from 60,000 people. which gave the model its food for thought.

Centaur got top rankings benchmarked against human psychologists in figuring out behavioral predictions for a human target audience.

Closer to Home

We also have our own human behavior projects at MIT – and I’ve been interested in compiling a lot more information about what our people are doing, the kinds of results that they’re getting, and the resulting insights.

For example, there are efforts made in this direction by researchers like Jacob Andreas and Athul Paul Jacob and Abhishek Gupta.

“To build AI systems that can collaborate effectively with humans, it helps to have a good model of human behavior to start with,” explained Adam Zewe in April of last year, describing related projects. “But humans tend to behave suboptimally when making decisions. This irrationality, which is especially difficult to model, often boils down to computational constraints. A human can’t spend decades thinking about the ideal solution to a single problem.”

That’s the noise.

Zewe continued:

“Researchers at MIT and the University of Washington developed a way to model the behavior of an agent, whether human or machine, that accounts for the unknown computational constraints that may hamper the agent’s problem-solving abilities. Their model can automatically infer an agent’s computational constraints by seeing just a few traces of their previous actions. The result, an agent’s so-called ‘inference budget,’ can be used to predict that agent’s future behavior.”

So these “inference budgets” can be spent in analyzing either a human or a machine… this is ground-breaking stuff that will probably be prominent in the history books when future writers chronicle the rise of AI as a predictive force.

Also at MIT, there’s the work of full-time AI researcher Rickard Bruel Gabrielsson, who also collaborates with Justin Solomon and others in the MIT community.

In an online presentation, Gabrielsson talks about how all of this works.

Data tools featuring what people say ends up being noisy input – a much more focused control set, these researchers suggest, might be composed of data about how people use their money and their time, or other action events that speak louder than words.

This makes sense to those who understand how AI scans for deep detailed data, and makes in-depth comparisons to drive insights. It’s not intuitive – it’s data-driven. But it works.

As an example, Gabrielsson talked about a project with old movie posters that shows the granular intentions of human users, and other projects involving AI helping to pick out gifts for other people.

A teenager, for example, where Gabrielsson invokes the inscrutable nature of the adolescent, or when he asked a model to help brainstorm a gift for his own wife. He described how the LLM tried to look up what a married woman would want – jewelry, etc. – but ultimately suggested a gift card.

His wife, he said, wondered why he didn’t pick a more personalized gift. So maybe that particular foray did not show off the LLM’s capability well. But in general, he explained, this technology is at our fingertips now.

In the end, he said, it’s all about AI figuring us out in more detailed ways.

“If you want AI to help us and make us better, it needs to know the true story,” he said. “It cannot just understand the filtered fictions we put online – it needs to understand the unfiltered reality that makes us who we are. It also needs to see the everyday heroes who don’t get recognition, how we care for loved ones, and how we do all those boring tasks for them that never (get represented) online. Any intelligence that doesn’t understand that will never truly understand us.”

So what is this study, really? You could call it behavioral AI. It’s the study of human psychology, using predictive tools that rely on technical indicators rather than human intuition.

Human intuition is powerful for a lot of things, but it may not be the best at predictive analytics.

We have to be humble about that, and keep our minds open, as we continue exploring what LLMs can do.