A Simple Idea behind World Models

we've never experienced reality, it was just what our brains thought would happen

Have you ever slipped on a wet surface (as a clumsy person, I have) and found your hands shooting out to catch you before you hit the ground, now, it’s not like as your feet went into the air, your body rotated and your hands happened to be there, if you would have noticed, even your palms started facing to the ground, that’s because something faster was at work, a REFLEX. Even before your mind could consciously decide, it happened, all on its own, before your brain’s permission.

This is the same reason a baseball batter can strike the ball because the usual pitching distance is about 60 feet (18.something meters), and the ball often travels at an average speed of 90 miles/hour, which means it reaches the batter in about 450 milliseconds, which is not even a complete second to react to. And you know, for your eyes to see, tell your brain what it saw and do the action via muscle movement, it takes around 250 ms, so if you keep waiting for your brain to kick in your muscles and do some action, you’ll never even hit the ball and that’s why striking in baseball is not easy, still professionals do it seamlessly.

This isn’t magic or something superhuman; it’s been happening your entire life. Your brain was never playing the action-reaction game; something much more elegant was at work

Prediction Engine

Have you ever noticed how a baby falls differently from adults? That tiny body just surrenders to gravity, this happens because as we grow, our brain continuously take notes about it so everytime you’ve stumbled, bumped into something or even better, if you’ve burnt your finger after touching something and when you go to something similar environment, you get that chill, that same feeling you felt last time, even though you haven’t touched anything it, that feeling is quite clear.

As we grow older, our brain sketches a mental model of the world from all these experiences of how things fall, how surfaces feel, how your body responds to imbalance, how your body reacts to those impulses and that is why it doesn’t wait for an event to happen, from whatever you are doing( your motor action) it knows what’s coming from all the experiences and goes into action beforehand. Crazy right?

Dual Realities

At any moment, your brain maintains 2 realities: one that is actually happening, and another that it expects should be happening based on everything you’ve experienced. The surprising part? Prediction is trusted more than what’s real.

Everything goes smoothly when your brain says, This will happen, so do this and that actually happens, and experience that surprised, “what just happened moment” when the prediction and reality don’t quite match.

That is exactly why you sometimes feel your phone vibrating when it hasn’t, or think someone just called your name when they didn’t. These false alarms happen when predictions and reality don’t match. But it’s better to be extra cautious than miss something important, isn’t it?

Because if this prediction wasn’t in place, you’ll literally see the lag when you move your eyes, this future prediction smoothens the movement of your eyes, this is exactly why you close your eyes when you see some dust coming your way and want to prevent it from getting into your eyes.

In Neuro-Science, this is called “Controlled Hallucination,” and you never directly perceive reality; it’s your brain’s best guess about the world, and it gets better as you do certain things again and again. The brain predicts things better. (So, if you follow F1, you know that Max Verstappen’s racing brain’s prediction is finest.)

So remember, your brain has a perception of the world around you, which it refines every day from what you do; that is why it makes you better at something if you keep doing it repeatedly.

Welcome, World Models

The concept of world models emerged from recognizing this gap. If the brain builds internal models that simulate how the world works, perhaps we should build artificial systems that do the same. A world model in AI is a learned system that predicts future states of an environment given current states and actions. Just as your brain predicts that a slipping foot leads to a fall, an artificial world model might predict that turning the steering wheel changes a car's trajectory.

This idea gave birth to what we now call World Models in AI—a bit of a buzzword these days, with many saying AGI will need them. Although the term was coined back in the 1950s, it gained serious attention when this landmark paper was released in 2018. (Nice paper if you are into Machine Learning).

This is significantly useful in fields like robotics, autonomous vehicles (again robot on wheels, but slightly different conditions), and is actively used to train such systems.

Now, replicating a human brain is not as easy as it sounds, but the idea, the intuition, is quite promising from my perspective, and many brilliant researchers are working on it as you read it. I look forward to where this takes us. But

That’s a wrap for today, and before I say goodbye for today, here’s a quote I’ve been pondering,

“Books are the nearest thing to a time machine humans have ever made."

Please don’t forget to share it with your friends, family, and strangers.

Have a Great Day 💖