Gearing Up

Is Unreasonable the New Normal?

When I was a student, I sometimes borrowed my dad’s 1985 BMW 320i. It was a great car, but it had one problem: the gas pedal sometimes stuck when floored. Pretty scary experience when the car continues speeding up and you can't do much about it.

Later on, I found out the throttle cable was slightly frayed at the far end, which is what made the pedal lock in place when floored.

The first time it happened, it was scary. I thought I would die—I was completely unprepared for the experience. But afterwards, I knew what to do: I would simply shift to neutral (the Beemer had an automatic transmission, so I couldn’t hit the clutch) and, with the engine revving up, pull over and deal with it. I was in control

The shifting gears of AI

Earlier this week, I sat on a panel exploring the future of AI. One of the questions posed was, 'How rapidly is AI evolving, and what lies ahead?'

Quick on my feet, I likened AI's advancement to a car's transmission system:

1st Gear: 'The Humble Start'

Throughout the last century and in the early years of this one, AI's development resembled a car stuck in first gear. Imagine trying to go faster, but all you hear is revs going up. Early AI solutions were incrementally improved by adding more data and sensors. In fact, we were surprised by the “unreasonable effectiveness of data”. Yet, we soon hit a plateau. For example, AI showed promising results in specialized tasks like natural language processing or predictive analytics in business but didn't make huge leaps beyond that.

2nd Gear: 'The Acceleration Epoch'

In the 21st century, the introduction of deep learning propelled image and speech recognition, while GPUs accelerated AI at an unprecedented rate. Pitting AI systems against other AI systems (Generative Adversarial Networks is the term for it) massively accelerated AI training. We began witnessing 'near-human' performance across various applications. This time we were surprised by the “unreasonable effectiveness of deep learning”.

3rd Gear: 'The Human Parity Era'

And then, just a few years ago, a new approach to machine learning, called “transformers” was proposed in a paper titled “attention is all you need” (“the unreasonable effectiveness of attention” would be just as good). Introducing transformers, faster computing systems, and massive training datasets resulted in even more potent AI. Consider OpenAI's Sam Altman's term 'median human worker' to describe the intended level of capabilities of ChatGPT. In this third gear, we can access AI systems that outperform some humans while lagging behind others. There are plenty of examples, including some that I wrote about, like large language models outperforming MBA students from top business schools. Not to mention certain 'superhuman' performances—like chess, where even the best human players are virtually guaranteed to lose against AI.

4th Gear: 'The Uncharted Frontier'

I am wondering when will we get to a stage where we can’t get any more speed out of the 3rd gear and manage to switch to the next one. What will we find unreasonably effective in this gear? Will the 'fourth gear' consist of AI systems that outpace human expertise across multiple domains? Would the fourth gear bring about the era of "Self-Optimizing Systems"—systems capable of introspection, self-improvement, and, perhaps, novel problem-solving approaches that we haven't yet conceived?

Could we stall? Should we stall?

After my panel response, the moderator threw in a zinger: "Could AI stall, and should we want that to happen?" Maybe the ethos of "move fast and break things" isn't universally applicable. Perhaps, at times, society should make a conscious choice to decelerate to better understand our current advancements rather than blindly forging ahead.

Perhaps the throttle of the AI revolution is slightly frayed as well. And maybe it's scary only the first time. Once we understand how to manage the pace, we can make adjustments as needed and continue our journey into the unknown.