The AI Seesaw

Is office work starting to resemble RollerCoaster Tycoon?

Is Excel finally dead?

Let’s investigate.

Is this different or are we just on the AI hype seesaw? Yes.

With generative AI, it feels like we’re on a perpetual seesaw that oscillates between “AI is a scam” and “AI super-intelligence is inevitable.” Currently, it feels like AI has passed a tipping point on the path toward knowledge worker replacement. Some, like Jack Dorsey, are even laying off 40% of their company in anticipation of this playing out. Let’s begin by investigating what changed.

Do the LLM models still just predict the next word?

Yes and no. The LLMs powering today’s agents and chatbots remain auto-regressive next-word predictors that AI luminary Yann LeCun declared doomed 3 years ago, but training has evolved a lot since GPT 3.5. Now, instead of just being trained on general text (called pre-training), they are being trained on how to solve problems using post-training reinforcement learning . Doomed or not, it is becoming abundantly clear that advances in training techniques and compute are making next-word prediction models quite powerful. But how?

The magic lies in the animated dots below. It truly is like magic, because while researchers understand some circuits and features, the full computation inside large language models remains mostly a mystery.

Even though the model’s output is simple, one token at a time, how it decides which token comes next can be incredibly complex.

In reinforcement learning with verifiable rewards (RLVR), the model isn’t necessarily shown how to solve a problem. Instead, it tries the problem itself, often many times, and an automated verifier checks whether it got the answer right.

PROMPT
“If 3x = 12, what is x?”
MODEL ATTEMPT #1
“To solve for x, divide both sides by 3:
x = 12 ÷ 3
x = 4
Answer: 4”
VERIFIER → CORRECT → REWARD: 1
MODEL ATTEMPT #2
“3x = 12
x = 12 - 3
x = 9
Answer: 9”
VERIFIER → INCORRECT → REWARD: 0

The model isn’t directly told, “use this reasoning strategy.” It generates its own attempts and receives a reward based on whether the result can be verified. Across huge numbers of problems and attempts, reinforcement learning adjusts the model’s weights so that the token sequences associated with successful solutions become more likely.

But wait, there’s more

There have also been major advancements in agent “harness engineering”. Harness engineering means designing the system around the model (including agent tools and skills) that helps it reliably complete real work. In the software engineering world, people are starting to crack the code on the harness engineering needed to orchestrate agents to develop and maintain production-ready software.

Now agents and tools, such as Codex, Claude Co-Work, Databricks Genie, and Cursor are moving beyond writing code to performing many of the same digital tasks that knowledge workers perform every day (ex. sales presentations, accounting spreadsheets, urgent care schedules, manufacturing diagrams).

Would this dream free us to do “higher-level thinking,” or render us virtual rail workers doing the tedious work of building tracks for the robots who will replace us once we finish constructing their theme parks? Stay tuned!

Will the seesaw tip back again?

As the seesaw oscillates, the arrow of progress continues upward, even as our goalposts keep shifting alongside it. I’ve witnessed agents like Databricks Genie compress weeks of analytical work into minutes, reshaping tasks that once depended on highly manual SQL, Tableau and Excel workflows.

But if history is any guide, the see will saw again. Soon, we’ll likely be reading another round of articles declaring that AI isn’t living up to the hype for some reason or another. We’ve been here before.

Further, agents are expensive and energy-hungry. AI backlash from workers and consumers is growing. Knowledge work is messy and subjective in ways coding is not. And the knowledge that makes organizations work often lives in people’s heads, not in documents waiting to be ingested by an agent.