The AI Seesaw

Is office work starting to resemble RollerCoaster Tycoon?

Is Excel finally dead?

Let’s investigate.

Is this different or are we just on the AI hype seesaw? Yes.

With generative AI, it feels like we’re on a perpetual seesaw that oscillates between “AI is a scam” and “AI super-intelligence is inevitable.” Currently, it feels like AI has passed a tipping point on the path toward knowledge worker replacement. Some, like Jack Dorsey, are even laying off 40% of their company in anticipation of this playing out (or so he claims that’s why…). Let’s begin by investigating what changed.

Do the LLM models still just predict the next word?

The LLMs powering today’s agents and chatbots remain the same auto-regressive next-word predictors that AI luminary Yann LeCun declared doomed 3 years ago (though they increasingly rely on advanced post-training alignment and tool-use training.) Doomed or not, it is becoming abundantly clear that advances in training techniques and compute are making next-word prediction models quite powerful. But how?

The magic lies in the animated dots below. It truly is like magic, because while researchers understand some circuits and features, the full computation inside large language models remains mostly a mystery.

Even though the model’s output is simple — a single word — how it arrives at that word can be incredibly complex. Hidden layers contain billions of parameters used to determine that next word. If we feed the word predictor thousands of training examples like:

”document”: “A tenant stopped paying rent after discovering severe mold in the apartment. The landlord sued for unpaid rent. The tenant argued the apartment was uninhabitable and that the landlord failed to fix the issue after repeated notices.”,
“think”: “Identify the legal issue (habitability). Determine the relevant rule (landlords must maintain livable conditions). Apply the rule to the facts.”,
“final”: “The court ruled in favor of the tenant, finding that the landlord breached the warranty of habitability by failing to repair the mold problem.”

The model may begin to internalize patterns of legal reasoning within the hidden layers of its neural network, enabling it to activate those patterns when users ask it to analyze documents or explain legal disputes. The emergent capabilities of LLMs continue to surprise, but that is not the main story behind why AI suddenly appears to pose an “intelligence crisis”.

If that’s not it, then what is it?

This is largely a story about advancements in agent development and “harness engineering” — designing the environments, data, scaffolding, and feedback loops of agents. In the software engineering world, people are starting to crack the code on the harness engineering needed to orchestrate agents to develop and maintain production-ready software.

Now agents and tools — such as Codex, Claude Co-Work, Databricks Assistant, Perplexity Computer, Google Workspace CLI, and custom-built systems — are moving beyond writing code to performing many of the same digital tasks that knowledge workers perform every day (ex. sales presentations, accounting spreadsheets, urgent care schedules, manufacturing diagrams).

The assumption is that it is only a matter of time before the code is cracked on orchestrating agents to handle all “knowledge work” tasks.

Would this dream free us to do “higher-level thinking,” or render us virtual rail workers doing the tedious work of building tracks for the robots who will replace us once we finish constructing their theme parks? Stay tuned!

Will the seesaw tip back again?

As the seesaw oscillates, the arrow of progress continues upward, even as our goalposts keep shifting alongside it. Something real is changing. I’ve witnessed agents like Databricks Genie compress weeks of analytical work into minutes, reshaping tasks that once depended on highly manual SQL, Tableau and Excel workflows.

But if history is any guide, the see will saw again. Soon, we’ll likely be reading another round of articles declaring that AI isn’t living up to the hype for some reason or another. We’ve been here before.

Further, agents are expensive and energy-hungry. AI backlash from workers and consumers is growing. Knowledge work is messy and subjective in ways coding is not. And the knowledge that makes organizations work often lives in people’s heads, not in documents waiting to be ingested by an agent.