Advanced AI models are forcing a re-evaluation of how we understand truth, reality, and computation. These systems, while powerful, operate on principles that challenge traditional notions of knowledge and information processing. Understanding their mechanisms is key to navigating their impact on human cognition and scientific inquiry.
Two Modes of AI: Statistical vs. Computational
Large Language Models (LLMs) like ChatGPT primarily generate human-like text. They operate on a wide and shallow computational principle. Given a prompt, an LLM predicts the next word. It does this based on statistical patterns. These patterns are learned from a vast dataset, such as a trillion words of text written by humans on the web. This process is akin to continuing a pattern, drawing from what has already been said.
In contrast, deep computational systems, exemplified by Wolfram Alpha and Wolfram Language, aim for deep and broad computation. These systems are built on formal structures, mathematics, and systematic knowledge accumulated by civilization. Their goal is to compute new answers, not merely to echo existing text. They perform arbitrarily deep computations. This allows them to derive facts that may never have been explicitly stated or computed before. The distinction is fundamental: LLMs focus on the statistics of human-produced language. Deep systems leverage formal logic to build complex, reliable knowledge. One is about matching existing patterns. The other is about deriving new truths through structured computation.
The Depth of Computation: Beyond Surface Patterns
Building deep computational systems is labor-intensive. It involves making as much of the world computable as possible. This means creating systems that can reliably answer questions based on expert knowledge. This approach contrasts with LLMs, which forage from existing web content. Historically, human intellectual progress has moved beyond immediate intuition. It developed formal systems like logic, mathematics, and science. These allow for tall towers of reasoning. Implications are worked out step-by-step, leading to conclusions not immediately obvious. Deep computation seeks to automate this process. It builds on formal foundations.
The very nature of computation reveals surprising complexity. Even extremely simple programs, sometimes less than a line of code, can produce really complicated things when run. This phenomenon was discovered in the early 1980s. It suggests that nature itself often operates with simple underlying rules. These rules then generate unexpected complexity.
The Enigma of Computational Irreducibility
A critical concept in understanding computation is computational irreducibility. This means that even if you know the rules governing a system, you cannot predict its outcome without actually running the computation. There is no shortcut to jump ahead and know the result after, say, a million steps. The value of computation often lies precisely in this irreducibility. You had to do the computation to find out the answer.
This principle applies not just to programs but also to the universe itself. At its lowest levels, the universe is a computationally irreducible system. It is constantly evolving through countless steps. We cannot simply deduce its future state. It must unfold.
Finding Predictability: Pockets of Reducibility
Despite the pervasive nature of computational irreducibility, human experience and scientific progress are possible. This is because we find pockets of reducibility. These are areas where we can locally jump ahead and make predictions or simplify complex processes. Science, inventions, and much of human understanding involve identifying and leveraging these predictable patches. They exist within an otherwise irreducible computational universe.
Life as we experience it, with its consistent patterns and predictable phenomena, relies on our ability to perceive and interact with these reducible slices. We do not need to track every molecule in the air to know that the air will still be there in a few seconds. This simplified understanding is a reducible fact. It exists about an underlying, complex, and irreducible process.
Symbolic Representation and Human Cognition
The bridge between the vast computational universe and human understanding is symbolic representation. Our minds take in detailed information. This includes lots of pixels in a visual scene. They then compress it into symbolic forms, such as “there’s a chair in this place.” This allows us to conceptualize and remember.
Human language is one form of symbolic representation. It uses words like “chairs.” However, human language is not inherently precise enough for direct computation. To make it computable, it must be formalized. This led to the development of symbolic programming, an idea conceived around 1979. It involves representing everything in the world as structured symbolic expressions. These are like functions with arguments that represent structures rather than just immediate values. This approach has proven to be a good match for how humans conceptualize higher-level ideas. It enables the creation of systems that can connect human thought with computational possibilities.
Our nature as computationally bounded observers means we cannot process all the underlying irreducibility. Instead, we extract a symbolic essence from the world’s details. This act of compression and focus on reducible aspects is fundamental to our perception. It even shapes the laws of physics we observe.
