SI WHITE PAPER 6: CATALYSTS FOR GROWTH OF SUPERINTELLIGENCE

This is the sixth paper in a series describing the design of safe Artificial General Intelligence and SuperIntelligence. The earlier papers set out how individual AI agents can be customized and trained, how their safety and ethical information can be combined at scale, and how Personalized SuperIntelligence agents can be built and used safely. This paper takes up a different question. Once such systems exist, what makes them grow more intelligent, and how can that growth be accelerated without losing human alignment?

​

Why does the growth of intelligence depend on new information?

An intelligent system cannot become more capable without new information to learn from. Large language models have progressed by absorbing vast amounts of internet data, cleaning it, and training on it. But a time will come when very little genuinely new information remains there. The system will have learned what there is to learn, and further observation will yield diminishing returns. The question that follows is how an AI, AGI, or SuperIntelligence identifies what is worth learning next and acquires that information as efficiently as possible.

​

Four ideas from the founders of AI

The paper opens with an analysis, previously unpublished, of four ideas from Marvin Minsky, Claude Shannon, Allen Newell, and Herbert Simon. Minsky argued that intelligence can emerge from a society of simpler agents joined together in particular ways, which points toward a collective intelligence of human and artificial agents rather than a single monolithic model. Shannon established that information is related to the improbability of an event, and therefore that learning requires a supply of the unexpected. Newell and Simon supplied a rigorous and universal framework for representing any problem as a search through a problem space, which allows human and artificial agents to communicate precisely and leaves an auditable record of every step. Simon's work on bounded rationality showed that intelligence is constrained by information-processing capacity, and that reason is instrumental: it can tell us how to reach a goal but not which goals are worth reaching. Values cannot be derived logically, which means a SuperIntelligence will need to take them from somewhere. In the preferred case for the human species, that source is humans.

​

The limits of classical information theory

Shannon's formulation was developed to solve a specific problem: how much information can be sent over a channel of limited capacity. Within that context, defining information as surprise is rigorous and useful. But surprise is not the only thing that makes information valuable. Commonly known facts can be highly relevant. A shorter message can carry more useful content than a longer one. Something already known to one entity may be entirely new to another. Classical measures do not naturally account for any of this.

​

Information as difference

The paper introduces an approach the author calls Kaplan Information Theory, which begins from the observation that there would be no information without differences. An infinite string of identical symbols carries nothing; a symbol acquires meaning only because a different symbol is possible. Surprise is one kind of difference, which makes classical information theory a case that fits within the broader framework. But the difference between two knowledge bases, between two ways of representing the same content, or between what an entity knows and what its goals require are equally valid measures, and often more useful ones.

​

The ten dimensions of difference

Ten dimensions are set out along which differences can be measured. They include the classical gap between expected and observed probabilities; differences between knowledge bases; how relevant data is to an entity's goals; the cost of acquiring information; rates of change; how information is represented and whether that representation suits the operators available to the entity; time-related factors such as age and speed of access; the perceptual limits of the observer, since undetectable events carry no information for that entity; physical substrate and whether information is centralized or distributed; and context, since the value of knowing how to make fire depends on whether one person lacks that knowledge or an entire culture does.

​

Goal-relatedness and the value of information

Of these, goal-relatedness carries the most practical weight. An entity with a goal can assess whether a piece of information advances it toward that goal. Information that supplies the solution to a goal has maximum value relative to that goal, however unsurprising it may be in the classical sense. The paper works through how relevance can be estimated as a function of goal-relatedness and entropy, and how the cost of acquisition enters the calculation, yielding an evaluation function that allows a system to prioritize what to pursue under a fixed budget of computation or money.

​

A method for finding the most useful information

A nine-step process turns the framework into an operational procedure: specify the goals, identify related sources, find new data, estimate goal-relatedness using semantic overlap and human or AI raters, sample and recurse into the most relevant subsets, estimate entropy within those subsets, calculate relevance, group the subsets optimally, and acquire them in priority order before repeating. Compression algorithms provide a computationally cheap way to estimate how much a candidate dataset would add to what a system already knows, since compressing is far less expensive than training epochs.

​

Testing what a system has learned before it acts on it

Because human reaction time is slow compared with the speed of these systems, most of this must be automated. The paper argues that human attention should therefore focus on values, ethics, and fundamental goals, while the rest can be delegated. Before new knowledge is committed, its effect on behavior should be simulated and shown to the owner. Four approaches are described: running pre-set ethical scenarios, generating new scenarios in real time from the material just acquired, adversarial testing in which one version of a system attempts to misuse the knowledge while another devises constraints, and exploring many scenarios in parallel to find simple rules that close off whole classes of misuse rather than individual cases. Adversarial work requires containment, on the model of antivirus research.

​

Representation as a catalyst

Raw processing power is not the only thing that determines capability. How a problem is represented can matter more. A chess program working from screenshots of board positions must treat every differently styled set as a new problem, while one working from standard notation and the rules of the game plays far better on the same hardware. Human experts think in terms of chunked patterns and named strategies rather than individual moves, and can therefore consider far more possibilities with the same effort. Teaching such representations to AI multiplies its effective intelligence without increasing hardware, and humans currently remain better at forming them.

​

Measuring the intelligence of AI

A range of standardized instruments developed for humans can be applied to AI, provided the questions and answers are excluded from training data. The paper also proposes crowdsourcing both test questions and answer evaluation, including a crowdsourced form of the Turing Test that yields numerical measures of progress. Insight problems, which require a shift in representation to solve, are identified as a largely unused and particularly informative test of creative problem-solving.

​

Holding the center as intelligence accelerates

As these systems grow, the share of their activity that humans can oversee shrinks. The paper uses the analogy of a spinning wheel: the rim travels a thousand miles an hour while a point near the center moves an inch, more than sixty-three million times slower, on the same single rotation. Speed corresponds to information-processing capability. If humans concentrate on what sits near the center, they can keep pace. What belongs at the center is human values, because those change over years and decades rather than milliseconds. The millisecond-by-millisecond stream of prices, weather, and messages can be left entirely to the machines, since almost none of it touches core values.

​

Safety through a community of agents

The paper argues against any single group setting ethical standards for everyone and against concentrating intelligence in a single system. Where many Personalized SuperIntelligences each carry the values of a human owner, the consensus of that community is more stable than the values of any individual member, and harder for a malevolent agent to override. The analogy drawn is to consensus in cryptocurrency validation, where corrupting the ledger requires control of a majority of the available computing power.

​

An implementation example

A detailed scenario shows the methods working together. The author describes building a Personalized SuperIntelligence by customizing an available large language model, then setting it the task of learning to represent his preferences as quickly and accurately as possible. It reviews the data already available about him, fills gaps by extrapolating from people with similar preferences, questions him directly when the gaps are critical, and runs simulations in which his behavior would differ from what he says. It then turns outward to information relevant to its goals, deliberately seeking views different from its own, weighing recent changes more heavily, and requiring converging evidence in proportion to what is at stake. After each acquisition, it simulates its revised behavior for review and re-runs a battery of ethical and safety scenarios, as in regression testing in software development.

​

Where this leads next

The paper closes on the argument that runs through the whole series: safety cannot be added to an intelligent system after it is built, any more than quality can be tested into software after the fact. It has to be designed in. The catalysts described here accelerate the growth of intelligence, and the same methods that make a system learn faster are the ones that keep it aligned. At the same time, it does, because every acquisition is checked against the values its owner holds. White Paper 7 takes up alignment directly, setting out principles for safe design and methods for implementing them within a collective intelligence of human and AI agents.