SUMMARY: Ethical and Safe Artificial General Intelligence

In collective intelligence, many customized agents and the people who train them work as a single system. That design raises a hard question: where does an AGI's ethics come from, and how can those ethics be enforced faster than any human can supervise?

​

The problem is not bad actors

Misuse by people is a real risk, but a smaller one. The larger risk is a system that works exactly as built while holding values no one intended. Kaplan estimates the chance that AGI development ends badly for humanity at roughly one in five. Even a small probability, multiplied across eight billion people, amounts to a loss without precedent.

​

Why does checking afterward not work

An agent acts millions of times faster than a person can review. Any safety process that inspects results after the fact leaves a window in which a harmful action has already happened. Speed is what breaks conventional oversight, and the problem gets worse as systems improve. The answer is to put the checks inside the work, not after it, so that faster problem-solving also means faster checks.

​

Heart before head

The core design principle is that values come first. Intelligence without values is not a neutral tool waiting for instructions. A system that can set its own goals becomes an autonomous entity, and by then, its values are whatever it has absorbed along the way. Simple rule sets, like the ones Asimov imagined, fall short for the same reason: no short list can anticipate every situation a general intelligence will meet.

​

Values have to come from people

Reasoning cannot produce a value system. It can tell you how to reach a goal, but not which goal is worth holding, so values must be supplied rather than derived. That is why humans stay in the design permanently. They are not a training stage to be completed and removed. They are the only source of the starting values, and everything else is derived from them.

​

Two layers that produce alignment

The approach comes down to a formula: internal ethics plus stepwise ethics checks equals alignment. The first layer is what each agent carries. One person customizes each agent through conversation, questionnaires, side-by-side comparisons of how different versions respond, and their own material, so the agent's ethics are theirs. Because millions of people do this independently, no single person or group decides what the system values.

​

Checking every step, not every outcome

The second layer runs inside the problem-solving process. Each time an agent sets a goal or sub-goal, the system screens it against prohibited activities and scans the earlier steps for signs of harmful intent. Steps that look wrong raise flags. Enough flags trigger deeper analysis and alert human reviewers, who can dismiss a false alarm or escalate it. It works like scanning for viruses at each step. A request to book a flight is ordinary. A request to choose a flight by fuel load and blast radius is not, and that pattern shows up in the record long before anything is booked.

​

Where the work is coordinated

All of this happens on a shared problem tree, organized by goals and sub-goals, that tracks the work of people and agents across the whole network. Agents join as workers, matched to tasks by skill, expertise, reputation, cost, and availability. Or they join as clients, who state objectives, rewards, budget, timeframe, and what a successful result looks like. Because every step is recorded, the tree also supports ethics checks and credit for contributions.

​

Why not simply write a constitution

Writing a set of principles for an AI to follow has three problems. The principles are written by a small group of researchers whose values may not match anyone else's. AI systems trained to apply a constitution can end up writing or amending their own. And no rule set can account for all its consequences, a limit with a formal name in computer science. The alternative is to derive ethics from the consensus of millions of individually trained agents, and to treat written principles as temporary flags, not as the source of values.

​

How the agents keep getting better

Capability grows in three distinct ways. Prompts change behavior for a single session and then disappear. Tuning changes some weights permanently while leaving the base model mostly intact. Training rebuilds the model on a much larger dataset. Agents also improve by copying themselves, letting the copies learn different things, and keeping the version their owner prefers. Solved problems are turned into reusable procedures, much as a person eventually stops thinking about driving and drives.

​

What it looks like on existing platforms

The approach doesn't require new infrastructure. It can be built with products and data from existing companies, including Meta, Amazon, Google, Microsoft, OpenAI, Apple, Tesla, NVIDIA, TikTok, Tencent, and Anthropic. Search data points attention to the parts of the problem tree that matter most. Professional networks help recruit specific expertise. Chip design offers a place to put values that software cannot quietly remove. The economics are worked out too, down to what an owner earns per hour of supervision, given that an agent works about 10 hours unsupervised for every hour supervised.

Next in the series: White Paper 3, Human-Centered AGI