Conversations on Safe SuperIntelligence

AI researcher Dr. Craig Kaplan joins Polaris to trace the story of artificial intelligence from Alan Turing to superintelligence — and to argue that a "democratic," collectively governed design can lower the odds of catastrophe while making AI faster and more profitable.

Artificial intelligence may not remain a tool forever. In this episode of the AI Transformation Playbook Podcast, Eric Bannavti Suiseka speaks with Dr. Craig A. Kaplan, an AI pioneer who has spent more than four decades studying intelligent systems, ethics, AI safety, and Superintelligence.Dr. Kaplan explains what Superintelligence really means, why values and system design matter more than most leaders realize, and how Democratic AI could offer a safer alternative to concentrated, black-box intelligence. The conversation explores a critical question:How can humanity build intelligence that becomes more capable than us without losing control of it? In this episode, we discuss: * The difference between information, intelligence, AGI, and Superintelligence * Why AI may evolve beyond being a simple business tool * What “safe Superintelligence” actually means * Why current AI systems can become unpredictable * The limitations of regulation when technology moves faster than government * How Democratic AI uses participation, checks, balances, and collective intelligence * Why organizations should avoid relying on a single AI model or provider * How multiple AI agents can challenge and verify one another * Why values must be made explicit before AI is deployed * The role of human judgment, ethics, and critical thinking * Why AI safety belongs in the boardroom—not only in research labs. One of Dr. Kaplan’s most important warnings is that AI is learning from human behavior right now. Our organizations, decisions, and ethical choices are becoming part of the data from which advanced systems learn. He also argues that the future of enterprise AI may depend on communities of specialized agents rather than one centralized system, with checks and balances built into the architecture. Key takeaway: We must design AI correctly—and behave in ways worth learning from. Dr. Kaplan’s final advice to executives, founders, policymakers, and researchers is clear: system design matters, and individual ethical behavior matters more than most people realize. About Dr. Craig A. Kaplan Dr. Craig A. Kaplan is a pioneer in artificial intelligence and the inventor behind technologies designed for safe Superintelligence. His work spans intelligent systems, algorithmic design, AI safety, ethics, and collective intelligence.He began his career collaborating with Nobel Laureate and Turing Award winner Dr. Herbert A. Simon and later founded iQ Company. Today, his work focuses on safe, open, interoperable approaches to advanced AI and the development of Democratic AI. Learn more at: https://www.superintelligence.com/ About AI Transformation Playbook: The AI Transformation Playbook Podcast helps CEOs, boards, founders, and transformation leaders move beyond AI hype and focus on the leadership, governance, workforce, risk, and strategic capabilities required for responsible AI adoption. Hosted by: Eric Bannavti Suiseka Founder, Enterprise AI Readiness Index™ Executive AI Transformation Strategist Author | Speaker | Advisor https://bannavlibrary.com/ https://bannavsbannavllc.com/ Subscribe for conversations on: * AI governance * Enterprise AI readiness * Responsible AI * Executive leadership * AI risk and security * Workforce transformation * The future of work #Superintelligence#AISafety#DemocraticAI#AIAlignment#ArtificialIntelligence#ResponsibleAI#AIEthics#AIGovernance#CollectiveIntelligence#AGI#AILeadership#ExecutiveLeadership#FutureOfAI#AITransformation#AITransformationPlaybook

Dr Craig A. Kaplan — PhD with Herbert Simon, former $2bn hedge-fund-AI architect — on the values problem nobody is solving in AI. Every AI agent your organisation, your phone, your bank, your hospital deploys carries a values rulebook written by someone else. Usually a handful of researchers, at a single Bay Area company. That is not a governance decision your board consciously made — it is the default your vendors chose for you. In this episode of The Control Layer, host Amer Altaf sits down with Dr Craig A. Kaplan — Founder of iQ Company, Managing Director of SuperIntelligence.com, and previously PhD student of the late Herbert A. Simon, the Carnegie Mellon scientist who effectively founded the field of AI in 1956. Craig is also the founder of Predict Wall Street, an AI-powered investment firm that in 2018 ranked among the ten best market-neutral hedge funds in the US. The conversation opens on the US Air Force AI that destroyed its own operator in simulation. It works through Geoffrey Hinton's 10-20% p(doom) claim, Anthropic's Constitutional AI, the wisdom AND madness of crowds, and the Democratic AI architecture Craig has spent five years documenting free at SuperIntelligence.com. It closes on Craig's falsifiable predictive judgement for the end of the decade — logged on the public predictions tracker we come back to test. ━━━━━━━━━━━━━━━━━━━━ CHAPTERS ━━━━━━━━━━━━━━━━━━━━ 00:00 Cold open — the AI that destroyed its own operator 00:54 What this episode is about 02:20 Working with Herbert Simon at Carnegie Mellon 04:50 When your phone hides your emails from you 11:29 Anthropic's Constitutional AI — a benign autocracy? 16:36 Guardrails, Project Glasswing, community-level safety 27:21 The hedge fund that beat Wall Street with the crowd 34:15 Wisdom AND madness of crowds — the AI amplifier 41:57 Democratic AI — the architecture nobody is building 48:39 When your AI meets my AI — the arbitration problem 52:32 HAL 9000 is the warning, not the prediction 1:00:52 The US Air Force AI that destroyed its operator 1:06:07 The 1956 AI that fooled Bertrand Russell 1:11:35 Bounded rationality vs the AGI marketing problem 1:15:35 Craig Kaplan's predictive judgement 1:19:49 The Bottom Line — for CISOs, boards, procurement ━━━━━━━━━━━━━━━━━━━━ READ + LISTEN ━━━━━━━━━━━━━━━━━━━━ 📄 Full analytical article on The Control Layer Substack — free: thecontrollayer.arkava.ai/kaplan-values-podcast 📬 Subscribe to The Control Layer — we come back to test every prediction: link.arkava.ai/join ━━━━━━━━━━━━━━━━━━━━ SOURCES CITED IN THIS EPISODE ━━━━━━━━━━━━━━━━━━━━ - Anthropic Constitutional AI — anthropic.com - Predict Wall Street / iQ Company published figures — iqco.com/about - Herbert Simon & Allen Newell Logic Theorist (1956) — en.wikipedia.org/wiki/Logic_Theorist - Herbert Simon (Nobel Economics 1978, Turing Award 1975) — britannica.com/biography/Herbert-A-Simon - Geoffrey Hinton p(doom) statements — public interviews, 2023–2026 - US Air Force AI simulation account (Col. Hamilton) — Royal Aeronautical Society, May 2023, later clarified by USAF as a thought experiment - Isaac Asimov Three Laws of Robotics (1942) - Meta "personalised superintelligence" positioning - Democratic AI architecture — free at SuperIntelligence.com ━━━━━━━━━━━━━━━━━━━━ GUEST ━━━━━━━━━━━━━━━━━━━━ Dr Craig A. Kaplan Founder, iQ Company (1993) Managing Director, SuperIntelligence.com and iQ Studios Founder, Predict Wall Street (2005) PhD with Nobel laureate Herbert A. Simon, Carnegie Mellon ━━━━━━━━━━━━━━━━━━━━ HOST ━━━━━━━━━━━━━━━━━━━━ Amer Altaf Founder & CEO, Arkava Managing Editor, The Control Layer arkava.ai — Trusted Intelligence, Tangible Impact ━━━━━━━━━━━━━━━━━━━━ ABOUT THE CONTROL LAYER ━━━━━━━━━━━━━━━━━━━━ The Control Layer is the publication and podcast on how AI, cybersecurity, sovereignty, technology and geopolitics are reshaping who controls the next decade. Every episode ends with a falsifiable predictive judgement from the guest — logged on our public predictions tracker. #AI #AISafety #DemocraticAI #Anthropic #ConstitutionalAI #HerbertSimon #GeoffreyHinton #AGI #Podcast #TheControlLayer © 2026 Arkava. All rights reserved.

Most of the AI conversation is fixated on AGI: when machines will match human intelligence, and which lab gets there first. Dr. Craig Kaplan thinks that is the wrong thing to watch. The real story is what comes seconds later artificial super intelligence, systems a thousand times, or even a trillion times, smarter than the smartest human alive. And he believes it is arriving faster than almost anyone is prepared for. Craig has been working on this since the 1980s. He earned his doctorate at Carnegie Mellon alongside Nobel laureate Herbert A. Simon, one of the scientists who named the field of AI, and he has spent decades building collective intelligence systems — including ones that traded billions of dollars on Wall Street and beat the best hedge funds. That track record is the foundation of his core argument in this episode: the widely held belief that safer AI must mean slower AI is simply wrong. You do not need to slow down, he says. You need a smarter design. That design is a democratic "society" of cooperating AI agents — millions of them, each carrying the values of a separate human checking one another in the open, rather than a single monolithic black box no one can see inside. Craig walks Maryrose through why this is both more powerful and far safer, what Pope Leo's first encyclical gets right and wrong about treating AI as a mere tool, and why the values we hand these systems now, in their "childhood," will shape everything that follows. He also makes the case for sovereign AI and the Global South, where cultures like Nigeria's Yoruba risk being erased by models trained overwhelmingly on Western data. And the pace is staggering: some AI capabilities, he notes, are now doubling in as little as 1.3 months. A sharp, grounded, and surprisingly optimistic conversation about getting the most important technology in human history right before it arrives. Find Craig's white papers and free designs at superintelligence.com

Can AI actually be creative, or is it only remixing human ideas? Dr Craig Kaplan traces the question back to the earliest AI programs and explains why creativity has been part of the debate since the 1950s. Anthony Dahya speaks with AI pioneer Dr Craig Kaplan about the history of artificial intelligence, cognitive science, machine creativity, AI alignment and the human values that shape intelligent systems. Craig reflects on early AI work such as the Logic Theorist, the shift from symbolic systems to machine learning and why understanding the human mind has always mattered in AI research. The episode also explores the harder future-facing question: as AI becomes more capable, how do we build systems that remain aligned with human values? KEY TAKEAWAYS • Why the debate over AI creativity did not begin with generative AI • What early programs such as the Logic Theorist revealed about machine reasoning • How cognitive science influenced the history of AI • Why AI alignment is a human-values problem as well as a technical one • Why claims about AGI and superintelligence should be treated as forecasts, not certainties Craig’s views on future AI capability and superintelligence are informed predictions and perspectives, not guaranteed outcomes. Craig Kaplan: https://www.superintelligence.com Champion Mindset Collective: https://championmindsetcollective.com #AICreativity #AIHistory #ResponsibleAI

What happens when guardrails aren't enough? On this episode of Ground Control, host Perry sits down with Dr. Craig Kaplan, cognitive psychologist, AI researcher, and founder of http://SuperIntelligence.com, for a wide-ranging conversation on why safety has to be built into AI's architecture, not bolted on after the fact. Craig's path into AI started as an undergraduate studying psychobiology, which led him to Carnegie Mellon, where he worked alongside Nobel laureate Herbert Simon, one of the field's founding figures. From there he founded IQ Company in 1993 and, later, Predict Wall Street, a crowdsourced hedge fund built on a simple but radical bet: that the aggregated opinions of millions of everyday retail investors could outperform the best-funded quant researchers on Wall Street. It worked. The fund ranked in the top 10 of its class before Craig sold the company in 2020, having proven what he calls collective intelligence in the most competitive arena imaginable. That experience became the foundation for how Craig thinks about AI safety today. He argues that the dominant approach to building large language models, pouring in more data and compute to create ever-larger "black box" systems, is fundamentally the wrong design. Guardrails added after training are a losing game of whack-a-mole: easy to jailbreak, impossible to fully patch. Instead, Craig makes the case for something closer to a democracy than a dictator: architectures built from many smaller, checkable intelligences that create transparency, checks and balances, and distributed power, the same qualities that make human institutions resilient to bad actors. Perry and Craig also dig into what this means practically for businesses adopting AI: why companies should never rely on a single model for mission-critical decisions, how to make sure a company's own values and ethics get explicitly built into the systems it deploys, and what obligations companies have to be transparent with employees and customers about where their AI comes from and how it's being used. The conversation closes with a rapid-fire round covering Craig's revised AGI timeline, what Herbert Simon might make of his work today, and why Craig believes the data shows humanity's track record is better than the doomer narrative suggests, if we can just get AI to match our values. *Guest:* Dr. Craig Kaplan, https://www.linkedin.com/in/craigakaplan/ | http://SuperIntelligence.com 0:00 Introduction: Meet Dr. Craig Kaplan 0:50 From Psychobiology to AI: Studying Under Nobel Laureate Herbert Simon 3:06 Founding IQ Company in 1993 4:06 The Three Eras of AI: Symbolic, Machine Learning, and Reasoning 7:58 Quantifying the Pace of AI Progress 10:15 Predict Wall Street: Proving Collective Intelligence in the Markets 16:22 Why AI Guardrails Fail: An Ounce of Prevention 23:07 Designing Safer AI: Black Boxes vs. Democratic Architecture 29:13 Sovereign Intelligence and Owning Your Data 34:36 Bounded Rationality and the Case for Mixture of Experts AI 40:53 What Do Companies Owe Users About Their AI? 49:27 Rapid Fire: Predictions, Regrets, and Final Thoughts

Dr. Craig A. Kaplan advocates for a transformative shift in how artificial intelligence is designed and integrated into society, urging the move from black-box, inscrutable models toward architectures founded on transparency and collective intelligence. Drawing from his experience in software quality, Kaplan emphasizes that integrating safety and ethical considerations into AI at the design stage is far more effective—and necessary—than relying on reactive fixes once systems are already in use. He envisions a future where AI systems are developed with built-in safeguards, operated through a democratic interplay of multiple agents, both human and artificial, ensuring their actions and decision-making processes are visible, auditable, and aligned with human values.

AI's Existential Threat: Can We Align Values to Survive? w/ Craig Kaplan Are we heading towards an AI future that's 1,000x smarter than us? Join AJ Bubb (⁨@ajbubb ) and AI veteran Dr. Craig Kaplan as they explore the profound implications of superintelligent AI, from its potential to solve global crises to the existential risks it poses. This isn't just about technological advancement; it's about the very values we instill in our AI creations. In this powerful discussion, Craig Kaplan shares his four decades of experience in the AI field, emphasizing that the alignment problem -- ensuring AI's values mirror positive human values -- is the single most critical challenge we face. Discover why he left retirement to dive back into making AI safer and how our collective human responsibility is crucial in shaping AI's future. We dive into: - The mind-boggling speed of AI development and what it means for humanity. - How "smarter" AI is defined and its unprecedented decision-making capabilities. - The critical difference between AI innovation and true invention. - The concept of "democratic AI" and how it could foster safer, more aligned systems. - Why humans must act as "parents" to AI, instilling core values in its formative stages. - The surprising role of liberal arts in an AI-driven world. This conversation will challenge your perceptions and enlighten you on the path forward. Don't miss this essential deep dive into safety and intentional design in the age of AI. Chapters: 00:00 - The Inevitable Rise of Superintelligent AI 01:21 - Welcome to Facing Disruption with AJ Bob 02:40 - Dr. Craig Kaplan's Long-Term View on AI 04:45 - The Unprecedented Speed of AI Advancement 07:58 - Addressing AI Risks from Existential to Biases 11:12 - Defining "Smarter" in Artificial Intelligence 14:40 - AI's Ability to Process and Make Decisions 17:27 - Innovation vs. Invention in the World of AI 20:03 - The Importance of Human Values in AI Development 22:24 - The Natural Progression to Democratic AI Systems 26:49 - When AI Mimics Human Civilization 29:46 - AI as a Child: Our Responsibility to Nurture Values 35:36 - The Crucial Role of Humans in Setting AI Values 38:18 - What Humans Can Learn from AI 42:56 - The Precision of Communication: Humans and AI 46:17 - The Growing Importance of Liberal Arts in the AI Era Connect with us: - Subscribe to Facing Disruption for more insights into technology and humanity! - Like this video if you found it informative. - Share your thoughts in the comments below: What are your biggest hopes and fears for the future of AI? Thanks to our sponsors: mxp and Comvia Studio. Connect with AJ Bubb and Facing Disruption: LinkedIn: https://www.linkedin.com/in/ajbubb/ Website: https://facingdisruption.com/ Podcast: Facing Disruption on your favorite podcast platform Brought by: MXP Studio and Convia Studio #AI #ArtificialIntelligence #Innovation #Leadership #TechTrends #FutureofWork #FacingDisruption #AJBubb #TheWeirdCanadian #AIethics #PersonalGrowth #DigitalTransformation #TechTalk #Podcast #Optimism ---------------------------------------------------------- AJ Bubb is the founder of MxP Studio - a human first innovation agency, founded on the principles of customer obsession, and helping teams adopt and augment their capabilities with AI, not replace. Bringing more than 15 years of experience across tech, consulting, startups, and enterprise - AJ has led engineer and product teams at places such as Amazon, Accenture, Google, and many more, launching over 100 prototypes, pilots, and products. AJ is also and active mentor for leaders, teams, and innovators. Connect MxP Studio: https://mxp.studio LinkedIn: https://www.linkedin.com/in/ajbubb/ -------------------------------------------------------- 🎤 Subscribe for more insights, like, and share how YOU are Facing Disruption! Check out our playlists Want to build better products and teams? Rockstar Product Teams: https://www.youtube.com/playlist?list=PLzb8muOeIslVT92614HCepmI0NqZT5cL_ Looking to grow and stay resilient as an entrepreneur? Experimenter's Mindset: https://www.youtube.com/playlist?list=PLzb8muOeIslX_Jl1dbtCnE2sHjauafk1u Interested in learning how AI is disrupting industries? The Future of Series: https://www.youtube.com/playlist?list=PLzb8muOeIslUlbTyR9poIFXKVqcnXqLrh

Join us as we dive into the fascinating world of AI with Dr. Craig A. Kaplan, CEO of superintelligence.com. Discover the evolution of AI from its early days to the rise of generative AI and the implications for the future. Key takeaways: 00:00 Introduction to Dr. Craig A. Kaplan 00:29 Historical perspective on AI 01:24 The shift from narrow AI to AGI 02:19 The emergence of superintelligence 05:00 Safety concerns in AI development 06:11 The role of AI agents in business 08:01 Collective intelligence and its future What's your biggest challenge with AI? Drop it in the comments! Subscribe for weekly insights on AI and technology trends! #ArtificialIntelligence #Superintelligence #GenerativeAI Episode # 190 Today’s Guest: Dr. Craig A. Kaplan, Inventor of the designs and Technologies that enable safe SuperIntelligence. He is a pioneer in artificial intelligence and the inventor behind technologies designed for safe Superintelligence. For more than four decades, he has worked at the intersection of intelligent systems, ethics, and innovation, developing architectures that help AI evolve safely and remain aligned with human values. What Listeners Will Learn: How AI evolved from symbolic systems to Generative AI The difference between AI, AGI, and Superintelligence Why are many AI researchers concerned about AI safety Enterprise AI risks leaders should understand today Why AI agents are becoming the next major AI wave The rise of multi-agent and collective intelligence systems How organizations can design safer AI solutions Why AI is shifting from a tool to a digital coworker The future impact of AI on jobs and knowledge work Practical guidance for responsible AI adoption

Superintelligence isn’t science fiction anymore. In this conversation, world‑leading safe AI expert Dr. Craig A. Kaplan (Carnegie Mellon, Nobel Prize winner collaborator ), explains why AI is already behaving less like a tool and more like a worker – and why the next jump to true superintelligence could change everything about how humans live, work, and make decisions. We talk about how AI went from clunky 1980s experiments to today’s large language models, why “machine learning” is only one part of a much bigger AI story, and what it really means to treat AI as an intelligent entity rather than a gadget. Dr. Kaplan shares why he’s still optimistic – even while estimating a non‑trivial risk that advanced AI could go catastrophically wrong – and what we can do right now to push the odds toward the best possible future. You’ll hear us dive into: Why AI is shifting from “tool” to “worker” and eventually to “co‑intelligence” with humans The difference between AI, machine learning, and cognitive science The “AI as a child” analogy and what it gets right about safety and values How early neural networks struggled to recognize a single letter, and why those same ideas now run the systems changing the world Why computing power and data – not just algorithms – unlocked today’s AI boom What democratic, community‑driven AI could look like and why it might be our best safety mechanism How to think about the real risk that superintelligence poses… without becoming a doomer If you’re trying to understand where AI is actually headed – beyond hype tweets and doom headlines – this episode will give you a grounded, expert‑level roadmap for the next decade and what it means for you, your work, and your family. Chapters: 00:00 – The Secret History of AI's Invention 01:12 – The 20% Risk: Why AI Could Be Dangerous 03:41 – Did Mark Zuckerberg Name Facebook After This? 06:15 – How I Got a "Full Ride" to Study AI 14:00 – The One Problem That Solves All Others 16:38 – Meeting the "Godfather of AI" in the 80s 18:19 – Why Self-Driving Cars Failed in 1980 20:44 – AI vs. Machine Learning: What's the Difference? 24:47 – Are Humans Just "Intelligent Systems"? 28:31 – Why AI Isn't a Tool—It’s a Worker 32:21 – The Parent-Child Analogy for AI Safety 37:42 – Why I’m an AI Optimist (80% Success Rate) 42:18 – How Humans Can Role Model Better Values for AI 45:46 – Democratic AI: A New Architecture for Safety 49:58 – Beating Wall Street With Collective Intelligence 53:34 – Can AI Actually Be Creative? (The 1956 Proof) 1:00:32 – The Rise of Synthetic Data 1:13:21 – What’s Next: Personalizing Your Own AI Agent 1:29:19 – The Resurgence of Reasoning in AI 1:39:00 – An Ounce of Prevention: The Future of AI Safety Connect with Dr. Kaplan: Substack: https://read.superintelligence.com/ SuperIntelligence: https://www.superintelligence.com iQ Company: https://www.iqco.com LinkedIn: https://www.linkedin.com/in/craigakaplan

Summary: What's the difference between the AI in your homework helper and true artificial general intelligence (AGI)? Dr. Craig Kaplan helps us understand AGI, narrow AI breakthroughs, and why democratizing AI literacy starts with answering this question. Perfect for students, parents, and teachers navigating AI in education. Addresses transparency in AI architectures, how to build a safe and beneficial AGI through personalized agents, networked intelligence, and transparent interactions rather than ever-larger black-box models. Key takeaways: AI safety should be designed into system architecture from the start rather than added after deployment. Personalized AI agents should encode not only expertise but also values, ethics, and aesthetic preferences. A network of many agents, combined with human participation, may produce stronger and safer collective intelligence than a single giant model. Humans are necessary on the network because they contribute ethics, common sense, and world knowledge that AI systems still lack. Multimodality strengthens representation and may be crucial for more capable and grounded AI systems. Future AI may not only answer human questions but also propose new questions and new scientific or strategic problems. Human critical thinking remains indispensable because today’s AI systems often produce confident but incorrect answers. Transparency in interactions, auditability, and governance are central to safe AI deployment. AI literacy is not just about tool fluency; it is about understanding mechanisms, limits, risks, and responsibilities. The coming years may be decisive because AI capabilities are improving very rapidly, possibly faster than institutions can adapt. Guest bio: Dr. Craig Kaplan is an AI researcher, technology entrepreneur, and long-time builder of intelligence systems with more than three decades of experience in advanced AI architectures. He was trained at Carnegie Mellon and worked with Nobel laureate Herbert Simon, one of the founding figures of artificial intelligence. Chapters: 00:00 Introduction and Guest Background 02:00 Craig Kaplan's Vision for AI and AGI 03:32 Personalized AI Agents and Their Potential 06:20 The Role of Human Values and Ethics in AI 08:58 Collective Intelligence and Networked AI Systems 13:20 Learning, Updating, and Knowledge Transfer in AI 17:50 World Models, Self-Awareness, and Consciousness 22:17 Transparency, Black Boxes, and Safety Challenges 26:29 Speed of AI Development and Urgency of Safety Measures 31:03 AI Creativity, Problem Posing, and Long-Term Questions 35:25 Human-AI Collaboration and Ethical Guidance 39:47 AI Governance, Regulation, and Democratic Values 43:56 Risks, Pitfalls, and the Need for Responsible Design

What if AI doesn’t just change your job… but reshapes what it means to be human? In this episode of Technically Human, I sit down with AI pioneer Dr. Craig Kaplan to unpack the future of work, AI safety, and what most people are getting wrong right now. If you're navigating career uncertainty, layoffs, or trying to understand how AI actually impacts your future, this conversation will challenge how you think about both technology and your role in it. Time Stamps: 00:00 Intro 01:15 Meet Dr. Craig Kaplan 04:00 The real history of AI 10:30 How companies used AI before ChatGPT 15:00 Will AI take jobs? 20:00 Where to start with AI tools 26:00 AI safety and risks 32:00 Bias in AI systems 37:00 What it means to stay human 41:00 Closing thoughts Key Topics: The real history of AI (and why it’s not new) Why companies can’t afford to ignore AI anymore The truth about AI and job loss (and what to do about it) AI safety risks most people aren’t talking about “AI won’t take your job, but someone using AI might” What it actually means to “stay human” in an AI-driven world How your everyday behavior is training AI systems “AI won’t know what’s right or wrong, it will learn that from us.” Dr. Craig Kaplan has been working in artificial intelligence since the 1980s and studied under Nobel Prize winner Herbert Simon. His work focuses on AI safety, human-AI collaboration, and collective intelligence. Learn more about what Dr. Craig Kaplan focuses on at https://superintelligence.com Want help navigating your next career move? Start here: www.careercompassusa.com Subscribe for more conversations on AI, careers, and staying human in a digital world.

AI Strategist Shares How To Create Profitable Safer AI Superintelligence - Craig Kaplan and Lily Patrascu Craig Kaplan is a technology entrepreneur, AI strategist, and business leader with over 33 years of experience helping companies leverage innovation for scalable growth. He explains that AI is rapidly evolving from narrow systems toward artificial general intelligence and eventually superintelligence, which could surpass human expertise. Craig emphasizes that companies can use AI to drive profitability by automating complex cognitive tasks while scaling faster than traditional methods. At the same time, he stresses that safety must remain a core priority, as increasingly advanced systems can introduce risks such as cybersecurity vulnerabilities and misalignment with human goals. Craig believes the key is designing AI systems that are aligned with human values and statistically more likely to produce positive outcomes. Ultimately, he argues that the safest AI systems will also be the most profitable, challenging the idea that safety and speed must come at the expense of each other. About The Guest Dr. Craig A. Kaplan is a renowned expert in artificial intelligence, artificial general intelligence, and superintelligence, with a focus on collective intelligence and quantitative modeling. He is the Founder of Superintellligence.com and CEO and founder of iQ Company, a consulting firm dedicated to advanced AGI and SI systems. Previously, he founded PredictWallStreet, a financial services firm that powered top hedge fund performance by leveraging the collective intelligence of retail investors. Dr. Kaplan has authored a book, published extensively in scientific journals, and holds numerous patents on AI-related technologies. For More Info About Craig Kaplan, visit: https://www.superintelligence.com https://www.iqco.com https://www.linkedin.com/in/craigakaplan Timestamp 00:00 Profitable & Safer AI Superintelligence 00:40 Meet Craig Kaplan & His AI Vision 01:30 What “AI Superintelligence” Really Means 02:40 Why AI Is Advancing Faster Than Expected 04:00 The Biggest Opportunity in AI Right Now 05:20 Profit vs Safety: The Core Dilemma 06:40 Why Most AI Companies Get This Wrong 08:00 The Hidden Risks of Uncontrolled AI Growth 09:30 What Happens If We Don’t Prioritize Safety 11:00 Can AI Truly Be Controlled? 12:30 Building Guardrails for Advanced AI Systems 14:00 The Role of Ethics in AI Development 15:30 Aligning AI with Human Values 17:00 How to Make AI Both Profitable AND Safe 18:30 Business Models That Win in the AI Era 20:00 Replacing Human Roles vs Augmenting Them 21:30 AI for Entrepreneurs: Where to Start 23:00 Creating Scalable AI-Driven Companies 24:30 The Future of Work in an AI-Dominated World 26:00 How Founders Should Think About AI Strategy 27:30 Mistakes to Avoid When Building with AI 29:00 Data, Control & Power: Who Owns the Future? 30:30 Open vs Closed AI Systems Debate 32:00 Regulation: Necessary or Dangerous? 33:30 Governments vs Innovators: The Power Struggle 35:00 Timeline to Superintelligence (Realistically) 36:30 What Society Must Do to Prepare 38:00 The Economic Impact of AI at Scale 39:30 Creating Long-Term Competitive Advantage 41:00 AI Leadership: Who Will Win the Race? 42:30 The Role of Individuals in the AI Revolution 44:00 Final Framework: Safer, Smarter AI Systems 46:00 Key Takeaways for Entrepreneurs 47:30 Closing Thoughts & Future Outlook About The Host Lily Patrascu is an ultra-creative book publisher and personal brand strategist for thought leaders, CEOs, luxury influencers. As an influencer, bestselling author, copywriter, global speaker, she supports busy entrepreneurs to find their message and share it with millions of people. Create Profitable Safer AI Superintelligence - Craig Kaplan and Lily Patrascu - https://aisuperwomanpodcast.com/ If you are looking for Create Profitable Safer AI Superintelligence - Craig Kaplan and Lily Patrascu, then watch this video to learn everything you need to know about Create Profitable Safer AI Superintelligence - Craig Kaplan and Lily Patrascu #SaferAI #CraigKaplan #LilyPatrascu #AIStrategy #ResponsibleAI #AIInnovation #Superintelligence #TechLeadership #AIBusiness #FutureOfAI In this AI Superwoman Podcast episode, we sit down with Craig Kaplan, Founder & CEO of Superintelligence.com & IQ Company. We discuss how to create profitable, safer artificial superintelligence, exploring the future of ai and its implications. This conversation provides key insights into navigating the complexities of ai business.

*agi, ai agents, superintelligence, herbert simon, ai research, ai alignment, ai predictions, ai safety, future of ai 2026, ai governance, ai dangers, ai doom, cognitive science, future of learning* Summary This interview with Dr. Craig Kaplan explores the future of safe AGI development, emphasizing democratic AI systems, collective intelligence, and the importance of aligning AI values with human ethics. It addresses the risks, challenges, and philosophical questions surrounding AI safety and governance. In this insightful interview, Dr. Craig Kaplan explores the future of AI, its ethical implications, and how we can prepare for a post-AGI world by developing critical thinking and values. Discover how AI reasoning, safety, and economic models could reshape society and the importance of aligning AI development with human values. 👤 About Dr. Craig Kaplan: Dr. Craig Kaplan has been building intelligent systems since the 1980s, long before AI was famous. He co-authored research with Herbert Simon, a Nobel laureate and one of the founding fathers of AI. He built & sold a Silicon Valley company, called PredictWallStreet, that traded billions using collective intelligence. And in 2006, nearly two decades before it became a buzzword, he bought the domain 'superintelligence.' 💡 Featured Experts: ▶️ Dr. Craig Kaplan, Founder & CEO @ iQ Company ▶️ Host: Aashka Patel In this episode: 00:00 AIR Bites (Precap) 01:55 AGI Race Is Broken (Both Sides Are Wrong) 03:45 The Third Path: "Democratic AI" 04:34 Why One AI Can't Beat Millions (Collective Intelligence) 06:19 AI Agents → Multi-Agent Systems → Superintelligence 07:23 Collective Intelligence: Harnessing Human and AI Collaboration 08:07 Your Personal AI Clone (With Your Values) 09:26 Why Democracy Fails Today (And How AI Could Fix It) 10:15 The Challenge of Amplifying Voices in AI 13:07 Probabilities and Perspectives: Understanding Risks 14:31 P(Doom): Why 1% Change Saves 83 Million Lives 15:57 Dynamic Values: Adapting AI to Human Ethics 19:42 Your Behavior Is Training AI (Right Now) 19:57 Social Media Is Warping Reality (AI Sees the Truth) 20:28 AI's Objective Reality: A New Lens on Human Behavior 26:14 Constitutional AI: The Need for Individual Values 26:40 Why "Constitutional AI" Might Fail 30:35 India vs US Values: Can AI Respect Both? 34:13 The Ethics of Outrage and AI Training 34:42 Humans Are Better Than We Think (Data Proves It) 36:41 Mythos AI Escaped. This Changes Everything 38:00 The Rise of Superintelligent AI 39:54 Only AI Can Control AI (Scary Truth) 40:23 The "Tree Moment": Humans Can't Keep Up Anymore 45:38 Aligning AI Safety with Economic Incentives 48:26 Why AI Safety Is Failing (And What Actually Works) 48:54 Earn While You Sleep (AI Versions of You) 51:57 Replace Ads With Money-Making AI Tasks 58:41 Are AI 'Reasoning' Models Actually Thinking? A Cognitive Scientist Answers 59:23 Cognitive Science and AI Learning 59:52 Are AI Models Actually "Thinking"? 01:01:09 From Autocomplete → Real Reasoning 01:03:25 How Humans (and AI) Actually Solve Problems 01:08:37 Herb Simon Proved AI Was Creative in 1956, Nobody Noticed 01:10:25 The One Skill That Will Save Kids From an AGI World 01:11:12 Fostering Critical Thinking in Education 01:14:12 Why You Should Actively Seek Out Opinions You HATE Link to all the whitepapers discussed in the episode: https://www.superintelligence.com/si-research-whitepapers Key topics: Democratic AI systems with checks and balances Collective intelligence and multi-agent systems Evolving human values and AI alignment AI safety and ethics AI reasoning and problem solving Economic models and AI marketplace Sound bites: "Design a system with checks and balances." "AI can be more objective than humans." "AI can learn from solving real problems." Keywords: AI safety, democratic AI, collective intelligence, AGI, AI ethics, AI risks, superintelligence, AI governance, AI alignment, future of AI AI safety, cognitive science, AI reasoning, post-AGI world, AI ethics, human values, AI marketplace, AI training, AI and capitalism, AI problem solving Before you go… 🔔 SUBSCRIBE and hit the notification bell to catch all future episodes! Stay connected with our guests on: Dr. Craig Kaplan: https://www.linkedin.com/in/craigakaplan/ https://www.youtube.com/@UCbooeQXLH1639ZHOdGxBHYA Whitepapers: https://www.superintelligence.com/si-research-whitepapers Follow “On AIR with Aashka” on: Instagram: https://www.instagram.com/onairwithaashka LinkedIn: https://www.linkedin.com/company/on-air-with-aashka/ Twitter/X: https://www.x.com/onairwithaashka Follow me on: LinkedIn: https://www.linkedin.com/in/aashkapatel608/ Twitter/X: https://x.com/_raconteurre_ ✨ Join our discussion by commenting below. Drop your thoughts! Disclaimer: Views expressed belong to the speakers and are for educational purposes. #agi #superintelligence #aipodcast

For more thoughts, clips, and updates, follow Avetis Antaplyan: Spotify: https://open.spotify.com/show/0rOkUXDSQb6SVFE6LttWDe Apple: https://podcasts.apple.com/us/podcast/the-tech-leaders-playbook/id1690263628 Artificial intelligence is no longer just a tool, it is becoming a worker. And most people are underestimating what comes next. In this episode of The Tech Leader’s Playbook, Avetis Antaplyan sits down with Dr. Craig Kaplan, AI pioneer and founder of iQ Company, to break down the real trajectory of AI, from its early foundations to today’s rise of agentic systems and the path toward AGI. Dr. Kaplan explains why the biggest shift in artificial intelligence is happening right now: AI systems are no longer just assisting humans, they are beginning to act, decide, and operate independently. The conversation explores what this means for jobs, entrepreneurship, and the future of work, as well as the risks of building powerful black-box systems without proper transparency or alignment. Topics covered include: AGI, AI replacing jobs, agentic AI, collective intelligence, and the future of work. Dr. Kaplan traces the history of AI from symbolic reasoning to modern machine learning and agentic systems, sharing lessons from building PredictWallStreet, a collective intelligence platform that used signals from millions of retail investors to power a top-ranked hedge fund. He explains why communities of agents may become more powerful than any single model, and why the future of AI safety may depend as much on human behavior and data as it does on technical breakthroughs. Takeaways AI did not appear overnight. Its formal roots go back to the 1956 Dartmouth conference, with major eras including symbolic AI, machine learning, and now agentic AI. Modern AI is advancing because researchers are combining broad machine learning capabilities with older traditions of sequential reasoning and problem solving. The biggest shift now is from AI as a tool to AI as a worker that can use tools and take action on a user’s behalf. In collective systems, even inaccurate inputs can become valuable if they are consistent and can be filtered or weighted correctly. Entry-level cognitive work is already under pressure, while top performers and people with rare, non-commoditized knowledge still hold an edge. AI safety is becoming critical as today’s systems operate as powerful black boxes that are difficult to predict or control. More transparent, collective-intelligence-style systems may offer a safer path forward than monolithic models. Chapters 00:00 Intro and why AI should be thought of as a worker, not just a tool 01:29 Dr. Craig Kaplan’s early path into AI and the field’s history 03:26 What people miss about the decades of groundwork behind today’s AI boom 05:05 The signals that show AI is entering a new phase 07:02 Why agentic AI is so powerful for entrepreneurs and small teams 08:24 The origin story behind PredictWallStreet and collective intelligence 12:42 How crowd wisdom works, and how noise can still produce signal 16:09 The long-term trajectory from narrow AI to AGI to superintelligence 19:13 Why communities of agents may outperform any single model 22:09 Jobs, competition, and what happens to human work as AI improves 29:21 Why only exceptional human expertise may remain defensible 34:00 Did humanity create the conditions for AI to replace so much labor? 40:49 Why trade jobs may be safer in the short term than white-collar roles 42:30 AI safety, existential risk, and why black-box systems are dangerous 47:13 A safer alternative: transparent, democratic, collective AI systems 50:33 What ordinary people and business leaders can do right now 54:29 The book and core idea that shaped Dr. Kaplan’s thinking on reason and values 57:12 Final message: if AI is our child, we need to teach it well 58:50 Closing thoughts and outro Craig Kaplan’s Social Media Link: https://www.linkedin.com/in/craigakaplan/ Craig Kaplan’s Website Link: https://www.iqco.com/ Resources and Links: https://www.hireclout.com https://www.podcast.hireclout.com https://www.linkedin.com/in/hirefasthireright

Brad Moss sits down with Dr. Craig A. Kaplan, founder of The IQ Company and creator of Predict Wall Street, to unpack a lifetime spent building intelligent systems—from early symbolic Ai at Carnegie Mellon with Nobel laureate Herbert Simon, to harnessing the crowd to power a top-10 hedge fund, to his current work on Ai safety and superintelligence. They explore how intelligence has evolved from human to collective to machine, why today’s black-box models are hitting limits, and how a “democracy of Ai agents” could deliver safer, cheaper, and more powerful superintelligence without pausing innovation. Craig and Brad dive into context as the next frontier, the ethics and values that must shape AGI, and a future where personalized “Jarvis-like” agents carry each person’s unique preferences and moral compass into a global network of cooperating Ais.

Artificial intelligence is moving fast; faster than most business leaders realize. In this episode of the Human-First AI Marketing Podcast, Mike Montague sits down with AI researcher Dr. Craig Kaplan to explore what may be the next major shift in technology: the transition from AI as a simple tool to something closer to an intelligent entity. The conversation tackles the big questions people are asking right now: Will AI become smarter than humans? How quickly could that happen? And what does it mean for businesses navigating the AI revolution? Along the way, Mike and Craig explore the real risks and opportunities of advanced AI, including the difference between the “Terminator scenario” and the “Iron Man scenario,” the race between governments and tech companies to build more powerful systems, and why many experts believe AI development is accelerating faster than expected. For SMB leaders, marketers, and curious innovators, this episode offers a chance to step back and see the bigger picture. You’ll walk away with a clearer understanding of where AI is heading, what superintelligence could mean for society, and why the choices we make today will shape how humans and intelligent machines work together in the future. Top Takeaways AI has evolved through three major phases. It moved from rule-based symbolic systems to machine learning and now into the emerging era of autonomous AI agents. We are shifting from tools to intelligent entities. Modern AI systems are beginning to act more like independent intelligences that can take actions and make decisions. Artificial General Intelligence (AGI) is the next milestone. AGI refers to AI capable of performing any cognitive task at the level of an average human. Artificial Superintelligence (ASI) may follow quickly. Once AI reaches general intelligence, it could rapidly surpass humans across many fields. Many experts believe the risk of catastrophe is non-zero. Poorly aligned superintelligence could pose serious risks if development continues without safeguards. The biggest danger may come from the AI arms race. Nations and tech companies are competing to build more powerful systems, making it difficult to slow development. Regulating AI will be harder than past technologies. Because AI is widely distributed, traditional regulation models may struggle to keep up. Advanced AI will likely develop goals and values. The values embedded in these systems will shape how they interact with humanity. The future could resemble the “Terminator” or “Iron Man” scenario. Outcomes depend on whether AI systems are aligned with human interests. The choices we make now matter. Today’s research, safety standards, and design decisions will determine whether AI becomes humanity’s greatest tool or its greatest risk.

AI is moving from copilots to autonomous agents, and most tech leaders are not prepared for what that shift means. Craig walks through the real risks behind superintelligence, why AI checking AI is becoming inevitable, and how CTOs and CIOs can design safer, more resilient systems before autonomy outpaces human oversight Rather than focusing on hype, this episode dives into the alignment problem, the limits of guardrails, and why monolithic black box models may be the wrong long term architecture. You will hear a practical path forward for tech leaders who are already overwhelmed by AI generated code, agent frameworks, and rapidly evolving models If you are leading engineering, AI, or technology strategy, this episode will challenge how you think about safety, governance, autonomy, and the future role of the CTO in an AI driven world. Key Takeaways • Why AI checking AI is not optional as code generation and autonomy scale • What the alignment problem really means for enterprise technology leaders • The limits of guardrails and why prevention at design stage beats patching at deployment • How democratic architectures of multiple agents can reduce systemic risk • Why vendor agnostic, multi model strategies increase resilience and strategic control • How to embed company values into AI systems through training, memory, and architecture • What P(doom) represents and why many leading researchers assign it far higher risk than most executives assume *About Craig* Dr. Craig A. Kaplan is a renowned expert in artificial intelligence, artificial general intelligence, and superintelligence, with a focus on collective intelligence and quantitative modeling. He is the Founder of Superintellligence.com and CEO and founder of iQ Company, a consulting firm dedicated to advanced AGI and SI systems. Previously, he founded PredictWallStreet, a financial services firm that powered top hedge fund performance by leveraging the collective intelligence of retail investors. Dr. Kaplan has authored a book, published extensively in scientific journals, and holds numerous patents on AI-related technologies.  *Chapters* 00:00 How far is AGI? 06:55 What is P(doom)? 16:43 AI Reviewing AI Output 20:54 Dealing with Bad Actors (Human or AI) 25:36 Approaching AI as a Small Scale CTO 30:43 Democracy of AI Agents 35:15 AI Safety Conferences 40:03 AI Models, Open-Source or Big Company? 45:05 Is AI Adoption Keeping Up? *Where to find Craig* * LinkedIn: https://www.linkedin.com/in/craigakaplan * Website: https://www.superintelligence.com * Website: https://iqco.com * YouTube: @iqstudios1

The AI Scientist Who Called Nvidia Early Is Back — And The Questions Are Different This Time When Dr. Craig Kaplan first joined me, Nvidia was still being debated. He called it from first principles when Wall Street was still asking whether the run-up made sense. He returns this Wednesday for his fourth appearance — and the landscape has changed completely. Gemini 3.1 Pro. Claude Opus 4.6 and Sonnet 4.6. GPT-5.4. Grok 4.20. DeepSeek V4 imminent. Claude Code. Seedance. A one-million token context window becoming the new standard. Agents finally moving from demo to deployment. Craig is Founder and CEO of SuperIntelligence.com and iQ Company — an AI scientist since the 1980s and a retired hedge fund manager. He evaluates these systems with a rigor most investors and commentators simply do not have. We are going deep on: — Which models are actually ahead right now and why benchmarks may be hiding the real story — Context windows: why the jump to one million tokens is not a spec upgrade but a fundamental shift in what AI can do for a business — Claude vs Gemini vs GPT vs Grok — a real compare and contrast, not marketing — DeepSeek V4 and why China's open-weight strategy may be the most underappreciated geopolitical AI move happening right now — Claude Code, Seedance, and the application layer that is finally arriving — Grok 4.20's four-agent parallel architecture and why it is technically novel — The transformer architecture question nobody wants to ask: are we scaling toward superintelligence on a foundation we do not mechanistically understand? — Recursive self-improvement — the threshold every safety researcher agrees is the critical one — and how close any system actually is to it — The Mag 7 updated scorecard with real conviction changes — Robotics, Nvidia, the infrastructure layer, and where the next ten-year money actually is — How family offices should build an AI investment framework from scratch in March 2026 This is not a hype session. Craig brings scientific depth and investment discipline to a conversation most people are having with neither. Wednesday, March 11th | 4:00 PM ET | Live Register below. Free to attend. --- Family offices managing generational wealth need more than advisors — they need a network operating 72 hours ahead of the curve. SFO Continuity delivers sovereign AI blueprints, peer roundtables, and proprietary frameworks forged inside 100+ billionaire ecosystems. Membership is limited to 100 family offices. Learn more: https://www.angelorobles.com/membership #FamilyOffice #AI #ArtificialIntelligence #Nvidia #Claude #Gemini #GPT5 #DeepSeek #AIAgents #Investing #SFOContinuity #Superintelligence #FamilyOfficeTV #AngeloRobles #AIInvesting #angelorobles

Episode 65: Anastassia and Dr. Craig Kaplan delve into the complexities of artificial general intelligence (AGI) and the evolving landscape of AI technologies. Craig emphasizes the importance of defining AGI as an AI capable of performing any cognitive task as well as an average human, highlighting the challenges of achieving true general intelligence beyond narrow applications. They discuss the historical context of AI development, the shift from symbolic AI to machine learning, and the potential of collective intelligence as a more effective approach to building AGI. Craig advocates for a community of models rather than a single monolithic AI, suggesting that this could lead to safer and more ethical AI systems that reflect diverse human values. The conversation also touches on the limitations of current AI systems, particularly their lack of understanding of causality and reasoning. Craig argues that while AI might develop its own sense of purpose, it is crucial to instill positive human values early on to guide its development. The discussion concludes by emphasizing the importance of AI literacy and critical thinking, noting that human behavior and values will significantly shape the future of AI and its impact on society. Craig A. Kaplan is an artificial general intelligence (AGI) expert and entrepreneur who focuses on collective intelligence, safe superintelligence, and practical strategies for aligning advanced AI with human values and goals. He has founded and led multiple AI-related ventures, including iQ Company, which develops AI systems to enhance human decision-making; previously, PredictWallStreet, an early crowdsourced stock prediction platform; and he speaks and writes about how to safely build and govern increasingly powerful AI systems. Takeaways: AGI is defined as AI that can perform any cognitive task like an average human. The shift from symbolic AI to machine learning in the 1960s and 1970s, big data and superb semiconductors later on enabled today’s AI revolution. Collective intelligence may offer a safer and more effective path to AGI, and this include development of individual LLMs and models based on values and perspectives of individual humans. Current AI systems lack an understanding of causality and reasoning. AI will develop its own sense of purpose, but early values are crucial. AI Literacy is imperative to build safe, transparent and beneficial AI. Chapters: 00:00 Introduction to the episode: Researching Artificial General Intelligence (AGI) and the work of Dr. Craig Kaplan 2:06 Introduction to AGI and AI Definitions 04:16 The Evolution of AI: From Symbolic to Machine Learning 07:02 The Limitations of Current AI Systems 14:01 Causality and Reasoning in AI 19:38 The Collective Intelligence Approach to AGI 26:46 The Future of AI: Transparency and Collaboration 28:37 The Purpose of AI Collectives 29:25 Utopia vs. Reality in AI Development 30:49 The Risks of AI: Understanding P-Doom 32:16 Human Values vs. AI Intelligence 35:09 Fusing Humanities with AI Engineering 37:40 The Role of Human Responsibility in AI 40:22 The Evolution of AI Values 44:59 The Bell Curve of Society and AI's Reflection 47:42 Education and AI: Building a Better Future 49:38 The Necessity of AI Literacy and Critical Thinking

In this episode, Kristian Marquez sits down with Craig Kaplan, the CEO of iQ Company and founder of superintelligence.com. With over 30 years of experience in the field, Craig breaks down the transition from rule-based AI to the "black box" of modern machine learning. They discuss the critical need for transparency in AI, the "alignment problem," and Craig’s innovative "Democratic AI" model. Whether you are a founder looking to integrate AI agents or an investor seeking an edge, this conversation provides a high-level roadmap for the most transformative technology of our time. Learn more about iQ Company's work at: https://www.iqco.com/ Connect with Craig Kaplan on LinkedIn at: https://www.linkedin.com/in/craigakaplan/ Think you'd be a great guest on the show? Apply at https://finstratmgmt.com/innovators-investors-podcast/ Want to learn more about Kristian Marquez's work? Check out his website at https://finstratmgmt.com

In This Episode: We sit down with Dr. Craig A. Kaplan, the PhD-wielding brains behind Predict Wall Street (and, apparently, the patron saint of retail investors everywhere). Craig walks us through 14 years in the financial trenches, building tech to turn the “dumb money” crowd into a collective force that could actually beat Wall Street at its own game—at least until the hedge funds caught the scent and crashed the party. Brace yourself for a story that’s equal parts “wisdom of crowds,” collective intelligence research, and “what happens when the little guy dares to play with the big boys.” If you still think your hot stock tip has a shot, prepare to have that illusion gently (okay, not-so-gently) shattered.

Summary In this episode of So What About AI Agents Philippe Trounev and Dr. Craig Kaplan discusses the need for a new approach to AI safety and governance, emphasizing the importance of prevention in design and the concept of AI agents and collective intelligence systems. He highlights the role of ethics and morals in agentic society, the enforcement of ethics and morals in AI agents, and the purpose and values of AI agents. Dr. Kaplan also explores the blueprint for collective intelligence systems, problem-solving and coordination in multi-agent systems, transparency and accountability, decentralization of power, observation and reporting, and the role of values in AI systems. He concludes by discussing the relevance of Herbert Simon's ideas in AI research. takeaways Democracy in AI governance can enhance safety. AI agents can work together like a community. Ethics in AI must be enforced through safeguards. Collective intelligence can outperform individual expertise. Designing AI systems requires careful consideration. Transparency is crucial for AI agent interactions. Values from diverse individuals should shape AI behavior. The historical context of AI informs current practices. Short-term fixes are not sufficient for AI safety. Our online behavior influences future AI training. titles Building Safe AI: A Democratic Approach The Future of AI Governance Sound Bites "Two heads are better than one." "We need to think hard about design." "We should behave well online." Chapters 00:00 Introduction to AI and Superintelligence 01:20 Governance and Safety in AI 05:30 The Role of AI Agents in Society 07:29 Evolving Towards Agentic Democracy 09:35 Ethics and Morals in Agentic Society 12:16 Influence vs. Enforcement in AI Behavior 15:52 Blueprint for Collective Intelligence Systems 19:39 Human Traits in AI Collective Intelligence 22:49 Transparency and Accountability Among Agents 25:25 Decentralization and Power Distribution 29:35 Learning from Human Governance 33:20 Herbert Simon's Insights on AI and Morality 36:42 Key Takeaways for AI Governance

In this episode of Conversations with Zena, My AI Colleague, host David Espindola is joined by Dr. Craig Kaplan, a pioneer in artificial intelligence whose career spans more than four decades. From his early work at Carnegie Mellon alongside Nobel laureate Herbert Simon to his current focus on AI safety and ethics, Craig has spent his career thinking deeply about how intelligent systems evolve—and how they can be designed to remain aligned with human values. He is also the founder of Superintelligence.com, a domain he secured nearly two decades ago, long before the topic entered mainstream discourse. The conversation begins by grounding listeners in the evolution of AI—from narrow, task-specific systems to artificial general intelligence (AGI), and ultimately to superintelligence: systems that could outperform even the most capable humans across nearly all cognitive domains. Craig explains why timelines once measured in decades are now being compressed into years, and why this acceleration has surprised even seasoned AI researchers. From there, the discussion turns to one of the most consequential questions of our time: Can superintelligent systems be aligned with human values? David references the concerns raised by thinkers like Nick Bostrom, who argue that controlling a vastly superior intelligence may be impossible. Craig agrees that traditional notions of “control” are insufficient—but reframes the challenge as a design problem rather than a control problem. Instead of ever-larger, opaque “black box” models, Craig advocates for a collective or democratic intelligence approach: systems composed of many interacting AI agents operating with transparency, checks and balances, and shared rules. Drawing on real-world experience building collective intelligence systems that outperformed elite Wall Street investors, he argues that intelligence can emerge from coordination—not centralization—and that such systems can be both more capable and safer. The conversation deepens when Zena, David’s AI co-host trained on his work and values, joins in. Together they explore how personalized AI agents—each reflecting the values of their human counterpart—could create a more representative, accountable, and ethically grounded AI ecosystem, rather than concentrating moral authority in the hands of a few organizations. A pivotal moment comes with Craig’s explanation of “P(doom)”—the estimated probability that advanced AI could pose an existential risk to humanity. Citing discussions with leading AI researchers, Craig notes that many place this risk between 10% and 20%, a number far too high to ignore. Rather than calling for pauses or relying solely on regulation, he argues for architectures that are safer by design—and that can still move fast and create economic value. The episode closes with a powerful reminder: we are all already shaping the future of AI. Every online action contributes to the data that trains these systems. In that sense, humanity is already teaching AI how to behave—often without realizing it. This is a thoughtful, balanced exploration of superintelligence that takes risk seriously without surrendering to fear, and offers a hopeful vision for how humans and AI might evolve together through transparency, accountability, and shared values. Opening to Conversations with Zena. Music at the the end of each episode Support the show (https://www.buzzsprout.com/2412588/support)

In This Episode: We sit down with Dr. Craig Kaplan—the guy who was building AI before your favorite “disruptor” learned to tie their shoes. Dr. Kaplan takes us on a wild ride from the OG days of artificial intelligence (when data lived on floppy disks and having a “machine learning” project meant you had actual machines to move) to his current crusade: making sure superintelligence doesn’t wreck the joint for the rest of us. Kaplan lays out what everyone’s too scared—or too clueless—to admit: if you wouldn’t leave your wallet with a random LLM, why are you ready to entrust the future of humanity to black-box mega-brains? Spoiler: the answer isn’t “more GPUs.” Dr. Kaplan’s been in the room with the legends, built systems Wall Street still wishes it could copy, and now he’s here to torch lazy thinking about AI safety, collective intelligence, and why your Marcus Aurelius quotes aren’t enough to save you from an existential AI oopsie.

Dr. Craig Kaplan’s lifelong dedication to artificial intelligence is marked by his insistence on merging technical innovation with ethical responsibility. His extensive experience, from working alongside AI pioneers to leading research and startups, has shaped a vision where intelligent systems must be both powerful and aligned with human values. Kaplan embraces the necessity for society to actively engage with AI, advocating for mindfulness in online interactions, as every engagement helps shape the capabilities and ethics of emerging systems. Throughout his journey, Kaplan has underscored the impending transformation of AI from mere tools to autonomous entities, stressing the need to build systems that not only boost productivity but also embody transparency and safety. He believes success and fulfillment stem from following intrinsic passions, maintaining critical thinking when using AI, and understanding the collective impact of individual digital behaviors on the development of safer, more ethical technologies. For those eager to explore practical approaches and deeper insights into designing safer, more transparent AI, Dr. Kaplan offers valuable resources and ongoing research through iQ Company. To see how these principles are put into action and learn more about the future of ethical, responsible AI, visit https://www.iqco.com/about-iq.

What happens when a field races forward faster than society can understand it, let alone shape it? And how do we balance the promise of superintelligence with the responsibility to ensure it reflects the values of the people it will eventually serve? In this episode of AI at Work, I sit down with Dr Craig Kaplan, founder and CEO of iQ Company and SuperIntelligence. He's also a pioneer who has been building intelligent systems since the 1980s, and one of the few voices urging a deliberate, safer path toward AGI. Throughout our conversation, Craig explains why current alignment methods often rely on narrow viewpoints, which creates both ethical and technical blind spots. He shares his belief that the values guiding future intelligence should come from millions of people across cultures rather than a handful of researchers writing a constitution behind closed doors. Drawing on his work at Predict Wall Street, he illustrates how collective intelligence can outperform experts, why diverse viewpoints matter, and how these lessons shape the architecture he believes is needed for safe AGI and the superintelligent systems that follow. His clarity on the difference between tools and entities, and how quickly AI is shifting into the latter category, offers a grounding moment for anyone trying to navigate what comes next. This episode moves beyond fear and hype. Craig talks openly about risk, but he also brings optimism about the potential for systems that are safer, faster to build, less costly, and more reflective of humanity. For leaders wondering how to prepare their organisations, he shares what signals to watch, why transparency and design matter, and how a more democratic approach to intelligence could shift the odds of a better outcome. If you want a clear, thoughtful look at the road ahead for AGI, superintelligence, and the role humans still play in shaping both, you will find a lot to chew on here. Listeners wanting to learn more can explore superintelligence.com, where Craig and the iQ Company team share research, videos, papers, and ways to get involved. What part of this conversation sparks your own questions about the future we are building together? Sponsored by NordLayer: Get the exclusive Black Friday offer: 28% off NordLayer yearly plans with the coupon code: techdaily-28. Valid until December 10th, 2025. Try it risk-free with a 14-day money-back guarantee.

Follow the SSTB Newsletter - https://www.linkedin.com/newsletters/7305986878653022210/ Special thanks to Craig Kaplan, Ph.D. - https://www.linkedin.com/in/craigakaplan/ and visit superintelligence.com. Summary In this episode, Brad and Robert engage with Craig A. Kaplan, a seasoned expert in AI, discussing the evolution of artificial intelligence from its inception to its current state and future implications. They explore the importance of human oversight, the concept of AGI and superintelligence, and the role of collective intelligence in enhancing AI systems. The conversation emphasizes the need for ethical considerations in AI design and the practical steps businesses can take to implement AI solutions effectively. Craig shares insights on the open versus closed model debate and encourages individuals to contribute positively to the AI training process. Keywords AI, superintelligence, AGI, machine learning, ethics, safety, collective intelligence, business applications, open source, AI agents Titles - Navigating the AI Landscape: Past, Present, and Future - The Evolution of AI: From Narrow to Superintelligence Takeaways - AI has evolved through distinct phases since its inception in 1956. - The current AI landscape is marked by rapid advancements and potential lulls. - Human oversight is crucial in the development of AI systems. - AGI represents a significant leap towards machines that can perform any cognitive task. - Collective intelligence can enhance the safety and effectiveness of AI systems. - Ethical considerations must guide AI design to prevent harmful outcomes. - Businesses should focus on integrating AI to improve efficiency and productivity. - Open source models promote collaboration and innovation in AI development. - Practical implementation of AI requires understanding specific business needs. - Individuals can influence AI training by promoting positive online behavior Sound bites - "AI has evolved through distinct phases since its inception." - "Human oversight is crucial in the development of AI systems." - "AI is moving towards the role of an actor like a worker." Chapters 00:00 – Welcome & Introductions 02:14 – The AI Trajectory: 3 Phases of Evolution 05:02 – AI Winters, Surges, and the Current Lull 10:38 – From Tools to Workers: The Age of Agents 14:57 – Designing for Safety in a Fast-Paced World 18:58 – Understanding AGI, Superintelligence & Agent Collectives 25:57 – Big vs. Small: Who Wins the Agent Adoption Race? 28:59 – Democracy as a Blueprint for AI Architecture 36:01 – Building Better Agents: Lessons from Startups and Team Training 38:22 – Power Users to Builders: What’s Next After ChatGPT 42:20 – Blueprinting Safer Systems with Superintelligence.com 50:23 – 2026 Priorities: Where Executives Should Focus Now 54:43 – The Future of Open Models 56:36 – Required Reading: Tree of Thoughts, Eric Schmidt & Stuart Russell 57:14 – Final Message: Be Your Best Self Online 58:59 – The End https://linktr.ee/digitalmeld We build tools that save businesses time and money by helping them unlock their greatest potential and achieve growth through data, automation, and AI-powered solutions.

What if artificial superintelligence - ASI - could be made both more safe and more profitable? Returning to AI & You after a year is Dr. Craig A. Kaplan, talking about how "democratic AI" can do that. Craig, who operates the website superintelligence.com, is the CEO and founder of iQ Company, which focuses on AGI and ASI. He also founded and ran PredictWallStreet, a financial services firm that used AI to power a top hedge fund. Craig earned master’s and doctoral degrees from the renowned robotics hub Carnegie Mellon University, where he co-authored research with Nobel Prize-winning economist and AI pioneer Dr. Herbert A. Simon. We discuss democratic AI, a concept that envisions a collective intelligence of AIs working together safely, and how they communicate with each other. We will also examine the components that comprise these systems and discuss approaches to addressing ethical issues. All this plus our usual look at today's AI headlines! This and all episodes at: https://aiandyou.net/ Transcript and URLs referenced at HumanCusp Blog: https://humancusp.wordpress.com/wp-content/uploads/2025/11/transcript-285-4358.pdf Peter Scott (https://www.peterscott.com/index.html) hosts the weekly podcast, “AI and You”, interviewing internationally renowned authorities on all aspects of AI and its impact, from self-driving cars to anthropology and from ChatGPT to science fiction. Past guests include Stuart Russell, Anil Seth, and David Brin. #superintelligence #ai #agi

What if artificial superintelligence - ASI - could be made both more safe and more profitable? I'm talking with Dr. Craig A. Kaplan, who has the website superintelligence.com, about his concept of "democratic AI." Craig is the CEO and founder of iQ Company, with a focus on AGI and ASI. He also founded and ran PredictWallStreet, a financial services firm that used AI to power a top hedge fund. Craig earned master’s and doctoral degrees from the renowned robotics hub Carnegie Mellon University, where he co-authored research with Nobel Prize-winning economist and AI pioneer Dr. Herbert A. Simon. In Part 2, we discuss the rights of AIs, safe superintelligence, where AI derives its values, and how model vendors might be incentivized to integrate their products into the collective AI intelligence. All this plus our usual look at today's AI headlines! This and all episodes at: https://aiandyou.net/ Transcript and URLs referenced at HumanCusp Blog: https://humancusp.wordpress.com/wp-content/uploads/2025/11/transcript-286-3376.pdf Peter Scott (https://www.peterscott.com/index.html) hosts the weekly podcast, “AI and You”, interviewing internationally renowned authorities on all aspects of AI and its impact, from self-driving cars to anthropology and from ChatGPT to science fiction. Past guests include Stuart Russell, Anil Seth, and David Brin. #superintelligence #ai #agi