1st of all there is no strict definition of what a superintelligence is. But for sure it’s not like a human intelligence. It’s different.
Here is a summary of the key points from the conversation between Sam Altman and Azeem Azhar:
- Sam founded OpenAI in 2015 to responsibly develop and democratize AI. He has been traveling the world to get diverse perspectives on AI hopes, fears, and governance.
- Sam sees tremendous excitement about AI’s potential, but also fear from those unfamiliar with the tech. Concerns vary globally — developing world is focused on near-term benefits, while developed world worries about long-term risks.
- GPT-3 was a small step towards the exponential progress we’ll see. GPT-4 took years more work and focused on alignment before release. We’re far from solving AI safety for superintelligence.
- LLM models have limitations, but will help discover new science. We need new paradigms beyond LLMs to reach AGI.
- Risks of AI disinformation are real but shouldn’t be exaggerated. Solutions require education, governance, and building model safeguards.
- Sam believes AI can bring economic growth to reduce inequality globally by making intelligence/cognition cheaper. But we’ll likely need new wealth distribution systems.
- As AI progresses, we’ll build tools, not independent creatures. Superintelligence will emerge from AI-assisted humans, knowledge, tech, and institutions we build. An exciting future if we work for it.
- With AI’s speed, even experts like Sam have much uncertainty. So this is a process we should all have a voice in shaping.
Sam Altman: “we now see a path… where the superintelligence that emerges is not just the capability of our single biggest neural network, but in… billions of AI-assisted humans” Does he actually think this future is possible? IMO, once recursive self-improvement becomes possible, it’ll be overwhelmingly difficult to avoid convergence towards some single entity. We have no idea how intelligence scales beyond human-level and there’s no reason to expect it to be linear in compute. Small gains in intelligence could translate to large gains in power. It’s a nice vision but would require basically absolute perfection in terms of coordination, governance, monitoring, alignment, and strategic planning. There would be almost no tolerance for error. Also: billions or trillions of human-level AI’s would create strong hendrycks-style selection pressures. But perhaps distributed superintelligence is the golden path. [link]
The core idea of Sam is that superintelligence is possible as a society of billions of people who are assisted by AI
Here are a few key points from the conversation between Marc Benioff and Sam Altman:
- Altman was surprised at how quickly AI capabilities like GPT have progressed and actually worked, exceeding expectations. He attributes this to talented people and an empirical, rigorous approach at OpenAI.
- Altman believes AI will lead to huge amplifications in individual capabilities — one person will be able to do what previously took large teams. This will transform many industries.
- Creativity and hallucination are two sides of the same coin in AI systems. The ability to generate novel ideas and concepts is powerful but needs to be balanced with alignment.
- Altman sees a path for AI and humans working together to provide personalized education at scale, which could greatly improve educational outcomes worldwide.
- The rapid advancement of AI abilities in creative domains like art and music exceeded expectations. Altman sees AI as an amplifier here, not a replacer of humans.
- On policy, Altman is pushing for frameworks to balance risks and benefits, and sees value in establishing a new government agency focused on AI.
“Collaboration allows us to know more than we are capable of knowing by ourselves. It empowers us to think differently, access information we wouldn’t have otherwise, and combine ideas as we work together towards a shared goal.” — –Paul Solarz [link]
Collective intelligence
// in progress…
