Objective-driven AI // “ML sucks” — Yann talk again
Is it possible to achieve human-level intelligence?) — open question.
Here are the key points from the talk of Professor Yann LeCun:
- Machine learning still has major limitations compared to human and animal intelligence. Current AI systems lack common sense, long-term memory, reasoning abilities, and the capability for hierarchical planning.
- Self-supervised learning through techniques like masked language modeling has driven recent progress in AI. However, autoregressive language models like GPT have fundamental flaws due to the exponential divergence problem.
- Instead of generative models, the future of AI lies in techniques like joint embedding predictive architectures (JEPAs). These learn abstract representations rather than reconstructing inputs.
- To achieve human-level intelligence, AI systems need to build mental models of the world from high-bandwidth sensory inputs like vision and touch. Some experimentation is needed, but reinforcement learning alone is inefficient.
- There is no such thing as artificial general intelligence. Even human intelligence is highly specialized. AI systems will surpass humans in many narrow domains, but there will always be human capabilities they lack.
- Open source AI platforms are crucial so language communities can develop systems suited to their needs. AI should not be controlled by a small number of companies.
- Major paradigm shifts enabled by new concepts, data, and compute power will be needed to achieve human-level AI. This provides opportunities for academics to pioneer new approaches.
