No intelligence can be even close to general // Yann LeCun
Intelligence is a phenomenon that actually nobody knows what it is and how it works.
Every intelligence is specialized, including human intelligence.
Intelligence is a collection of skills and an ability to acquire new ones quickly.
It cannot be measured with a scalar quantity.
No intelligence can be even close to general,
which is why the phrase “Artificial General Intelligence” makes no sense.
There is no question that machines will eventually equal and surpass human intelligence in all domains. But even those systems will not have “general” intelligence, for any reasonable definition of the word general.
AI, AGI, etc. — is just marketing, nothing more. There are many questions in what domains and what machines can do. Computers are just tools, they have no will of their own. A some model of will is possible to simulate, but it’ll be just a model. Truly autonomous systems and self-aware systems are projects of the very distant future.
Finite state machines are working right now everywhere, but they are not autonomous.
How could machines learn as efficiently as humans and animals? How could machines learn to reason and plan? How could machines learn representations of percepts and action plans at multiple levels of abstraction, enabling them to reason, predict, and plan at multiple time horizons? This position paper proposes an architecture and training paradigms with which to construct autonomous intelligent agents. It combines concepts such as configurable predictive world model, behavior driven through intrinsic motivation, and hierarchical joint embedding architectures trained with self-supervised learning.
Here are a few key points:
- The paper proposes an architecture for autonomous intelligent agents that includes modules for perception, world modeling, cost evaluation, short-term memory, acting, and configuration.
- A key component is the world model, which learns to predict future states of the world. This allows the agent to simulate and evaluate different courses of action. The world model uses a hierarchy of joint embedding architectures (JEPAs) to make multi-level predictions.
- JEPAs make predictions in a learned representation space rather than predicting every detail. This allows ignoring unpredictable or irrelevant details. JEPAs can represent uncertainty and multi-modality through encoder invariance and latent variables.
- The world model and other components like the cost function are trained with a regularization approach rather than contrastive methods. This avoids issues like the curse of dimensionality that affect contrastive methods.
- The cost module drives the agent’s behavior and incorporates intrinsic costs (for basic drives) and a trainable critic that predicts future costs. The actor module searches for action sequences that minimize predicted cost.
- The configurator oversees the system and adjusts components for the task at hand. This allows sharing knowledge across tasks by reconfiguring a general world model.
- Hierarchical planning is enabled by multi-level predictions and cost functions. High-level plans are refined into lower-level actions.
- The architecture is proposed as a path towards autonomous machine intelligence, common sense, and more human-like learning. Connections are drawn to concepts in cognitive science and neuroscience.
