Overview
Mitchell, Cohen, et al. (AAAI 2015) — Introduces the Never-Ending Language Learner (NELL), a system that has been learning to read the web 24 hours/day since January 2010. Proposes “never-ending learning” as a paradigm closer to how humans learn: many functions, from diverse self-supervised experience, in a staged curricular fashion.
The Never-Ending Learning Paradigm
Unlike standard ML (one function from one dataset), never-ending learning requires:
- Learning many different types of knowledge/functions
- From years of diverse, mostly self-supervised experience
- In a staged curricular fashion (earlier knowledge enables later learning)
- With self-reflection and ability to formulate new representations to avoid stagnation
NELL’s Achievements
- 80+ million confidence-weighted beliefs (e.g.,
servedWith(tea, biscuits)) - Millions of learned features and parameters for web reading
- Learned to reason over beliefs to infer new ones
- Extended its own ontology by synthesizing new relational predicates
- Running continuously since January 2010
Architecture
NELL is defined as an ordered pair L = (L, C) where L is a set of learning tasks and C is a set of coupling constraints among task solutions. This formalization enables staged, interdependent learning.
Historical Context
Builds on SOAR, ICARUS, PRODIGY, THEO, and Lenat’s AM/Eurisko, but NELL is the first such system allowed to run for a sustained period and demonstrate continuous improvement.
Related
- path-ranking-algorithm — PRA: the random walk inference method that powers NELL’s knowledge base reasoning
- knowledge-vault-probabilistic-fusion — Google’s Knowledge Vault: a related web-scale knowledge system using probabilistic fusion