Overview
Liang et al. (Northwestern University, Tel-Aviv University, Google; arXiv:1611.00020, 2017) — Introduces Neural Symbolic Machines (NSM): a hybrid architecture combining a neural “programmer” (seq2seq model) with a symbolic “computer” (Lisp interpreter) for semantic parsing against large knowledge bases.
Note: Source filename references “Memory Augmented Policy Optimization” but actual content is the Neural Symbolic Machines paper (known filename-content mismatch).
Architecture: Manager-Programmer-Computer (MPC)
| Component | Role | Implementation |
|---|---|---|
| Manager | Provides weak supervision (answer, not program) | Reward signal |
| Programmer | Maps natural language → program tokens | Seq2seq with attention + key-variable memory |
| Computer | Executes programs against knowledge base | Lisp interpreter with code assistance |
Key Innovations
-
Key-Variable Memory: Augments standard seq2seq with memory to save and reuse intermediate execution results. Novel application of pointer networks to compositional semantics.
-
Search Pruning: The Lisp interpreter executes partial programs and prunes the programmer’s search space by checking syntax and semantics — generalizes weakly supervised semantic parsing.
-
Augmented REINFORCE: Combines REINFORCE with iterative maximum-likelihood training using beam-searched pseudo-gold programs for stability.
Results
State-of-the-art on WebQuestionsSP dataset when trained from question-answer pairs only (weak supervision), without feature engineering or domain-specific knowledge. Significantly closes the gap between weak and full supervision.
Significance
Demonstrates that combining neural flexibility with symbolic precision outperforms either approach alone. The Lisp interpreter provides the exact, scalable operations that neural networks struggle to learn, while the neural programmer provides the language understanding that symbolic systems lack.
Related
- ngram-machines-knowledge-qa — Related knowledge base QA approach (P1 integration)
- r-zero-self-evolving-reasoning — Later work applying REINFORCE-based training to self-evolution