Neural Program Planner for Structured Predictions

Jacob Biloki (SayMosaic), Chen Liang (Google Brain), Ni Lao (SayMosaic). ICLR 2019 Workshop Paper.

Key Contribution

Applies model-based reinforcement learning to weakly supervised structured prediction. Builds on MAPO (Memory Augmented Policy Optimization, Liang et al. 2018) by adding a planning component.

Approach

• Problem: Given a database table and a question, perform a sequence of computation actions to generate a response with binary success/failure reward
• Key insight: Unlike typical RL settings, the environment dynamics are deterministic and known in structured prediction — this can be exploited by model-based methods
• Neural Program Planner (NPP): Scores candidate programs from a pretrained search policy, deciding which program is most promising by utilizing the space searched by the underlying model
• Result: 47.2% accuracy on WikiTableQuestions (new SOTA at time of publication)

Connection to Vault

Extends the Neural Symbolic Machines (Liang, Berant, Le, Forbus, Lao) approach to semantic parsing. Part of Ni Lao’s research trajectory from graph-based reasoning to neural program synthesis.