Fast Reading Comprehension with ConvNets

Felix Wu (Cornell), Ni Lao, John Blitzer (Google), Guandao Yang, Kilian Q. Weinberger (Cornell). Under review at ICLR 2018 (arXiv:1711.04352v1, November 2017).

Key Contribution

Proposes dilated convolutional architectures as an alternative to recurrent neural networks for machine reading comprehension, achieving comparable accuracy with two orders of magnitude speedup.

Problem

State-of-the-art deep reading comprehension models were dominated by RNNs. Their sequential nature:

• Precludes parallelization within instances
• Becomes a bottleneck for latency-critical deployments
• Especially problematic for longer texts

Approach

Simple dilated convolutional units replace recurrent ones. The convolutional architecture enables parallel processing of text, dramatically reducing inference time while maintaining competitive accuracy on SQuAD and other QA benchmarks.

Significance

Demonstrated that reading comprehension doesn’t require sequential processing — parallel architectures can match quality at fraction of the cost. This insight preceded the Transformer revolution (Attention Is All You Need).

Connection to Vault

Part of the reading comprehension research program that includes Efficient Reading Comprehension (FastFusionNet + Integrated Triaging). Ni Lao’s work at Google bridging knowledge graph methods with NLP.