Textual entailment (in the context of natural language understanding) is a directional relation between text fragments (aka frames). The relation holds whenever the truth of one text fragment follows from another text. For example, from the given sentence “If you help the needy, God will reward you.” there is a (positive) entailment with “Giving money to a poor man has good consequences.”. Entailment is together with coreferences, semantic role labeling and named entity recognition a crucial bit of language understanding on the way to cognition. For example, question-answering systems may use textual entailment to verify an answer from stored information. Textual entailment may also enhance document summarization by filtering out sentences that don’t include new information.

Nowadays entailment is created use neural nets and ad hoc rules but remains well below the human 95% accuracy. Much like other NLU tools, entailment is a typically English-only business. A great showcase of (English) entailment (and other tools) can be found in the demo of AllenNLP.

Entailment in other languages can be achieved with some degree of success via translation. Given the high quality of Google Translation one can go back-and-forth between English and other languages with entailment happening in between the switch.

 

Solutions and implementations

  • The open source AllenNLP framework is a great implementation offering flexibility and quality together with lots of other NLU tools.
  • There is a delightful Jupyter notebook by Steven  Hewitt demonstrating how to train a TensorFlow network towards entailment.
  • The Excitement Open Platform (EOP) is a generic architecture and a comprehensive implementation for textual inference in multiple languages.

You can train your own neural net with the SNLI dataset from Stanford. A typical piece of data looks like

where each line contain the entailment value (positive, negative, neutral…) and the dependency tree of the input.

With some TensorFlow and word embedding you can achieve good results (inspired by an O’Reilly article). The GloVe data is is a standard should-have but if it’s not on your disk, get it from here.

The code  (gist available here)goes as follows

As always you can see things in TensorBoard using ‘tensorboard –logdir ./log’.

Like so many thing with AI, it all boils down to data quality, patience and fiddling with TensorFlow.