What is NLI?
Natural Language Inference (NLI) is the task of determining, given a context, whether a hypothesis is:
- Definitely correct (Entailment); or
- Definitely incorrect (Contradiction); or
- Neither (Neutral).
NLI is one of the most important tasks in natural language processing and natural language understanding.
What did you do?
We collected a new NLI dataset over multiple rounds, where annotators tried to find examples that current state-of-the-art models get wrong. In other words, we asked annotators to be adversarial, and to try to find weaknesses in current models. We then used the newly collected data to train an improved model, and repeated the process. This principle could be applied for many consecutive rounds, continuously finding and repairing weaknesses until, in the limit, we would achieve full natural language understanding :)
Why this demo?
We want to make it possible for others to probe these models in a similar way to our annotators, in order to develop an intuition for where state-of-the-art models fail and succeed on this important task. Currently, the deployed model is BERT, trained on SNLI and MNLI (the model used in round 1 of the paper), but we hope to add the others soon.
How does it work?
You are presented with a context and a target label. Your goal is to find a hypothesis that fools the system into predicting the wrong label (but another person would correctly assign the target label). Once you've entered your hypothesis, you will get feedback on the model's prediction.