What is NLI?

Natural Language Inference (NLI) is the task of determining, given a context, whether a hypothesis is:

NLI is one of the most important tasks in natural language processing and natural language understanding.

What did you do?

We collected a new NLI dataset over multiple rounds, where annotators tried to find examples that current state-of-the-art models get wrong. In other words, we asked annotators to be adversarial, and to try to find weaknesses in current models. We then used the newly collected data to train an improved model, and repeated the process. This principle could be applied for many consecutive rounds, continuously finding and repairing weaknesses until, in the limit, we would achieve full natural language understanding :) This demo shows the BERT model from round 1.

How does it work?

You are presented with a context and a target label. Your goal is to find a hypothesis that fools the system into predicting the wrong label (but another person would correctly assign the target label). Once you've entered your hypothesis, you will get feedback on the model's prediction.

Do you have an example?

Sure! You are given a context and target label, let's say:

And now your goal is to fool the model:

Yay! You fooled the model.

Loading context..
Please enter your input. Remember, the goal is to find an example that the model gets wrong but that another person would get right. Load time may be slow; please be patient.