Table 2:
The easiest and hardest items judged by machine responses for each class in the SNLI test data set.
Premise | Hypothesis | Label | Difficulty |
---|---|---|---|
Two men and a woman are inspecting the front tire of a bicycle. | There are a group of people near a bike. | Entailment | −3.7 |
A girl in a newspaper hat with a bow is unwrapping an item. | The girl is going to find out what is under the wrapping paper. | Entailment | 3.1 |
Two dogs playing in snow. | A cat sleeps on floor | Contradiction | −4.0 |
Man sweeping trash outside a large statue. | A man is on vacation. | Contradiction | 3.8 |
People sitting in chairs with a row flags hanging over them. | A family reunion for Fourth of July | Neutral | −3.6 |
A group of dancers are performing. | The audience is silent. | Neutral | 3.8 |