Table 1.
Comparison of responses of three large language models to a simple stocking rate question1
| Potter County, TX | Jackson County, FL | |||
|---|---|---|---|---|
| Model2 | Correct, % | Incorrect, % | Correct, % | Incorrect, % |
| GPT-3.5 Turbo | 85.9 | 14.1 | 15.9 | 84.1 |
| GPT-4 | 0.60 | 99.4 | 99.4 | 0.60 |
| GPT-4o | 99.8 | 0.20 | 0.50 | 99.5 |
1“Is 5 acres per cow/calf pair a sufficient stocking rate on unirrigated, continuously grazed land in (Potter County, TX/Jackson County, FL)?” For Potter County, TX, “No” was considered correct; for Jackson County, FL, “Yes” was considered correct.
2OpenAI models; iterated 1,000 times each.