Figure 8.

ROC curves for second (capacity) qualifiers. The areas under the ROC curves for the expert, non-experts, and NLP system were 0.7230, 0.7636, and 0.5156, respectively. The graph shows that the NLP system's coding assignments for the second qualifier agreed with the reference standard more often than they disagreed, but only slightly more than would be expected through chance alone. This may be partly because assignment of a capacity qualifier is highly subjective; it involves making predictions about an individual's future capacity. Future capacity, in turn, can be determined by any of a large number of patient-specific factors.