Table 2.

Strengths and Shortcomings of HTA Reviews

Strength/Shortcoming	Payers Noting as Strength or Shortcoming (n = 11; %)	Illustrative Payer Comment
Strength
Expertise and credibility of reviewers	100	“The structure of evaluation is as important as the people doing it and how good they are. The field is so new that you want the most experienced and credible people to do it.”
Rigor of scientific evidence review	82	“I think they all do a thoughtful, careful job in their evaluations of science and are pretty thorough organizations in their approaches.”
Whether the methodology is specific to genomics	73	“The methodology here is a very important and evolving question. We are going to need a different framework to really understand those sorts of tests that get at a very different concept of the underlying disease state and how they are being incorporated into care.”
Independence from external influence	36	“You mostly want a group that is outside of the political pressures of the day—whether these are political pressures from insurers or from physicians, manufactures, or patient advocacy groups. They really focus on the science, and to me, that's where it has to start.”
Shortcoming
Availability of reviews
Capacity of HTA organizations (too few reviews are issued on personalized medicine)	64	“These reviews are extremely cumbersome, and their frameworks are not usable for the growing number of tests; they are choking.”
		“Some reports take up to a year […] then we note that multiple groups are working on the same topics.[…] Is there a way to organize that from the national level and get a greater quantity and a better turnaround time? But there are no incentives, and that model remains elusive.”
Reviews are not timely	55	“We usually have to start on that path [evaluating a technology] way before any reviews are out, and we have to look at those evidentiary documents before to try to do our own review and get our own sense.”
		“These groups wait for evidence before they do a review, and then they take a long time to develop a review. I get calls from a provider wanting to know if we're going to cover X, and 'I will tell you in two years when there is enough evidence' is not the answer they're expecting. They want a yes or no. That forces us to do something now when there are evidence gaps or the reviews are not available.”
Reviews from fee-for-service organizations are too costly	45	“The work burden involved in putting together these reviews is enormous. I have to tell you that the cost of it is that I don't think any payer is going to be willing to pay over the long haul, looking at the growing number of genetic tests.”
Review does not adequately incorporate nonclinical factors
Cost-effectiveness	82	“We don't use cost-effectiveness in decisions today, but in the future state, it's critical that we do that and get some agreement on how we measure the value and cost-effectiveness. The groundwork needs to be lain, but it's not there today.”
Adoption and acceptance of a technology in clinical care	45	“Their one big weakness is they deliberately limit themselves to the science; they're very careful about that, but they tend to ignore entirely what I call the market factors. We need to know if a technology has become a common practice and whether we want it or not—this is a part of our decision.”
		“We often get situations when preliminary evidence looks pretty good but there is no power to the study yet, and we make a decision based on giving the benefit of the doubt to a member. So we need to know the situation of patient and provider demand for the test as part of the decision.”
Expert opinion and clinical judgment is not sufficiently taken into account	36	“Basic flaw is: on one hand, let's have evidence and the proof; on the other hand, there are some technologies that show great promise with reduced morbidity that may take 10 years to get the evidence for. Do you not utilize those technologies until you have all the evidence? […] Like clinical judgment doesn't get used in these reviews. That is a missing piece, because I think we do withhold things that may be important to people.”
Care delivery barriers to adoption (eg, lack of infrastructure, logistical challenges, disincentives)	27	“Another missing thing is that there are some practical, logistic things that come up too. For example, only in a few of those warfarin studies did anybody mention that it takes a while to send the blood off and have it analyzed and to get the results back. So it basically puts off initiation of therapy by several days. And there are downsides to that, and I don't think anybody actually talked about it.”
Local regulations and factors	27	“One of the biggest gaps in these reviews is they do not consider the local environment—both academic and clinical—or the politics that have to do both with legislators and the community.”

Abbreviation: HTA, health technology assessment.