Making Pervasive Computing Technology Pervasive for Health & Wellness in Aging

During a research encounter I asked a ninety-year-old retired engineer, Mr. M, who for many years had been volunteering in a longitudinal healthy aging study, “would you use a fitness watch that measured your steps and active time?” He quickly replied, “why would I need that?” And then with a sly smile, added, “I’m still good at math. When I go for my daily walk, I note the time on my old watch and then when I return, I note the time again. A little subtraction tells me that I’ve been out for my usual hour constitutional.”

This brief exchange encapsulates many of the key reasons why the adoption of various contemporary technologies— collectively called pervasive computing technologies (the ecosystem of technologies including sensors, mobile devices, and wireless communications) that harnesses the power of embedding computational capability into everyday life—by the aging population has been slow. This paper examines the challenges in this adoption phenomenon, especially in light of popular interest in harnessing technology in the service of helping to maintain health and wellness.

Why Technology?

Mr. M is very fortunate. He is healthy, active, cognitively intact, self-motivated, and remains independent. Unfortunately, he is not the norm for most of the aging population. For most, as one ages, the ability to remain independent and in particular to age in place or pursue one’s life of choice becomes a risk-laden venture, especially for those age 85 and older, a large and rapidly-growing portion of the population. After age 65, 70% of Americans will need some long-term care services (adult day services, home care, assisted living, nursing home) to maintain independence during the remainder of their lives (U.S. Department of Health and Human Services, 2015; Vassilev, 2015). The amount spent on these services in the United States in 2012 was $219.9 billion (George Washington University, 2012), not including care provided by family or friends on an unpaid basis (often called informal care). Approximately 42 million Americans (mostly women) provided unpaid support to an adult at an estimated economic value of $450 billion (AARP Public Policy Institute, 2011). Considering the growth of the aging population and that even just one major chronic condition such as Alzheimer’s disease is projected to require national care costs of over $1 trillion dollars by 2050, the status quo is not tenable (Kaye, Harrington, & LaPlante, 2010).

Key to addressing these challenges is the ability to provide more effective means of facilitating independence and health for as long as possible. Aside from proposed shifts in policy such as age-friendly work rules or retirement programs, there has been continuing interest in what might be called “out-innovating aging”: developing and applying new technologies to facilitate sustainable wellness and effective interventions when decline occurs. In response to this idea a profusion of technologies and protocols have been introduced and developed over several decades. These technologies take advantage of important developments in sensing and pervasive computing, wearable technologies, mobile and wireless communications, health information technology systems, and “big data” analytics. Notwithstanding this abundance of opportunities, the true value of these approaches has yet to be fully evaluated, developed, or implemented. Despite indications of high promise (Lyons et al., 2015; Merrell & Doarn, 2015; Darkins, Kendall, Edmonson, Young, & Stressel, 2015), the evidence base remains incomplete. As I will develop further, this continues as a key challenge to realizing the potential of technology for the aging.

A Brief History of Technology in Aging

The history of technology as a “fix” for aging’s challenges is relatively short. As noted, the need has grown out of the realization that a transformative response is needed that is equal to the profound impact that the “age-wave” itself will have on our health systems and communities. In this context, the vision of how technology can be transformative has been alluring.

“Rodney selected his primary medical team from a variety of providers by comparing their credentials, performance rankings, and pricing online. Because of the widespread availability and use of reliable information, which has generated increased provider-level competition, the cost of health care has stabilized and in some cases has actually fallen, whereas quality and efficiency have risen. Rodney periodically accesses his multidisciplinary primary medical team using e-mail, video conferencing, and home blood monitoring. He owns his privacy-protected, electronic medical record. He also chose to have a tiny, radio-frequency computer chip implanted in his abdomen that monitors his blood chemistries and blood pressure.”

This description of a fictitious patient, Rodney Rogers, was presented in 2005 as a vision of healthcare in 2015 when then–senate majority leader William Frist delivered the venerable Shattuck Lecture, titled “Health Care in the 21st Century”, to the Massachusetts Medical Society (Frist, 2005). Although much of this now decade-old vision was quite prescient, even with the advent of new incremental technologies (such as smartphone-based apps and wearable devices) much of this vision remains aspirational or simply a series of yet-to-be-met goals.

Dr. Frist’s vision and the general concept of technology as a facilitator toward better health and wellness for the aging has roots that go back more than a decade ago. In 2001 (well before the iPhone, Facebook, or Fitbit were available) the National Research Council contracted with the National Institute on Aging to conduct workshops on applications of technology needs of the aging population. Out of that effort a series of papers were compiled that included an important review authored by Eric Dishman (at Intel at the time) and colleagues titled “Everyday Health: Technology for Adaptive Aging” (National Research Council, 2004). The review pointed out not only the promise but also six key challenges of health-related technologies research: 1) imagination: moving beyond today’s clinical and computing models; 2) identification: finding and prioritizing problems to pursue; 3) iteration: concept testing and refinement; 4) infrastructure: deep dives on enabling technologies; 5) interfaces: exploration of human-machine interaction; and 6) integration: testing whole systems in situ. These challenges began to be tackled with a proliferation of smart home demonstrations and small pilot studies, but there remained a need for more work on integration and scalable, real-world demonstrations of efficacy and effectiveness (National Research Council, 2004).

In the ensuing decade, several major efforts were launched to move in this direction. With National Institute on Aging (NIA) support, the Oregon Roybal Center for Aging & Technology (orcatech.org) established a fully integrated multi-domain home assessment platform, the Life Laboratory, in 2004. A collaboration supported by the Robert Wood Johnson Foundation, National Institutes of Health, National Science Foundation (NSF), and Intel established the Senior Independent Living Research initiative to determine the requirements for a scalable technology platform for independent living research across the United States. Extensive visioning and roadmap initiatives (with evocative acronyms) were also conducted across the European Union, including Ambient Assisted Living Innovation Alliance (AALIANCE), Common Awareness and Knowledge Platform for Studying and Enabling Independent Living (CAPSIL), Extending Professional Active Life (ePAL), Social, Ethical and Privacy Needs in ICT for Older People (SENIOR), and Bridging Research in Aging and ICT Development (BRAID) (Van Den Broek, 2010; Bennis, McGrath, Caulfield, Knapp, & Coghlan, 2010; Camarinha-Matos & Afsarmanesh, 2011; Wright, 2009). These initiatives created roadmaps for research to achieve effective and sustainable solutions to independent living based on in-depth analyses of independent living and information communication technology scenarios for Europe’s growing aging population. Subsequent workgroups (e.g., President’s Council of Advisors on Science and Technology [2014], the Trans-NIH/Interagency Workshop on the Use and Development of Assistive Technology for the Aging Population and People with Chronic Disabilities, and other initiatives like NSF Smart and Connected Homes) provided further grist for building the pathway forward to real-world evidence.

In addition to the profusion of guiding documents and a few focused deployments, there more recently have been important islands of larger real-world efforts, forged to build in situ evidence of the value of some of these technologies. These have largely been within the domain of telecare, telehealth, or telemedicine (each related, but not the same) where the available technologies and response systems have been in development for many decades. Notable among these real-world experiments has been the U.S. Department of Veterans Affairs (VA) and their Care Coordination/Home Telehealth Program, which has enrolled tens of thousands of veterans (Kidholm et al., 2015); the Renewing Health Consortium, deployed across nine European countries (21 telemedicine pilots; 7000 patients; Cartwright et al., 2013; Henderson, Knapp, & Fernandez, 2013); and the Whole System Demonstrator, a randomized, controlled trial of telecare and telehealth completed in the United Kingdom (Bower et al., 2012). These important programs have remotely monitored thousands of patients with multiple chronic medical conditions, facilitated by dedicated home monitoring devices and messaging services supervised by care coordinators. In summary, the results have unfortunately been mixed. This has been borne out by a number of systematic reviews (Wootton, 2012; Black et al., 2011; Car, Huckvale, & Hermens, 2012; Deshpande, Khoja, & McKibbon, 2008; Polisena, Coyle, Coyle, & McGill, 2009; Pope, Rowsell, & O’Cathain, 2011; Vassilev, 2015). Many issues have been identified as contributing to these inconsistent findings: differences in patient populations, deployed technologies, user experience, training, costs, and variable outcome measures and analyses, to name a few.

Where We Stand Today

Despite many potential challenges and shortcomings, continuing interest in technology as a catalytic engine for health care improvement remains strong, fueled by the serious health policy implications of the aging of the population. Continuing rapidly-evolving technical capabilities in mobile and pervasive computing capacity, wireless technologies, and computationally intensive analytics, particularly in non-health care sectors (e.g., consumer electronics, manufacturing, finance) has shown tremendous growth. Today five of the top six most valuable companies are giants of the technology industry: only one was on the list 10 years ago (Microsoft). Technological innovation and growth has spawned more recent major movements that have become the buzz-words of today, with “digital health”, “the internet of things”, and “big data” touted as phenomena that will improve the health and well-being of everyone, including the aging population. As a measure of the unflagging interest in these areas, in just over the past four years, searches on these terms have increased up to three-fold (Google Trends, searched February 14, 2017). Over the next seven years, investment in the health technology sector alone is projected by some consultancy firms to increase exponentially from over $20B to over $150B (Global Market Insights, 2016).

Given the strong continuing interest in technology in general and more specifically for improving health and wellbeing, one might expect rapid digital adoption and dissemination in the health care sector. The field has certainly evolved apace on the technical side. However, with a few exceptions, recent experience has found little in the way of effective wide-spread or deep adoption of many health-related technologies in the day-to-day experience of the senior population or as a means to sustain their independence. Much of the action currently remains in legacy technologies seated in the mainframe health system, such as classic telemedicine and electronic medical data or record systems (EMRs). These technologies have required enormous effort over decades to gain adoption among providers in health systems. In the case of EMRs, billions of dollars were needed in incentive payments to drive wider adoption even in a health system that has in many ways built its reputation on high-tech medicine. And to this day, interoperability and functionality in these systems remain a work in progress.

Outside of the bricks and mortar of hospitals and clinics, within the aging population itself it is hard to find meaningful adoption into the wider community and bring technologies directly into the home. Why might this be? Although the answer is obviously multifaceted, a general case can be made that many of the answers reside in how the technologies have been developed and applied across the diverse health-care sector and then applied to the aging population. This might be considered as a special case of the last mile problem: the challenge of delivering the final to-the-home leg of telecommunications networks and services to aging end-users. In large measure this is not just a question of technical access, such as a lack of an Internet connection, but about user experience and gerontological cultural. With this in mind, it is of great value to consider use cases among the aging.

Technology Use Cases, Use, and Adoption Among the Aging

There are many major use cases. Among broad categories, these include, for example, using technology to monitor a large population’s health (an epidemiology case), employing technologies that facilitate assessment of individuals being monitored and treated for specific health conditions (patient-centered health maintenance case), or using technologies to improve the conduct of clinical trials (intervention research case). Each has tremendous promise. But the approach for each case from a technical and adoption standpoint is often quite different. Further, within a larger use case, approaches much be tailored to more specific needs and outcome measures. There are important differences, for example, in instituting a simple step counter device as a measure of activity verses the technology employed in a fully-integrated remote home health monitoring system where the detail and granularity of possible outputs and usage would be much different. Thus, at the front end of consideration is: what are the desired data and what technologies might be deployed to optimally provide the data in light of the users involved?

The lure of technology solutions must be tempered with perhaps the most crucial of all considerations: will the population of interest accept or engage with the technology? Will they see it as useful or worth paying for? To date, the senior demographic has voted with their fingers, keeping them off most devices when compared to the younger generations. They have not adopted widely to smartphones, smartwatches, fitness bands, or health applications delivered electronically. In 2015 only 55% of the U.S. population over age 65 owned a computer and less than 50% had broadband. Even fewer (30%) in this demographic owned a smartphone. There is a steep drop off in use of information communication technology and devices at increasingly older ages. For example, while 38% of those age 60–69 own a tablet, only 25% of those over age 70 do (Anderson, 2016). Newer devices such as wearables (e.g., smartwatches, fitness bands) overall have very low adoption regardless of age, but the age drop-off is also observed here as well: 11% among those over age 50 possess a wearable as compared to only 3% among those over age 70 (Anderson, 2016). These age considerations are particularly important as these older age groups are more likely to have multiple chronic conditions, need assistance, and live alone.

Reviews of the literature on motivations and cautions about technology adoption among the older population identify several important themes and gaps (Mitzner et al., 2010; Claes, Devriendt, Tournoy, & Milisen, 2015; Peek et al., 2014). Common concerns were device or system complexity, cognitive effort to learn, cost, privacy, obtrusiveness, and forgetting or losing technology. In our own research we found these same common themes with regard to technologies related to aging independently (Wild et al., 2012; Wild, Boise, Lundell, & Foucek, 2008; Boise et al., 2013). Among the most important concerns is privacy and related to this is data security. Only 20% of those over age 50 in the United States are very confident that wireless devices are private and will not be seen by others (Anderson, 2016). Related to this are consistent concerns raised regarding use of video in the home, even if anonymized (Claes et al., 2015; Wild et al., 2008; Demiris, Hensel, Skubic, & Rantz, 2008). Comfort with technology use clearly depends on experience, personal preferences and especially among the aging, cognitive ability (Wild et al., 2012; Czaja et al., 2006).

On the other hand, most found that a technology would be adopted if it is easily usable and facilitates aging in place by promoting independence and safety and lessening burdens on family. In this light, the barrier to realizing the potential of technology to aid good health and independence in aging is first providing convincing evidence that the technology solution works, that it provides an advantage over the status quo, and that it is worth spending money on. Although providing this evidence begins with the end-user, this evidence is also at the same time uniquely important for adoption more widely up the chain of users in health care. Whether clinician, health insurer, regulator, or investor, unlike consumer electronics or more commercial technology use, there is a criterion standard of efficacy and safety that ultimately pushes adoption beyond simple popular public perceptions of desirability.

Creating Evidence to Drive Adoption

Why is it hard to create acceptable evidence for what works and move adoption forward? One answer is that, even with the appropriate focus on the senior user at home or in the community, in the overall health care space there will inevitably be a health professional involved at some point in the decision equation despite the best person-centered, self-directed intentions. These professionals are also crucial to adoption and are also perhaps misunderstood by the technology development community. The current clinical care scenario is built on a legacy of methods deeply rooted in the brief clinician-patient interaction where, even with improvements in patient-centered approaches, the clinician is still the power broker of information and writing of “orders.” In this transaction, the sources of information and, more fundamentally, the methods of data discovery that drive outcomes rely on highly-entrenched point-of-care routines that often yield poor-quality data.

Consider that each day hundreds of thousands of clinicians around the world practice a similar exercise. We conduct interviews in office or clinical settings, trying to get the facts straight about our patient’s latest concern or perform a less acute check-up, reviewing the health history for the past year or more. The time available for this sleuthing is short; the average visit time is about 10–20 minutes in the United States (Tai-Seale, McGuire, & Zhang, 2007). What is the key focus during these precious moments in time? Traditionally, the majority of this time is spent in a special class of data collection that might best be termed data recollection: the reconstruction from memory and pieces of incomplete data the experience and circumstances of events that surround patient complaints or concerns. Based on this brief interview, many critical decisions are made, although the time in discussion of and directions for carrying out these decisions are left to brief periods during the interview or to the waning minutes of the visit. Although as clinicians we are well aware of the limitations of this exercise, especially in light of challenges to meet documentation standards (including data entry into the EMR) and guidelines for care, we have yet to fundamentally change the imbalance in the equation of data collection verses provision of care. Much of the time spent with aging patients or families is not spent in active problem solving with the patient, but in practicing this exercise in medical anthropology, piecing together from inherently incomplete records or the patient’s memory what might be happening so as to inform rational medical decisions.

We know that this exercise is laden with potential error. Examples abound. Thirty per cent of people hospitalized for a major depressive episode did not recall this episode 25 years later and of those who did recall being depressed, half could not provide enough detail to ascertain the severity of the episode (Andrews, Anstey, Brodaty, Issakidis, & Luscombe, 1999). There is a pervasive discordance in symptoms between self-reports and what is recorded in the EMR (Weng, 2017). None of 15 studies of self-reported medication-taking accuracy showed high concordance between interview-based medication-taking reports compared to electronically-verified medication-taking behavior (Garber, Nau, Erickson, Aikens, & Lawrence, 2004). After receiving advice for health behavior modification, less than half of patients recalled that discussion (Flocke & Stange, 2004). Even recall of details of what a person has been doing in their own home in the last two hours is highly inaccurate. About a quarter of cognitively-intact seniors were entirely mistaken as to what they were doing in their own homes when compared to the objective data captured from a sensor system installed in their homes (Wild, Mattek, Austin, & Kaye, 2016).

On top of the fundamental “noise” or inaccuracy in EMR data that all clinicians know exists, there are increasing amounts of practice requirements that demand large amounts of time and attention of the working clinician. If a clinician were to follow national practice guidelines for just 10 common chronic diseases, this goal alone would translate into a 10–11 hour workday.

It should thus not be surprising that the practicing clinician in the community is not eager to lead a charge toward adopting new technologies that, although they may deliver more objective and ecologically valid data direct from the home, are daunting to use in day-to-day practice given current culture. In this milieu, creating the evidence that adoption of new data capture and reporting technology and systems is worth the disruption is a big stretch for many. The evidence needs to be worth changing entrenched practices. It needs to clearly be shown to change the outcomes for patients and families for the better. What is the incremental advantage? What are the effect sizes? In the end this is also what health systems, payers, and investors ask.

What Will it Take?

There are no magic formulas to creating evidence. White papers and technology punditry alone are not adequate. Depending on the use case and the technologies being examined, the types of evidence needed will vary. There are many paths to generating this evidence, ranging across innovative n-of-one studies; large-scale adaptive, randomized, controlled trials; or incisive data mining of massive existing data sets. Importantly, there is no quick fix. It is notable that health systems that have seen system-wide uptake of technology (e.g., the VA or Kaiser Permanente) have mindfully been working in this area for decades. Their success has been facilitated by being strong integrated health systems willing to sustain investment in end-to-end technology and systems.

An additional essential consideration is the need for transparency with regard to aspects of technical development, deployment, and testing. When proprietary systems are employed and details are hidden to protect potential intellectual property, it becomes difficult to replicate outcomes and ensure that evidence presented is valid. This is not to say that commercialization of technology should be discouraged: quite the contrary. However, the validity and trusted value promised ultimately require unbiased assessment. Replication and iterative modification of independently evaluated research is the lifeblood of scientific progress. Lack of transparency in the research and development process holds back timely and effective evolution of innovations and ideas. Ultimately, products or services that don’t work or, worse—especially in the area of health-related solutions—can result in significant harm require a high standard of evidence for adoption.

With these considerations in mind, more emphasis needs to be placed on systematic investigation of technologies for the aging population. Until recently, there was little major or organized national investment in this research. This is changing. A transNIH initiative (which includes five NIH institutes, the office of the director, and the VA) has recently been launched to facilitate the creation of needed evidence specific to aging populations. This initiative is called CART (Collaborative Aging Research using Technology; National Institutes on Aging, 2017). The program is intended to build a scalable and sharable infrastructure for research tuned toward understanding medical illnesses as well as independent living among the aging. It is intended to involve not just highly educated, high socio-economic participants (typical of much prior research), but to specifically ensure inclusion of those of low income and be representative of wide racial and ethnic diversity. The ultimate goal is to create a 10,000 home “life laboratory” of homes across the United States that are available for iterative evidence building.

CART is creating a technology-agnostic platform (i.e., not reliant on any specific proprietary device or system) that can be deployed in multiple homes of seniors that will enable a range of sensors and devices to capture multiple domains of health and wellness data for research (see Figure 1). The development is highly focused on not only individuals or couples aging in place, but also another important group of users: the persons that deploy, operate, analyze, and act on the data. This system will be iteratively developed over several years with a particular eye toward understanding how to be relatively “future-proof,” realizing that technology evolves rapidly over time. Ideally CART will create a community of dedicated research volunteers as well as scientists to more effectively build the evidence for using technology intelligently to assist in healthy aging.

Figure 1.

A pervasive computing home-based system developed for initiation of the Collaborative Aging Research using Technology (CART) program and currently used by the Life Laboratory of the Oregon Center for Aging & Technology. The technology-agnostic (i.e., any standard device or sensor can be incorporated, represented by “Device/Sensor X”), continuous-assessment platform is capable of providing a wide array of data capture inputs: e.g., mobility assessment with passive or wearable sensors, automated physiological monitoring, automated medication adherence tracking, social engagement indicators, driving assessment (via vehicle data port monitoring), and other relevant outputs. Data is sent securely to a central server for system provenance monitoring, data annotation, curation, analytics, and storage. Representative cohorts or studies (A, B, C, etc.) are listed on the left to indicate that any number of studies or projects may use the system.

The CART platform and research community that will grow around it will take time to reach scale. While this activity grows, there will be other fronts that will aid in the building of evidence for use of health-related technology within the aging population. Among the most promising fronts that may have a deep and long-lasting influence on both the practicing clinical community and aging patients is the implementation of pervasive computing technology in clinical trials. This is not simply advancing electronic data entry, utilizing online patient registries, or improving information reporting systems. These important improvements are underway. But the transformative opportunity for Pharma is to fundamentally change the conduct of research so that treatments may be more cost-effectively and efficiently developed. Billions of dollars are spent advancing therapies based on uncertain data or surrogate markers not clearly related to meaningful functional outcomes. The predictable result is many failed trials. A major reason for this is that, just as the clinic visit relies on episodic, brief in-person visits with much highly variable self-report data collection, so too do the clinical trial study visits. This is particularly important for treatments developed for the aging population where the most meaningful outcomes are improvements in function (e.g., cognition, mood, mobility, sleep, etc.).

Pervasive computing technologies embedded in the everyday lives of seniors in clinical trials provide the opportunity to inform trial outcomes with objective, continuous, ecologically valid, and immediately tangible evidence of efficacy. Thus, for example, instead of individuals trying to report on salient nighttime behaviors or mobility during many months where day-to-day and week-to-week change is hard to personally recall, the technology can provide this objective data across multiple domains (e.g., sleep metrics, trips to the bathroom, room transition times, interactions with spouses, etc.) directly with a precise time resolution that is humanly impossible to self report. The frequency and fidelity of the data provides the opportunity to dramatically reduce the sample sizes needed to conduct trials, thus also reducing the number of older volunteers who may be exposed to unforeseen treatment side effects (Dodge et al., 2015). Further, because of the remote sensing and assessment design of the systems, the older chronic disease population, who may be more home-bound or have limited transportation options, can be more readily included into trials, making the evidence base more representative of the real world. Finally, once the system is deployed in a home, with consent it can continue to report data long-term, long after a trial has reached its primary endpoint, thus providing insight into even post-marketing periods.

Pharma is aware of this potential, although there is still much work to be done (Shaywitz, 2015; Hird, Ghosh, & Kitano, 2016). Because the clinical trial paradigm for generating evidence is highly structured, the use of pervasive computing technology in trials may become a major propellant for the use of pervasive technology more broadly in research, and then in various health care settings as well. If the majority of trials begin to adopt these methodologies as accepted best practice, one can envision a penumbra effect of greater acceptance and desire for these more objective and time-sensitive data pervading clinical practice.

Conclusion

In conclusion, there is acknowledged apprehension about meeting the challenges of our aging population. This is especially true with regard to the potential social and economic costs of maintaining health, independence, and a high quality of life in the face of chronic disease and specific age-associated conditions. Innovation is welcome and needed; the status quo won’t do. In this context, the use of pervasive computing technology writ large has the potential to make a major impact in the provision of assessment and assistance to the aging population. However, despite some exceptions, its uptake more widely has been relatively slow. This has been the result of a number of forces. I suggest that the under-appreciation of the challenges of the major users of these technologies—the older person and the professional clinical provider community—has contributed greatly to this lack of advancement. Further, in all cases effective adoption and utilization of these innovations will require the generation of believable evidence of what truly works for these key constituencies. The path forward for this transformation is likely to require “old fashioned” research studies facilitated by commitments from the major funders of biomedical research: federal agencies as well as industry. In particular, pharma and related industries may lead this charge, as they have much to immediately gain and because the evidence standards required in this sector will generate exemplary use cases that may then go on to become increasingly adopted in general practice.

There are no magic formulas to creating evidence. White papers and technology punditry alone are not adequate. Depending on the use case and the technologies being examined, the types of evidence needed will vary. There are many paths to generating this evidence, ranging across innovative n-of-one studies; large-scale adaptive, randomized, controlled trials; or incisive data mining of massive existing data sets.

In this context, the use of pervasive computing technology writ large has the potential to make a major impact in the provision of assessment and assistance to the aging population. However, despite some exceptions, its uptake more widely has been relatively slow. This has been the result of a number of forces. I suggest that the under-appreciation of the challenges of the major users of these technologies – the older person and the professional clinical provider community –has contributed greatly to this lack of advancement. Further, in all cases effective adoption and utilization of these innovations will require the generation of believable evidence of what truly works for these key constituencies.