Assessing clinical researchers' information needs to create responsive portals and tools: My Research Assistant (MyRA) at the University of Utah: a case study

Abstract

Question:

How can health sciences librarians and biomedical informaticians offer relevant support to Clinical and Translational Science Award (CTSA) personnel?

Setting:

The Spencer S. Eccles Health Sciences Library and the associate vice president for information technology for the health sciences office at the University of Utah conducted a needs assessment.

Methods:

Faculty and staff from these two units, with the services of a consultant and other CTSA partners, employed a survey, focus groups, interviews, and committee discussions. An information portal was created to meet identified needs.

Results:

A directive white paper was created. The process employed to plan a virtual and physical collaborative, collegial space for clinical researchers at the university and its three inter-institutional CTSA partners is described.

Conclusion:

The university's model can assist other librarians and informaticians with how to become part of a CTSA-focused infrastructure for clinical and translational research and serve researchers in general.

STATEMENT OF PROBLEM

Identifying the information needs of clinical researchers is a foundational step for meeting the challenge of the Clinical and Translational Science Award (CTSA) program of the National Institutes of Health (NIH). The CTSA program was founded to enhance the efficiency and quality of translational research and provides funding specifically to motivate researchers to collaborate across disciplines for the good of human health. To provide relevant support to CTSA personnel and biomedical researchers at the University of Utah (U of UT), faculty of the Spencer S. Eccles Health Sciences Library and the associate vice president for information technology for the health sciences office (AVP ITHS) conducted an extensive needs assessment with a resulting directive white paper. The following case study describes the process used to plan both a virtual Internet portal as well as a physical collaborative space for the U of UT and three inter-institutional CTSA partners. U of UT's model and experience can assist other librarians and informaticians with how to become part of a CTSA-focused infrastructure for clinical and translational research and serve their researchers in general.

THE UNIVERSITY OF UTAH'S CLINICAL AND TRANSLATIONAL SCIENCE AWARD

The U of UT's Center for Clinical and Translational Science (CCTS) received its CTSA in 2008. The CCTS is a close-knit scientific community that includes basic and clinical researchers, strong genetics and biomedical informatics departments, and a wealth of genealogical information in the Utah Population Database (UPDB). The CTSA includes three other partner institutions: the Department of Veterans Affairs (VA) Salt Lake City Health Care System, Intermountain Healthcare, and the Utah Department of Health.

The CTSA grant proposal envisioned a statewide web portal that would act as an access point for researchers, clinicians, community providers, patients, and other CTSAs to discover information about CTSA programs and opportunities, institutional resources, and each other. The portal would be the electronic entréz to the Federated Utah Research and Translational Health eRepository (FURTHeR) and other research resources supported by core facilities. FURTHeR is a standardized query engine for accessing the rich clinical, research, and public health data from the U of UT and its partner institutions. The web portal, which would collect information already available to researchers, would also link to other research resources supported by the U of UT, including related staff. The portal would be called My Research Assistant (MyRA).

Though the collaborative portal was not funded, the AVP ITHS did not lose sight of the need for a portal. She met with the director of the Eccles Library, and they assembled a committee that would determine researcher needs and content for a portal (the MyRA Committee). This inter-institutional committee, cochaired by the Eccles Library director and a biomedical informatics professor, met frequently for two years to discuss its collective vision for MyRA and identify key and relevant content as well as targeted audiences. Other members of the MyRA Committee included a CTSA community engagement core codirector, an academic librarian, representatives from Intermountain Healthcare (librarian and research office administrator), an informatician with a joint appointment with the VA hospital, a CCTS data analyst, informatics software engineers, and a MyRA consultant.

Through AVP ITHS office funding, a contractor, Margaret Reich Consulting, was hired to conduct a needs assessment to determine what researchers would find useful or desirable and to inventory what products and resources (especially social media tools) MyRA could offer. All of these findings were consolidated into a recommendation white paper to assist in MyRA's development. The MyRA mission and vision statements can be found in Figure 1.

Figure 1.

MyRA mission and vision statements

A librarian conducted a literature review to determine if any web portals similar to MyRA had been developed and how the needs for such portals were assessed. PRIMER and StarBRITE are two such portals, built for the research communities at Duke University 1 and Vanderbilt University 2, respectively. Both portals are similar to MyRA in that their development was related to CTSA funding. Several other articles discuss using similar researcher needs assessment methods as those employed by the MyRA development team 3–5.

NEEDS ASSESSMENT METHODOLOGY

The needs assessment methodology consisted of four phases: (1) assessment of tools used nationally; (2) local qualitative work, including interviews and focus groups; (3) a quantitative survey; and (4) a strategic planning retreat for stakeholder group discussion.

Tool assessment overview

In preparation for querying U of UT researchers about their needs, the consultant researched what tools were already available and being used by scientists, including those created or provided by other CTSA awardees.

1. A list of tools created or used by other CTSA awardees was created using the CTSA Biomedical Informatics Resource Inventory (now maintained at NIH and called the Clinical and Translational Resource Explorer) and by searching these awardees' CTSA-related websites. A group of national evaluators of the first cohort of CTSA awardees conducted a CTSA-wide survey to identify the tools being used and the needs of scientists for conducting their work. Tools were categorized into the following groups: research/collaborative, educational, administrative, and community outreach. The list, along with tool descriptions (Table 1, online only) was made available to the MyRA Committee for discussion and review.

2. A list of social networking and collaborative tools developed for and used by research scientists was created (Table 2, online only) via searching websites and science blogs and through personal communication with science dissemination experts. This list was divided into the following categories: networking, publication sharing and organizing, data sharing, document collaboration, clinical trials, and new search engines. Again, this list was made available to the MyRA Committee for discussion and review.

Qualitative assessments

Focus groups

Two focus groups (n = 18) were conducted by members of the MyRA Committee. The focus groups consisted of fellows, postdocs, K30 Clinical Research Curriculum Awardees <http://www.grants.nih.gov/training/k30.htm>, and master's of clinical investigation students. The focus was to assess the quality of the clinical scientist training program including the degree of research support, quality of mentoring, breadth of the curriculum, and appropriateness of committee support for conducting research.

Interviews

Twenty individual semi-structured interviews were conducted as part of the needs assessment process. Eight were conducted by the evaluation team chaired by Charlene Weir, the evaluation expert for the CTSA. Four focused on junior faculty (at the end of their first year and repeated at the end of their second year) and four on CCTS faculty who varied in their research experience. In all cases, the focus of the interviews was on what kind of support was needed to conduct research, what tools would enhance collaboration, and what would improve mentorship.

Another group of interviews was conducted by the consultant and a librarian and included thirteen short, focused interview questions (Figure 2). Interviewees were suggested by members of the MyRA Committee and included researchers and CCTS core directors. The subjects were familiar with the CCTS and were asked to give some thought as to what they would like to see offered as assistance to researchers. Interviewees were sent an early version of the MyRA mission and vision statements (Figure 1) to review before the interviews.

Figure 2.

Researcher interview and focus group questions

Quantitative survey

An annual survey developed by the CCTS evaluation team was created to measure awareness and satisfaction with the functions of the CCTS (Appendix, online only). The survey was sent to faculty and fellows (including affiliated faculty from the VA and Intermountain Healthcare) in the U of UT Health Sciences (n = 2,032, 133 respondents, 7% response rate). Included were faculty who had published at least 2 articles within the prior 2 years, all junior faculty (arriving within the prior academic year), and all fellows. As this was the first annual survey, the results are considered baseline data. Although 2 reminders were sent out, the response rate was very low. An analysis of responders versus nonresponders indicated that many of the nonresponders did not engage in research directly and, because it was the first year, few knew the reason for the survey. As a result, the results were considered suggestive, but not representative. Subsequent annual surveys were sent to those faculty and fellows more directly engaged in translation research and/or affiliated with the CCTS.

Strategic planning retreat

A strategic planning retreat held in January 2010 was attended by about thirty stakeholders, including technology experts, the consultant, and the MyRA Committee. Several exercises and presentations were scheduled and were designed to inform the planning decisions. Each is discussed below.

Persona exercise

An exercise was devised for the MyRA strategic planning retreat. Each attendee was asked to take on a specific “persona” and describe the informational and research support needs from each persona's point of view. The personas were: student, fellow, junior researcher, senior researcher, statistician, librarian, and community member. Persona needs were recorded on flipcharts labeled with the following research process categories: hypothesis generation, protocol development, approval, data gathering, data analysis, publication, and general.

MyRA's supporting technology

The mechanics of building the supporting technology were discussed. Resource limitations were identified, including staff, constantly changing university platforms, and technological support. An evolutionary approach was identified as the best solution, in that initially, MyRA would be constructed to be a simple, information-based website, with future development enabling more sophisticated functionality as additional funding was obtained.

Review of other university resources

The U of UT and its health sciences already had in place a diverse set of resources to support collaborative, translational research, but the resources were scattered and not integrated. The U of UT's Clinical Research Informatics Systems Plan (CRISP) project results were shared. This presentation included a research process flowchart that outlined all stages of research, from idea to grant submission, summarized in Figure 3. The presentation highlighted the need for an informational website about project scheduling tools, research, and grant application procedures.

Figure 3.

Research life cycle

Other university resources that were reviewed included UNITE, REDCap, and institutional review board (IRB) educational classes. UNITE, a tool released by the U of UT in February 2010, uses Vignette/Open Text software to assist collaboration, with links to investigators' calendars, email, and wikis. Researchers are able to create and maintain private and team work spaces and communities and can collaborate on document authoring.

RESULTS

The consultant and the MyRA Committee qualitatively analyzed the answers to the interview questions, the focus groups, the CCTS survey (Table 3), and the needs recorded during the strategic planning persona retreat (Table 4, online only). The needs expressed at the retreat varied according to the type of person conducting research, with more support for the entire research process being requested by junior researchers. Senior researchers wanted more assistance with grant and contract post-award administration. The process was iterative and consisted of repeated discussions with the goal of integrating the various sources of information in order to guide recommendations. The themes that emerged are presented below. The key theme that emerged from all of these data sources was the overall lack of awareness of the services that the CCTS offered. The results associated with information assistance fell into the following categories: data, grant writing, IRB navigation, statistical and technical resources, collaboration, and physical space and staff.

Table 3.

List of needs from interviews, focus groups, and Center for Clinical and Translational Science (CCTS) survey

Lack of awareness of the Center for Clinical and Translational Science

The results from the CCTS survey and the interviews indicated an overall lack of awareness of the CCTS, its goals, and its resources. The average awareness of the CCTS components from the survey respondents ranged from 12%–34%. The educational component had the highest awareness and the informatics component the lowest. In the interviews, participants expressed confusion about the role of the CCTS as well as confusion about where to go to get research help and how to navigate the maze of existing sources.

Easily accessible data

The accessibility of data was the source of most frustration for senior researchers, and the topic surfaced frequently in the open comments from the CCTS survey. The respondents expressed an appreciation for the wealth of clinical data that exists at the partner institutions, but their inability to effectively access that data was frustrating. The lack of standard fields, coding, and search queries made it nearly impossible to run replicable experiments.

Confusion about the grant writing and institutional review board processes

Another frequent source of frustration in respondents' comments was the lack of centralized, clear information about the process of grant application, project management, and related IRB procedures. Concomitant with this frustration were comments about the lack of administrative support for grant writing and management, which has become extraordinarily complicated at every level, particularly with funding agencies.

Some departments and individual labs have been successful in building their own infrastructure for grant preparation and management. The benefit of interviewing the more senior scientists was that they could recognize and speak to the frustrations of their less experienced colleagues and suggest resources that would be useful. Some of these senior scientists expressed a willingness to offer their knowledge and services to CCTS/MyRA.

Statistical and technical resources

A sense of confusion about the availability of statistical and design resources was salient. Participants lacked an understanding about how to access and use statistical resources and the extent of such resources. This lack of understanding came through especially in the persona exercise done at the retreat.

Desire for efficient collaboration

When taken in the larger context of researchers finding like-minded researchers, this topic came up as a need more readily and frequently for the junior personas in the persona exercises and in the social media focus groups. A great desire for finding effective mentors was expressed for both academic and career balancing issues. The senior researchers interviewed did not mention mentoring as a need. There were fewer positive remarks about social networking tools, although two interviewees wanted to see a sophisticated search tool that would know their interests and push appropriate information about other researchers and opportunities to them. Many others were not comfortable or familiar with these tools. There seemed to be a particular aversion to creating another researcher database, because interviewees found the U of UT's existing systems onerous to use. Some wondered why such a database could not be auto-populated with already available data (e.g., lists of publications and funded grants).

MyRA purpose

Although MyRA was initially conceived as a simple web portal, the current needs assessment expanded and reinforced a design where investigators could find everything they need in one place (including committee minutes, group documents, reports, etc.) in a clearly organized way and where other important sites could be easily identified and accessed. The U of UT already has a wealth of informational sites that describe offered resources as do its partners. There are also outside resources that should be linked to (e.g., NIH, Grants.gov). MyRA would be the one place or clearinghouse pointing to useful information in an organized, easy-to-find manner. Table 5 (online only) is a list of informational sites that were identified.

Content identification

There was consensus that the processes identified in the shortened CRISP flowchart (Figure 3) should be used on the MyRA website, with links providing relevant information about each stage of the process (including information unique to each CTSA partner). There should also be an alphabetical hierarchical list of links to information, so that visitors to the site have a choice about how to browse or search.

Content about the research process that was identified to be part of MyRA included how to:

• develop a research project

• find forms and information about procedures for submitting grant applications

• find information about applying for and getting IRB approval

• find funding opportunities and collaborators

• get access to, and help with, clinical data

• get information about clinical trials

• reach out to the community

• find researcher orientation information

• get help with statistics, measurement methods, and research design

• find ongoing training and other educational opportunities both in or outside of the U of UT

The interactivity of MyRA was also addressed. Members of the stakeholder team thought it should be easy to submit research-related questions, needs, or suggestions, both formally (as in asking for a statistical consultation) and informally (as in a quick request for a location of services). A frequently asked questions (FAQ) list was considered important.

Tracking and monitoring

The ability to track usage, monitor satisfaction, and continually improve MyRA was identified as important. Functionality to gather that information was discussed.

Physical space and staff

The needs assessment findings also strongly recommended employing a “research concierge,” a staff member whose job it is to help investigators and others find what they need. This position is widely used in other CTSA organizations.

A physical office for MyRA had always been part of its vision, and a majority of interviewees agreed that the office was an excellent idea. Most also agreed it would be best housed in the Eccles Library, as it is a neutral governing entity, it is centrally located on the U of UT Health Sciences Campus, and people think of it as a central meeting place.

DISCUSSION

The needs assessment described above illustrates the complexity of building an information source that meets the needs of a diverse set of stakeholders and users in two basic ways. First, the research process is complex and not particularly linear, and the developmental process of gaining research expertise is long and time consuming. The result is that information needs about conducting research are highly contextually dependent. Any information source and support needs to have multiple layers, ranging from an available list to browse, “decision support” that guides the asking of appropriate questions, and the inclusion of a real person to maximize the efficiency and the quality of the experience.

The second major issue was the existence of multiple websites, information support, and educational activities across the university. Although there was a large number, they were not integrated and were often duplicative, each with their own websites and contact information. The result is that users were confused and frustrated.

Overall, the findings of the needs assessment laid the groundwork for the development of a comprehensive information site called MyRA and physical research support space in the Eccles Library that hosts a research concierge: a librarian who connects those with specific research information needs with experts who can provide direct assistance and who is responsible for further MyRA portal development and maintenance. He is working closely with the Offices of Sponsored Programs and Research to integrate their research support content and regulations.

The needs assessment was a complex process, involving individuals with a wide range of expertise. Librarians and informaticians played a central role in identifying specific information needs of the CTSA researchers and with shaping the MyRA concept. The Eccles Library director cochaired the MyRA Committee with an informatics professor, while two other librarians served as members (one from the U of UT academic library and one from Intermountain Healthcare). The library director worked with the Office of Research to help gather content from the university community and hired and guided the consultant. She also helped to perform the researcher interviews and cochaired the planning retreat. She helped to guide the development of the MyRA templates that were funded by the National Network of Libraries of Medicine, MidContinental Region. She also worked closely with the informatics data analyst and with software engineers to create the various versions of the MyRA portal. A librarian continues to manage the adoption of the templates by others.

Biomedical informatics software engineers and professors determined and developed the system infrastructures for various versions of the MyRA portal. They also hired and guided the work of a graphic artist to create a MyRA brand as well as a dynamic and attractive site. A professor cochaired the MyRA Committee and retreat, while others from the AVP ITHS office served as members. Two professors have guided the portal development. A professor developed, conducted, and evaluated the CCTS survey and assisted with planning the retreat. The AVP ITHS also provided a portion of the funding and offered guidance for developing the MyRA portal.

Word about MyRA has spread rapidly throughout the U of UT, further illustrating the need for a collective research support entity. Data on usage are being monitored (average 700 hits per quarter, with 40% being new users), and content continues to be added and reorganized for clarity of discovery, to replicate researchers' cognitive frameworks and information needs, and to reflect the research workflow.

CONCLUSION

Designing and implementing an integrated information environment to support translational research requires the collaboration of both librarians and informaticians. Integration and support for all phases of the research process are necessary in order to be able to provide support for clinical researchers' information needs. Although an integrated web portal is necessary, an actual physical space with access to in-person assistance is also required. The library is seen as the hub for information flow and the natural leader to guide both efforts as well as to ensure integration. A basic MyRA portal has been available since the middle of 2011, with a more sophisticated version (using a combination of Drupal and Alfresco) offered as of early January 2012. Templates for both portal site styles—basic hypertext markup language (HTML) and the Drupal/Alfresco versions—were released to the public in mid-September 2011, as they were developed with National Network of Library of Medicine funding. For information on how to implement these templates locally, refer to https://www.sites.google.com/site/myrawebproject/. A librarian research concierge has also been hired as of May 2012, with one-time funding from the School of Medicine's research office.

Experiences gained by conducting the needs assessment at the U of UT offer valuable lessons to others wishing to lend support to researchers. A major lesson is that needs assessments take time, especially if a mixture of methods is employed. Interviews are particularly time consuming as researchers are busy and often interviews have to be rescheduled. Another major lesson was trying to quantify researchers' knowledge of the social media resources that are available to them. Many stated they did not need such resources, but upon receiving more detail about perceived applications, researchers recognized potential applications. Yet another lesson is that while a portal and physical support center are now available to researchers, unless they are made aware of these resources, they will not use them. Promotion of the support and assistance that such tools and spaces provide for researchers needs to be done frequently and via as many communication venues as possible.

As illustrated in this article, librarians and informaticians are taking a central role in developing useful research support tools and portals. Their natural support roles are enabling researchers to find relevant and timely information when and where they need it thus enabling them to spend their time conducting valuable research.