JMIR Publications

ABSTRACT

Background:

Evaluating outcomes of a Clinical and Translational Science Award (CTSA) hub’s clinical and translational research (CTR) training (e.g., KL2 program) requires selecting reliable, accessible, and standardized measures. Since measures of scholarly success usually focus on publication output and extramural funding, CTSA hubs have started to use bibliometrics to evaluate the impact of their supported scholarly activities. However, the evaluation of KL2 programs across CTSAs is limited, and the use of bibliometrics and follow-on funding is minimal.

Objective:

This study sought to evaluate scholarly productivity, impact, and collaboration using bibliometrics and federal follow-on funding of KL2 scholars from three CTSA hubs and define and assess CTR training success indicators.

Methods:

The sample included KL2 scholars from three CTSA institutions (A-C). Bibliometric data for each scholar in the sample were collected from both SciVal and iCite, including scholarly productivity, citation impact, and research collaboration. Three federal follow-on funding measures (at the five-year, eight-year, and overall time point) were collected internally and confirmed by examining NIH RePORTER. Both descriptive and inferential statistical analysis were computed using SPSS to assess bibliometrics results and federal follow-on funding of KL2 scholars.

Results:

A total of 143 KL2 scholars were included in the sample with relatively equal groups across three CTSA institutions (A-C). The included KL2 scholars produced more publications and citation counts at the eight-year than the five-year time point (3.4 vs. 3.75 publications per year on average; 26.16 and 26.44 citations per year respectively). Overall, the KL2 publications from all three institutions were cited twice as much as others in their fields based on NIH Relative Citation Ratio. KL2 scholars published work with researchers from other US institutions over two times (five-year point) or three and a half times (eight-year point) more than others in their research fields. Within five-year and eight-year post-matriculation, 44% (n = 63) and 52% (n = 74) of KL2 scholars achieved federal funding respectively. Institution C's KL2-scholars had a significantly higher citation rate per publication than the other institutions (p < .001). Institution A had a significantly lower rate of nationally field-weighted collaboration compared to the other institutions (p < .001). Institution B Scholars were more likely to have received federal funding than scholars at Institution A or C (p < .001).

Conclusions:

Multi-institutional data showed a high level of scholarly productivity, impact, collaboration, and federal follow-on funding achieved by KL2 scholars. This study provided insights on using bibliometric and federal follow-on funding data to evaluate CTR training success across institutions. CTSA KL2 programs and other CTR career training programs can benefit from these findings in terms of understanding metrics of career success and using that knowledge to develop highly targeted strategies to support early-stage CTR investigators' career development.