Modeling patient recruitment
Feasibility is an increasingly important step in defining optimal site and country mix, as well as for generating a patient recruitment plan for a clinical study. With enrollment delays continuing to impact on clinical trial delivery1, there is room for improvement and a need for more accurate recruitment projections to improve the reliability of up-front planning. We examine some of the principles and practices behind the collection and analysis of robust feasibility data.
Patient accrual is a function of the total accumulated recruitment time (the sum of the country/site-specific active recruitment period across all sites) and the rate at which each site recruits patients. Recruitment planning must take into account the influences on both parameters to improve predictability (Table 1) and to incorporate:
– A modeling tool that considers the influences on the recruitment period
– Evaluation of how the recruitment rate has been defined.
The modeling tool needs to take into account anything that will impact the available total accumulated recruitment time, such as:
– Country-specific start-up times
– The predicted behavior of sites when they start to recruit
– Country-specific holidays
– Country-specific recruitment / screen failure rates
When these are built into the recruitment planning tool, even a simple liner formula can be used for recruitment planning2.
Defining recruitment rates
The best recruitment estimate will be when exactly the same study is conducted again at the same sites. However, there will always be differences between one study and the next that need to be considered. Estimated recruitment rates can be derived from a variety of sources:
– Evaluation of past performance metrics from your own database
– Analysis of published studies
– Therapeutic expert knowledge to identify protocol nuances and their impact on recruitment
– Local expert analysis on the local acceptance of the protocol
– Epidemiological data
– Site surveys to collect feasibility data
It is also important to consider the impact of criteria or procedures on patients’ willingness to participate and/or investigators motivation to recruit patients.
Use of site surveys to
collect feasibility data
Investigators are often asked to provide recruitment projections at a very early stage in the planning process, and without access to the full protocol3. Therefore, care has to be taken in how data is collected, analysed and interpreted, and any assessor has to be aware of potential investigator overestimation. When investigators assess their recruitment rate, there is general acceptance that this will need to be reduced.
The questions asked will drive predictability. When Investigators were asked the two to three questions shown in Table 2, the feasibility results were on average eight times as high as the actual study results. However, there was considerable variability from country to country, with the ratio of planned versus actual recruitment ranging from 0.9 (Portugal) to 28 (Italy). This data was from four protocols, 23 countries, 145 feasibility sites and 395 study sites.
By adopting a more specific approach – drilling down through the key entry criteria and asking five patient population specific questions designed to assess the specific patient population – the variability improved considerably, giving more confidence in the reproducibility of the data generated by the approach presented here.
These feasibility results were consistently one to four times as high as the actual study results. The country specific ratio between planned and actual has been used to define the country specific “discount factors”, for example the percentage by which the investigator estimate has to be reduced to give an
estimate of the projected recruitment
Country specific discount
One approach is to apply a reduction of 50% (discount factor of 2) to investigator estimates4,5. In order to use country-specific discount factors effectively, they have to be fully calibrated for the feasibility approach and derived for each country. When this has been done, they can provide direction on the potential recruitment rate, taking into account therapeutic variation6.
Discount factors have been defined for the drill down approach (12 studies, 35 countries, 963 feasibility investigators, 550 study investigators). These were equivalent to 2 in some countries. But investigators in the US and many Western European countries tended to further overestimate their projections and required a higher discount factor of 2.3 to 3, while the Eastern European and Latin American Investigators tended to be more realistic with their estimates, requiring a factor of 1 to 1.5. These factors can be used to compensate for investigator overestimates.
Planned vs. actual
By applying the US discount factor (2.3) to a study to compensate for potential overestimation of US investigators, and using the principles described in this paper, we were able to effectively model the planned recruitment plan (Fig. 1).
Use of median to determine country averages
Analysis for both feasibility projections and the actual data (Fig. 2) of this hypercholesterolemia study shows that the frequency distributions were not symmetrical, but that there is a positive skew (i.e. the distribution has a tail to the right). This effect is more pronounced in the feasibility data than in the actual study data (Fig. 2), a finding that is consistent with data from other feasibility versus actual study comparisons (8 studies, 20 frequency distributions, 567 feasibility investigators and 326 study investigators), in which the average skew of the actual data (0.99) is consistently lower than that of the corresponding feasibility data (1.79).
The median should be used when calculating the average of a skewed distribution, as it is not influenced by outlier values in the tail of the distribution. If the arithmetic mean is used, it will be strongly influenced by the tail of the distribution, meaning those few investigators reporting high estimates of patients to be enrolled.
And as this effect is stronger in the feasibility data than actual recruitment data, this would lead to elevated projections. The data shown in Table 3 illustrate the value of using the median, which also reduces the temptation of removing some of the high outlying data points in calculating the mean (hypercholesterolemia study). This should become the measure of choice for recruitment planning7.
Well-planned feasibility allows for the collection of meaningful data that can be analyzed and reliably used for recruitment planning. We have described some simple modeling using a well designed tool, the use of calibrated, country specific discount factors and using the median to define the country average. With this approach, investigator estimates obtained from well-designed questionnaires can be converted into meaningful recruitment projections. This will allow more accurate planning of clinical trials, and prevent the accrual of additional costs due to recruitment delays.
Anne Schneeberger, PhD, and Janet Jones, PhD, Icon plc, Dublin
 Centerwatch 2008
 Reinventing Patient Recruitment, 2007
 Scrip June 2009
 PRA International: Better Feasibility for Global Clinical Trials, July 2007
 Joshua Schultz: Improving Subject Recruitment, Applied Clinical Trials, March 2008
 Benjamin Quartley: Partnering for Better Performance, www.GCPj.com, February 2009
 Michelle Jones and Stephen Jones: Data Based Predictions, Applied Clinical Trials, March 2008
Icon Clinical Research Inc.
320 Seven Springs Way
Brentwood, TN, US 37027
eMai l: firstname.lastname@example.org