Pragmatic trial design elements showed a different impact on trial interpretation and feasibility than explanatory elements

Abstract

OBJECTIVE: To illustrate how pragmatic trial design elements, or inserting explanatory trial elements in pragmatic trials affect validity, generalizability, precision and operational feasibility. STUDY DESIGN AND SETTING: From illustrative examples identified through the IMI Get Real Consortium, we selected randomized drug trials with a pragmatic design feature. We searched all publications on these trials for information on how pragmatic trial design features affect validity, generalizability, precision or feasibility. RESULTS: We present examples from the Salford Lung Study, INterSePT, STAR*D and CRUCIAL trial. These examples show that incorporating pragmatic trial design elements in trials may affect generalizability, precision, validity and may lead to operational challenges different from traditional explanatory trials. Inserting explanatory trial elements into pragmatic trials may also affect validity, generalizability and operational feasibility, especially when these trial elements are incorporated in one arm of the trial only. Design choices that positively affect one of these domains (for example generalizability), may negatively affect others (for example feasibility). CONCLUSION: Consequences of incorporating pragmatic or explanatory trial design elements in pragmatic trials should be explicitly considered and balanced for all relevant domains, including validity, generalizability, precision and operational feasibility. Tools are needed to make these consequences more transparent.