Statistical Analysis Plan- A beginner’s guide

A Clinical trial involves the testing of the drug(s) of interest on humans before making it available for commercial use. Clinical trial is a highly regulated process in which every small detail is documented and submitted for approval and audit.

One of the most important and obvious stages in a clinical trial is the analysis of the data that has been collected. Now, that the clinical trial process is highly regulated, it mandates to prepare a protocol as well as a Statistical Analysis Plan (SAP) before initiating the Clinical trial. These documents serve as the Standard Operating Procedures (SOP) for the Clinical trial processes.

There are many types of SAPs viz., Clinical Study SAP, Interim Study SAP, Data Monitoring Committee SAP, and Integrated SAP. These different types of SAP play a crucial role during different stages and for different departments in the clinical trial.

A statistician in combination with the investigator is present during the preparation of the SAP as well as while interpreting and gathering the requirements from it. This article is for those statisticians who are starting out new and have difficulties in understanding the SAP. This article focuses on addressing all the questions a statistician would have about the SAP.

What ?

According to the ICH E9 Statistical Principles for Clinical Trials, “A statistical analysis plan is a document that contains a more technical and detailed elaboration of the principal features of the analysis described in the protocol, and includes detailed procedures for executing the statistical analysis of the primary and secondary variables and other data.”

The SAP serves as a guide or a compilation of preset rules that are to be followed during the Clinical trial data analysis. Every detail starting from the sample size calculation, cleaning the data, handling the bias, descriptive analysis, etc., to the inferential analysis and visualization if required is pre written in the SAP. It also contains the name of the software to be used for statistical analysis and the details of the deliverables that has to be submitted in the Clinical Study Report (CSR).

Image source: wikihow

Why?

Now, as a statistician who is a beginner in this field, one would wonder, why this cumbersome process of preparing a 40 page SAP is mandatory and why only the Protocol isn’t enough. Well, any action could be performed efficiently when it is planned ahead of time. Especially in a highly audited clinical trial process, the prior planning is often required because any small deviation from the protocol may lead to the loss of integrity of the entire study. This not only affects the CROs and the trial sites but also the credibility of the sponsors. Thus, the Statistical Analysis Plan is very crucial and every step of the data analysis must be planned way ahead and should be strictly adhered to. The lack of reproducible results and the transparency in the methodology used during the data analysis, makes SAP highly mandatory. The other advantages of developing the SAP beforehand helps in the clear communication between the statistician and the investigator. Several research have proven that the SAP present at the time of the data analysis have improved its internal and external validity.

When?

The Statistical Analysis Plan is always created at the beginning of the data analysis. It is also called the Data Analysis Plan (DAP) in some organizations. It has to be developed before the treatment un-blinding. This ensures to curtail the possibility of “data-dredging” or “p-hacking”. Every clinical trial will have a DSMB (Data Safety Monitoring Board) that constantly monitors the safety of the trial subjects. During the course of the study, interim analysis is performed at the prespecified time to monitor the adverse events among the trial subjects. The DSMB utilizes the SAP and the Protocol as a guide during this process to decide on whether to continue or stop the study, based on the severity of the adverse events.

Where?

SAP is a guide that is similar to the protocol and is prepared cautiously like that of the Informed consent form, the CRF, etc,. The SAP is stored along with other documents in the clinical trials and is made available to all the personnel. The investigator, the statistician, and the DSMB have a copy of the SAP. NIH-funded interventional studies are mandated to submit a copy of the SAP to ClinicalTrials.gov. The SAP is also attached to the Clinical Study Report (CSR) while submitting to the regulatory authority. SAP is mandatory for interventional studies, whereas some of the journals require an SAP for observational studies too.

Who?

SAP is prepared collaboratively by the statistician and the investigator. Sometimes the statistician is also listed as the co-investigator. The investigator provides the study design, desired variables, exposure and desired outcomes. The statistician is responsible for framing hypotheses based on the study design and the outcomes, the sampling methods and the sample size calculation, the method of data analysis and the presentation of the results. Statisticians also include mock shells of Tables, Listings and Figures in the SAP, however it is not mandatory.

After the SAP is developed, it is reviewed by another senior statistician who was not involved in the development of the SAP. All of these have to be completed before the treatment or data un-blinding.

How?

Once we understand what is an SAP and why do we need it, the question of how to prepare the SAP arises. In 2017, Gamble et al., provided an exhaustive list of all the components required in a SAP. These are divided into 6 major sections of:

• Administrative Information
• Introduction
• Study Methods
• Statistical Principles
• Trial Population
• Analysis

The entire list can be found in the Table below:

Components of a SAP document

Summary

As a statistician, the Statistical Analysis Plan (SAP) is an important document that decides the integrity of the clinical trial. The SAP must contain the in-depth information of all the details needed for the data analysis. The statistician must possess comprehensive and critical evaluation skills in order to prepare as well as review the SAP without looking at the data. SAP is attached to the Clinical Study Report (CSR) and also submitted to the ClinicalTrials.gov, for interventional studies. SAP is highly important so as to avoid “data-fishing”.

References

1. https://www.clinicalleader.com/doc/fairly-simple-statistical-analysis-plan-elements-to-implement-in-your-clinical-study-0001
2. Pharmaceutical Programming: From CRFs to Tables, Listings and Graphs, a process overview with real world examples. Mark Penniston et al., Lax Jensen 2005.
3. Statistical Analysis Plan — Clinical Programming Reviewers Guide. Xiaoyin (Sherry) Zhong et al., Lax Jensen 2018.
4. https://cdsatoolkit.thsti.in/statistical-analysis-plan/
5. ICH Topic E 9 Statistical Principles for Clinical Trials
6. Gamble C, Krishan A, Stocken D, Lewis S, Juszczak E, Doré C, Williamson PR, Altman DG, Montgomery A, Lim P, Berlin J, Senn S, Day S, Barbachano Y, Loder E. Guidelines for the Content of Statistical Analysis Plans in Clinical Trials. JAMA. 2017 Dec 19;318(23):2337–2343. doi: 10.1001/jama.2017.18556. PMID: 29260229.