This article was first published on June 9, 2017 on The #MEAction Network Australia Facebook page. Neil also sent this to Australia’s Chief Medical Officer, Dr Brendan Murphy.
Harm; The definition of:
“A physical injury, especially that which is deliberately inflicted”. “Physical harm that impairs the value, usefulness, or normal function of something.”
Using these definitions, we will evaluate the data presented in the PACE trial1,2 to assess if there is any evidence of “HARM”.
Two major issues are evident in the trial in relation to harm:
- Patient selection. The patients were chosen on the basis of the Oxford criteria and the following were excluded: “patients who were younger than 18 years or at significant risk of self-harm, unable to attend hospital appointments, unable to speak and read English, had medical needs that made participation inappropriate, had previously received a trial treatment for their present illness at a PACE trial clinic (we initially excluded anyone who had received a trial treatment, but found the nature of treatment given elsewhere hard to establish). “Participants were also assessed by international criteria for chronic fatigue syndrome, requiring four or more accompanying symptoms, and the London criteria for myalgic encephalomyelitis (version 2), requiring post exertional fatigue, poor memory and concentration, symptoms that fluctuate, and no primary depressive or anxiety disorder (interpreted as an absence of any such disorder).”
- Adverse Events. Adverse events (HARM) was defined as “any clinical change, disease, or disorder reported, whether or not related to treatment. Three scrutinisers (two physicians and one liaison psychiatrist who all specialised in chronic fatigue syndrome) reviewed all adverse events and reactions, independently from the trial team, and were masked to treatment group, to establish whether they were serious adverse events.”
In the patient selection criteria, the worst affected subjects, those that could not attend the evaluation clinics, were excluded. No data was presented that allowed assessment of those excluded. Any conclusions that may be made can only be applied to the CFS patients with lower levels of syndrome severity. To apply the study findings to any seriously affected patients, those excluded, is not valid. However, conclusions from this study appear to have been accepted universally and those with more severe disease are essentially treated as with those with lesser disease. No valid data exists for the seriously ill patients regarding these therapies.
In a second study addressing the criticism re adverse events (AE) the authors included the following definitions.2
They determined whether each AE was serious or non-serious. A serious adverse event (SAE) was an event that resulted in one of the following outcomes:
b) threat to life (i.e., an immediate, not hypothetical, risk of death at the time of the event),
c) required hospitalisation except for elective treatment of a pre-existing condition,
d) increased severity and persistent disability, defined as:
(i) severe, i.e. significant deterioration in the participant’s ability to carry out their important activities of daily living (e.g. employed person no longer able to work, caregiver no longer able to give care, ambulant participant becoming bed bound); and
(ii) symptom and disability persistent, i.e. of at least 4 weeks continuous duration,
e) any other important medical condition which, though not included in the above, might require medical or surgical intervention to prevent one of the outcomes listed, and
f) any episode of deliberate self-harm.
For any AE established as serious, the scrutineers were unmasked to treatment allocation to establish whether or not the event was a serious adverse reaction (SAR). A serious adverse reaction was considered to be a reaction to one of the supplementary therapies or a drug prescribed as part of SMC. All those judged as definitely, probably, or possibly related were considered to be SARs.
A non-serious adverse event (NSAE) was any health event, which was not categorised as an SAE or SAR. Each NSAE was ascribed to the appropriate body system (gastroenterological, neurological, etc.) independently by two senior medical clinicians (one a consultant infectious diseases physician, the other a consultant liaison psychiatrist; both experienced in CFS), who were different from the independent scrutineers. NSAEs attributed to CFS (i.e. considered to be a symptom of CFS) were put into a separate category since there is no consensually agreed body system for CFS, and because of specific interest in these symptoms. Differences in clinicians’ ratings were resolved by discussion until consensus was reached. To summarise, adverse events were any new health related event reported by the participant in any context. These were independently judged as serious adverse events, using an a priori guideline of seriousness, and as serious adverse reactions if independently judged to be a reaction to a trial intervention.
From these definitions, the factors which the average MECFS patient knows are associated with symptom exacerbation, are considered “NON-SERIOUS”. These are symptoms that patients see as “HARMFUL” to their daily lives, yet the PACE study does not consider these as important.
Evaluation of the data presented in the paper shows that using the “clinicians’” definition of adverse events (NON-SERIOUS v SERIOUS) has largely excluded the majority of adverse events from the study conclusions. Therefore, it is worth evaluating this adverse events data within the papers.1,2
The percentage of subjects reporting “NON-SERIOUS” negative adverse events is shown in Figure 1. As can be seen from this data >89% of subjects reported “NON-SERIOUS” negative adverse effects.
The mean number of “NON-SERIOUS” adverse events per subject is shown in Figure 2. As can be seen from this data, the mean number of “NON-SERIOUS” events per subject is >3.27 events. Thus factors associated with MECFS disease exacerbation are not considered separately.
These data show that the patients within the study had a high reporting of adverse events (>89%), with a mean number of adverse events of >3.27 events per subject. These data indicate that multiple adverse events occurred in the vast majority of subjects within the study. The percentage of subjects reported to have “SERIOUS” adverse events and those with “SERIOUS DETERIORATION” during the study are shown in Figure 3. Graded exercise had the lowest rate of serious adverse events at 6%, whilst the highest rate was found with CBT (9%). Yet only 1% of subjects within that group had “SERIOUS” adverse events.
There is a significant anomaly within this data. The percentage with SERIOUS DETERIORATION should be equal to or less than those reporting a SERIOUS ADVERSE EVENT within the same group. This indicates a significant problem in reporting of the results.
Disease processes have variability, and the symptom sets within the patients will fluctuate with time. The accepted proportions for this variation are that 33.3% have an improvement, 33.3% do not change and 33.3% show an exacerbation of their condition. Figure 4a shows the percentage of subjects reporting a positive response with the various therapies. For there to be a significant positive outcome there should be a significant variation above the expected 33.3% distribution within the various groups. Figure 4b shows the percentage of subjects reporting either a negative outcome or no change as a result of therapy.
Initial therapy intervention at 12 weeks showed that all therapy types had a positive therapeutic outcome distribution between 8% and 28% below the expected normal variation, suggesting a predominately negative outcome for positive responses at this point in the therapy. By 52 weeks the variation was below the expected normal distribution for Adaptive pacing (2% below) and Specialist medical (8% below) therapies. At 52 weeks the variation was above the expected normal distribution for CBT and graded exercise therapies (both 8% above). We can conclude from these data that a positive outcome to therapy when considered against expected normal variation of disease was not significantly different. Less than 10% of subjects for CBT and Graded exercise showed a positive outcome against expected disease variation.
Assessing the NO CHANGE plus NEGATIVE OUTCOME responses against the expected normal variation for disease, Figure 4a shows that at 12 weeks all the patient responses were between 9% and 29% above the expected variation. This indicates a significant negative response at 12 weeks to all the various therapies. By 52 weeks only CBT and graded exercise were above the expected normal variation, both 7% above.
The second paper assigned the non-serious adverse events into fifteen different categories and they attributed an undefined cluster of symptoms as CFS/ME/PFS symptoms. Figure 5 shows the distribution of these symptoms across the various therapeutic groups. This shows that between 41% and 49% of subjects had these symptoms and that there was no significant difference between the therapeutic groups.
Subjects given CBT, which is predominately a counselling therapy had the lowest level of “CFS” adverse events (41%). Specialist medical inventions had the highest rate of CFS symptom adverse events (58%) followed by graded exercise (49%). However, it is well known that multiple symptoms may be associated with symptom exacerbations of CFS and these include, sore throat, gastrointestinal disturbance, musculoskeletal and cardiovascular events. Figure 6 shows the results for these symptom clusters, including the CFS symptoms. None were significantly different between the groups.2
The telling analysis in the second paper is the percentage of subjects who deteriorated in their physical activity over the 52 months of the trial (p<.0007). The data for the percentage of subjects reported to have deteriorated over the trials are shown in Figure 7. Eleven percent of the graded exercise therapy group deteriorated over the 52 weeks of the study but even worse was the number who deteriorated with Adaptive pacing (25%).
The PACE studies have suggested that graded exercise and CBT are the preferred treatments for CFS, compared with adaptive pacing and specialist medical care. However, these conclusions were drawn from assessment of the various therapies included within the study, and not against the normal expected variation of disease over time.
The PACE authors concluded that specialist medical therapies were more likely to have negative outcomes than CBT or graded exercise. However, by 52 weeks, graded exercise and CBT therapeutic outcome distribution variation was not significantly different from the normally expected disease variation. The authors concluded that graded exercise and CBT may be potential therapies for MECFS. This was based upon the false assumption that because one treatment is better than another it is therefore of use in treating patients. However, these therapies were no different from the normal variation of disease over a longer time period.
By 52 weeks most of the variation within the therapy responses was not significantly different from the normal expected disease variation. The data actually shows that the majority of subjects (>90%) actually had an increase in reporting of adverse events (Mean >3.27 events per subject). This increase in reporting of negative outcomes in the vast majority of CFS subjects should be combined with the fact that <10% of subjects actually reported a therapeutic benefit, and between 9 and 25% actually reported a deterioration in their physical functioning. Thus according to the definition of “HARM”, these therapies had a greater potential to be harmful than to be of potential benefit. This is supported by the 12-week patient assessment, which showed a greater proportion of subjects with negative outcomes compared with the expected normal symptom distribution patterns.
The following points can be made:
1. ~10% of patients may have gained a positive benefit from the therapies studied in the PACE trial, whilst the vast majority had an increase in adverse events and initially a negative outcome to therapy.
2. Between 9 and 25% of patients reported a significant deterioration in physical function over the course of the study, with those therapies requiring higher physical effort showing the highest rates of deterioration.
3. The worst affected patients were excluded from the study, yet the therapies studied in the PACE trials are recommended for all CFS patients. Clearly the trial did not assess the effects of the therapies on the worst affected patients. No data exists for assessment of the influence of these therapies upon the excluded patients. The recommendation for grade exercise therapy or any of the other therapies within the PACE trial, for all CFS patients is NOT evidence based and should be discontinued.
4. The therapies studied in the PACE trial seem to have greater negative outcomes and are potential sources of HARM to at least the worst affected patients, and up to 25% of subjects that use these therapies.
Recommendation of therapies of this type, based upon the PACE study data which shows such low (non-significant) positive outcomes and an increase in markers of potential harm (increased adverse events and deterioration of function), would be highly questionable. Importantly, the costs related to providing therapies of this type to ME/CFS patients would far out way the benefits to the patient population. This is supported by the patient outrage expressed over the use of these therapies. The conclusions within the PACE studies is consistent with a highly biased study which under-estimates, and does not assess, the adverse outcomes in relationship to the disease process.
The use of therapies of this type cannot be recommended based upon the data within the PACE study and must exclude the worst affected patients as they were not included in the study assessments.
(1) White PD. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. 2011.
(2) Dougall D. Adverse events and deterioration reported by participants in the PACE trial of therapies for chronic fatigue syndrome. 2014.
Make a donation to help Melbourne Bioanalytics continue their research into ME/CFS. Note: the button below will take you to a donation page on the University of Melbourne website. Funds go directly to support Melbourne Bioanalytics research.