Oxford University Press's
Academic Insights for the Thinking World

Scientific progress stumbles without a valid case definition

Current estimates from the Centers for Disease Control and Prevention (CDC) of the number of people in the United States with chronic fatigue syndrome (CFS) increased from about 20,000 to as many as four million within a ten-year period. If this were true, we would be amidst an epidemic of unprecedented proportions. I believe that these increases in prevalence rates can be explained by unreliable case definitions. For example, in 1994, the CDC’s case definition did not require patients to have core symptoms of the CFS. Making matters worse, in 2005, in an effort to operationalize their inadequate case definition, the CDC broadened the case definition so that ten times as many patients would be identified. Even though these estimates were challenged as bringing into the CFS case definition many who did not have this illness such as Major Depressive Disorder, as late as 2016, the CDC re-affirmed the merit in this broader case definition.

Another misguided effort occurred in 2015, when the Institute of Medicine (IOM) developed a revised clinical case definition that at least did specify core symptoms, but unfortunately also eliminated almost all exclusionary conditions, so those who had had previously been diagnosed with other illnesses such as Melancholic Depressive Disorders, could now be classified as meeting the new IOM criteria. This case definition has the unfortunate consequence of again broadening the types of patients that will now be identified, thus their effort also will inappropriately select many patients with other diseases as meeting the new IOM criteria. Making matters even worse, the clinical case definition was not designed to be used for research purposes, but it is clearly being used in this way, and one group of researchers has already inaccurately reported that the new clinical case definition is as effective at selecting patients as research case definitions.

Increases in prevalence rates [of CFS] can be explained by unreliable case definitions.

This comedy of errors becomes even more tragic with the recent development of a new pediatric case definition. As with past efforts, data were not collected to field test this new set of criteria. Even worse, medical personnel are asked to make decisions regarding symptoms without being providing any validated questionnaires, and this has the effect of introducing unacceptable levels of diagnostic unreliability. Scoring rules are so poorly developed that guidelines indicate that a child needs to have most symptoms at a moderate or severe level, but in reality, according to the flawed scoring rules of this case definition, youth can be classified as having the illness even if they report all symptoms as mild. These criteria further suggest that “personality disorders” should be assessed in children, as these disorders are listed as psychiatric exclusionary conditions; however, personality disorders cannot be diagnosed (or reliably assessed) prior to the age of 18, as personality characteristics are not fully developed until adulthood. Finally, these authors also require the youth to have at least six months of illness duration, whereas the Canadian criteria and others suggest that children with three months duration can be diagnosed with the illness. Other significant limitation of this primer have been mentioned by others. In summary, these authors failed to incorporate standard psychometric procedures that include first specifying symptoms and logical scoring rules, developing consistent ways to reliably assess these symptoms, and then collecting data to ensure that the proposed criteria are reliable and valid.

When a field of inquiry is either unable or unwilling to develop a valid case definition, as has occurred with CFS, the repercussions are catastrophic for the research and patient community. In a sense like a house of cards, if the bottom level is not established with a sturdy foundation, all upper levels of cards are vulnerable to collapse. Science is based on having sound case definitions that allow investigators to determine who has and does not have an illness. Having porous and invalid case definitions, whether clinical or research, affects not only prevalence estimates of CFS, but also has dire consequences for treatment approaches, as when individuals who have solely affective disorders are misclassified as having CFS, and when they improve from psychological interventions, it is easy to erroneously conclude that CFS is a psychiatric illness, which further stigmatizes patients.

Featured image credit: Guy by marusya21111999. CC0 public domain via Pixabay.

Recent Comments

  1. Wendy Boutilier

    I don’t see how a valid diagnosis can be made without a valid criteria. CFS is used as a bucket diagnosis because there is no criteria and the doctor is too lazy to investigate their patient’s health. What good is a bucket diagnosis if a patient says they are tired all the time. That can mean anything but a lazy doctor calls it CFS.

  2. Jeff

    Very good article. The hope is soon there will be a definitive bio marker that will put a solid boundary around those with cfs/me

  3. Rivka

    Thank you to the author for articulating a key problem facing patients, researchers and doctors who deal with ME/CFS.

  4. Mary Dimmock

    In the words of the 2011 State of Knowledge report from NIH, lack of consensus on the research case definition has created problems in “the entire scientific enterprise.” The 2015 P2P report called for HHS to sponsor a meeting to reach consensus on the case definition. And yet HHS is not doing this and the chaos continues.

  5. john duncan

    Professor Jason hits the nail on the head with his commentary on the damage wrought by poor case definition usage in Myalgic Encephalomyelitis(and continuing).

    It is impossible to read more than two pubmed articles on this disease without reading it is a “heterogeneous condition” somewhere in the introduction. Surely the simplest, most parsimonious, explanation is surely that this heterogeneity is an artifact of poor, varying case definitions; indeed often assessed in an incongruent, ad hoc manner by different research groups when selecting patient subjects.

    While there should certainly be discussion about what case definition is “best” it beggars belief that case definitions remain in use which:

    A.) Do Not include the core symptom of a 24-48 hour delayed marked worsening following exertion (usually called “Post-Exertional Malaise”)
    B.) Do Not specify a 50% + reduction in function.
    C.) Do Not include cognitive (eg trouble with recall memory, brain fog), neurological (e.g trouble with balance, incoordination) muscle pain, and unrestful sleep which are symptoms also present in classical cases which, research definitions, at the least, should be select for.

    It goes without saying that definitions such as the Fukuda ( which does not select for the defining feature ) and the Oxford ( which requires only persistent fatigue ) should be immediately retired.

    It is extraordinary that the definition most widely used in Great Britain and on which NICE bases its treatment recommendations is the Oxford Definition.

    As for definitions which do select for “post-exertional malaise” those which do not require some quantification of severity (e.g. An obese person who suddenly goes jogging will likely experience muscle aches and discomfort the next day…But probably NOT dizziness, headache, shortness of breath, increased confusion, difficultly standing upright, ataxia) should certainly not be used in research at the least…

    Contrary to the prevailing notion of a medically “mysterious” illness there, in fact, ARE tests which can aid tremendously in diagnosis and standardization of ME patient selection in medical research. One of the most useful is the two day exercise test. ME patients have a drop in vO2 max on the second day but not the first. To my knowledge this finding is unique to this disease. Researchers at University of Dundee have found that patients have increased isoprostanes–another test that which could be used.

    Tldr:
    1.) The definitions which do not even select for the core symptom of delayed post-exertional malaise should be retired immediately and studies using such should be evaluated skeptically.

    2.) Confirmatory tools such as the 2-day exercise test**, increased isoprostanes, decreased NK cell function and lightheadedness upon standing among others should be used to help standardize the populations of ME patients being researched.

    **For some of the most severely affected use of these tests may not be ethical due to risk of precipitating a worsening. In such patients it is important to note that in addition to meeting the criteria of a strict definition they suffer from light and sound sensitivity, have difficulty regulating body temperature and seizure-like episodes.

    –jd

Comments are closed.