Clinical observation is important because it remains the primary generator of hypotheses in the biomedical sciences. Dr. Edward F. Wright’s article in September JADA ("Referred Craniofacial Pain Patterns in Patients with Temporo-mandibular Disorder) illustrates many pitfalls encountered during the conduct of clinical research. Therefore, an examination of several of these potential problems should be instructive for clinicians and inexperienced researchers contemplating similar activity. It is to these individuals that this letter is directed, although a response from Dr. Wright is warmly welcomed.
Start with a clearly defined hypothesis designed to increase knowledge in a particular area. This is harder than one may think at first. You should aim to show that all other cases not in your sample are fairly consistent with the initial hypothesis and that another hypothesis does not fit the data as well or better.
Additionally, the greater the degree of exposure, the higher the incidence rate. For example, in the case of chronic pain studies, variables such as greater pain intensity, more manual pressure on examination or greater chronicity should also lead to higher rates of the outcome variable.
When stating the aims of your study, do not use phrases like "I speculate that ..." or "patients with TMD may be quite similar." Speculation refers to reasoning often based on inconclusive evidence. Additionally, "quite similar" leaves too much room for maneuvering; it is an imprecise phrase, counter to rigorous (statistical) interpretation.
When reading a study, begin by asking the following questions: "Can the results be applied to my patient(s)’ care? To what extent do patients in the articles resemble my patient(s) in terms of symptoms, disorder severity and duration and demographic factors that may affect treatment outcome?" (
Marbach JJ, Raphael KG. Future directions in the treatment of chronic musculoskeletal facial pain: the role of evidence-based care. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 1997;83[1]:170–6[Medline]
).
When recruiting research subjects, strive for diagnostic homogeneity. There is good reason why the National Institutes of Health recommended abandoning the term "temporo-mandibular disorders," or TMD (
National Institutes of Health Technology Assessment Conference on Management of Temporomandibular Disorders. Bethesda, Maryland, April 29-May 1, 1996. Proceedings. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 1997;83[1]:49–183[Medline]
).
The well-recognized heterogeneity of TMD as identified in the NIH Technology Assessment Conference and even heterogeneity of disorder within the myofascial subtype (
Greene CS, Lerman MD, Sutcher HD, Laskin DM. The TMJ pain-dysfunction syndrome: heterogeneity of the patient population. JADA 1969;79[11]:1168–72
) make it virtually impossible to answer the questions posed in reference to the referred pain patterns reported.
It seems unlikely that patterns would be identical for clinically different conditions unless they are the same for everyone. Moreover, the reader is furnished with no information about severity, duration or distribution of pain, any or all of which could have influenced the results.
However, all of these issues pale before two major problems, namely blindness and the lack of a control group. Dr. Wright discusses the potential for subject bias and states: "I was careful not to bias the subjects; only 29 percent of the subjects reported referred pain from the masseter muscle." Prior to this he reports: "I asked subjects whether pain was developing or intensifying in a different location than that being palpated."
Does anyone believe that the subjects were unaware of his goal to identify patterns of referred pain?
This phenomenon is what researchers call subject bias. Subjects know what the investigator is looking for and frequently are willing to cooperate. The problem is that no one knows how frequently any particular sample of subjects complies. To help control for this, someone unfamiliar with the hypotheses of the study (blind), not the chief investigator, performs the clinical examination. While not a perfect or the only solution, this method is preferable to the one employed. Why is this so? Not primarily because the subject is biased but because of the inherent bias of the investigator (
Cohen P, Cohen J. The clinician’s illusion. Arch Gen Psychiatry 1984;41[12]:1178–82[Abstract]
). Does anyone believe that Dr. Wright was unaware of his goal to identify referred pain patterns?
In a recent popular movie, a youngster, and only he, sees ghosts. Since he is the only one who sees them, he is helpless to prove his unique power to anyone else. Finding referred pain patterns is analogous to identifying ghosts; only one person performed the examination and wrote the report. One cannot deny that Dr. Wright has special skills to identify referred pain patterns, but neither can he prove the validity of these skills.
The study states that "patients with TMD often report referred craniofacial pain generated from head and neck palpation ... ." How does he know this without a control group? Perhaps anyone would respond with reports of pain at sites other than those palpated when Dr. Wright performs his examination.
For example, maybe only depressed pain patients report referred pain. And since rates of depression, common in facial pain patients, were not reported, the reader just does not know. The list of absent variables is nearly endless, and one cannot expect Dr. Wright to include all of them. That is why researchers employ control groups.
The appropriate selection of a control group, while not simple (
Marbach JJ, Schwartz S, Link BG. The control group conundrum in chronic pain case/control studies. Clin J Pain 1992; 8[1]:39–43[Medline]
), is absolutely necessary in studies such as this one.
Because no control group was used, we have, in fact, learned nothing about the relation between facial pain patients and referred pain patterns. The asymmetry makes it impossible to determine if differences detected are due to the presence of the disorder under investigation, or some other uncontrolled and unidentified variable. The lack of controls in the present study does not inform us about potential correlates of so-called TMD because the study did not compare patients with controls.
One final point is important to mention. The paper includes a lengthy discussion of referred pain theories. Nevertheless, the data do not move forward our understanding of whether or not referred pain is a ghost or real. It does not tell us whether it is related to or—if it exists—is independent of facial pain. Remember, no hypothesis was presented, so it is not surprising that no problem was solved.
For those who want to move knowledge forward, do not despair. However, before you undertake lengthy and time-consuming activities, spend some time thinking through the problem. Clinical researchers who are also child prodigies are rare indeed. Even among those who succeed, learning research techniques takes time. If you get stuck, find yourself a mentor; they are out there. Do the right thing.
