Issue link: https://www.massageandbodyworkdigital.com/i/72195
SOMATIC RESEARCH they took to strengthen their methods and what challenges remain to consider in evaluating the study. Those steps include such things as making sure the study was carried out on a large enough population—a sufficient sample size— to be as sure as possible that they are seeing the results they think they see. As the reader, your job is to evaluate how well they succeeded at their task. The ideal, of course, is a perfect methodology, but in the real world, researchers have to work around methodological issues to make the results as dependable as possible. Not only are there funding constraints and other pragmatic and logistical issues that work against the design of a perfect study, but there are also trade-offs between study issues that have an impact on methodology. For example, if you want your massage research protocol (your "treatment recipe") to be repeatable by other researchers, you have to spell it out in detail for them to follow. But if you do that in advance, then you're shutting out the interactive part of massage, where the therapist responds to the verbal and nonverbal feedback from the client—so the massage protocol is not very representative of what a session is really like. You could, of course, free up the therapists in your study to do whatever they would normally do in a session, responding freely and interactively to the needs of the client in the moment, but then, how would you tell researchers who want to replicate your study later how they should proceed? Those two important concerns, real-world representativeness and replication by other researchers, are actually in opposition to each other, because as one increases, the other necessarily decreases. So even in theory, assuming unlimited funding and total availability of other resources, methodological perfection is an unattainable goal. The best that researchers can do instead is a balancing act—make the study The risks of false negative and false positive errors can never be totally eliminated. as strong as possible and give the reader a heads-up on what factors of the study design should be taken into account in interpreting the results of the study. So even in good and strong research articles, you will find the researchers addressing "weaknesses" or "limitations" of the study. How all of these factors strengthen or weaken the study will vary in different contexts. For example, researchers studying a hospital-based treatment may focus on a very strict protocol to achieve the replicable and quantitative results they want in their particular situation, while researchers studying parents massaging ill children in their own rooms may focus less on the protocol and more on the interaction between the parents and children. Considerations like this make methodological issues and trade-offs into open-ended questions, so there is no checklist to use for every article. Think about what the ultimate purpose of the research is and how well researchers succeeded at their stated goals. POWER AND SAMPLE SIZE One limitation often found in massage research methods relates to study size—you'll find statements in the literature like, "Most studies contain methodological limitations including … few subjects …",1 or "These conclusions are limited by the small sample size of the included [research studies]."2 Clearly, when it comes to results, something methodologically important is going on with small studies. Additionally, you may have heard people say a massage research study needs about 35 or 40 people, more or less, to have a large enough sample size—what's up with that? What's so special about that number? Like the indicator of statistical significance p discussed in the last issue, the power of a test is a probability. In this case, it is the probability that the test will not make a Type II error (false negative) by missing a treatment effect that is really there. When p = 0.05, for example, it represents a 5% chance, or 1 time out of every 20 that you rerun the study, that you would make a Type I error (false positive), or think that you were observing a real effect, when it was really due to chance. While there is no universal measure of power, you'll often see 0.80 as a target that researchers aim for—it means that they expect that 80% of the time, or 4 times out of 5, if there is a treatment effect in the study, they will detect it. (Remember, for both p and power, when it is represented as a decimal number, multiply that number by 100 to get the percentage it represents.) The risks of false negative and false positive errors can never be totally eliminated, but judicious use of statistical significance and of power allow both of those risks to be managed, resulting in a certain degree of confidence in the validity of the study results. The ideas of statistical power, sample size, and the null hypothesis are tightly linked to each other. For reasons we'll get deeper into in a 138 massage & bodywork september/october 2008