luke richards wrote:A common complaint is that you can't distill the essence of a clinical encounter out in a randomised trial. But you can actually measure the effects of a real life encounter, which is why I am interested in the current thoughts of acupuncture academics on using pragmatic trials
Ah yes - the pragmatic trial - a shelter for rapscallions.
I'm sure there is some proverb along the lines of ethics being the shelter or refuge of some sort of dishonorable character but i can't think of it for the moment - but lets leave ethics for another time - I actually don't mind what the committees do- its just the ridiculous constraints they put on you - cover every conceivable ethical dilemma that may occur, put it in an acupuncture context (which you have to explain to us because we don't understand it) and do it all in 300 words????
Anyway back to the topic at hand - pragmatic trials. No doubt this was thought up by someone after the fact in an attempt to explain why they did not use a placebo and why they stuffed up their randomisation - and a bloody good strategy from one perspective. Pragmatic trials have their place but to cut through all the crap - your chances of having a large scale pragmatic trial (which by their very nature require huge subject numbers to maintain experimental power) is substantially less than if you propose a standard RCT (explanatory) study (at least here in Australia). Why - because the powers that control the $$$ understand RCTs and they don't understand or don't like or don't see the value in pragmatic trials.
The most important question you need to ask yourself is why are you undertaking this research?
Are you trying to prove the effectiveness of acupuncture or the importance of research to an existing segment of the profession that already operate without a research base (just fine thank you very much - the yin/yang/5 qi/8 evils/9blah blahs of it all) and likely don't want proof of what they already "know" - preaching to the choir?
No - let me guess - your a crusader - the joan of ark of acupuncture needing to fight those evil western doctors who never understand the yin/yang,9 blah blah of it all - lets open their eyes, integrate it into hospitals and get a medicare rebate for ALL - acupuncture jihad!!
OR are you undertaking research in an attempt to CHANGE CLINICAL PRACTICES? This is one of the fundamental questions asked by funding agencies - how will your research change fundamental clinical practices. If you like, place it back into the context of the beautiful baby point we were discussing in another thread. There are those that say the beautiful baby point is a load of cobblers, those that say it will change the genetic coding to produce superbaby and those (like me) who say - I don't know, but lets design a trial that will test the concept so that if it is true we can all use KI9 to produce superbabies that will save our planet or we can stop wasting our time and ripping off patients on something that has no validity. (ie a trail whose results will change clinical practices - hopefully for the better)
So this is a very long winded way of saying you design your study according to the outcomes you are trying to achieve.
If my goal in an exploratory study (mechanism) - placebos are irrelevant, subjects can act as their own controls.
If my goal is an explanatory study (efficacy) - i must build in suitable placebos, blinding etc
If my goal is a pragmatic study (effectiveness) - i should build in placebos and blinding if i can, but if i cant who cares because the ultimate question is not
WHY does it work or
HOW does it work, but is there an
EFFECT?
If my goal is to explore the types of research questions that may be relevant to clinical practice and hence guide the design of RCT then i will engage in clinical auditing (which i do at UTS) or an SSRD.
Each method is valid in its own context and in most cases of acupuncture research today we actually bastardise the lot in an attempt to find a hybrid that cover most (but not all) sins.
Coming back again to pragmatic trials, as this was the central question, there are certain issues that arise.
1) because you have no placebo, your effect size (outcome of treatment) has to be very large otherwise your experimental power is reduced. The only way to boost the experimental power back up is to increase the subject numbers which increases costs.
2) if you are testing a standard acupuncture protocol (a set of points) against standard practice (pointed individually selected according to patient) what are you actually testing? Does it really matter? How is your outcome going to actually change clinical practice and be of benefit to the profession and community? Testing a free set of points compared to a standard set really does not tell you much if acupuncture has a system wide, physiologically balancing effect. Its not like testing one form of manipulation for back pain compared to a completely different manipulation because the different manipulations would be designed to activate different muscles/nerves etc. QI however is QI and in many respects the channel system is closed so sticking one acupuncture point somewhere in the system will ultimately affect the qi elsewhere - this is the basis of 5 element treatments - balance the liver to affect the spleen.
3)pragmatic trials by their nature can be seen to be loose with their randomisation because they already work from a biased population. In that respect do we choose the sickest patients or only those that are mildly sick. What intervention will we use if it is not standardised and how does the knowledge that you are dealing with the sickest of the sick affect the experimenters interaction with the subject?
there are many other issues to deal with as well but this is already getting too long. At the end of your pragmatic trial, what are you actually left with? What have you measured? Certainly not the acupuncture effect. What you have measured is a change in that patients outcomes based on a range of variables (acupuncture, experimenter bias, placebo, time etc) most of which can't be effectively separated out.
consequently, return to your base question - how do my results change clinical practice - if they don't why did you bother? If they did or at least show the potential that they will, then your pragmatic trial design was clearly appropriate to your end goals.
there is no one fits all - all models have inherent problems - it is the case of finding the best fit for the outcome you want to achieve