Perhaps the patients should be allowed to use experimental treatments only under the condition that it is offered for free; the upside for the company is that if it works, they get the evidence of efficacy.
That would be the bare minimum in practice. That’s already the case: without premarketing approval (clinical trials), companies are not allowed to charge anything for medicine or medical devices. They’re not even legally allowed to ship them interstate until the premarketing application is accepted.
This should be the solution. If they’re trials for an unproven solution, why should they pay anything at all. I’m any case, it should be the other way around; it’s the companies who should pay the person for subjecting them to something very likely to be dangerous and painful.
In any case, I agree with the sentiment here. People should have the right to choose.
This is how it already works - this is what a clinical trial is. If you want to include the data point of a medicine working well as admissible, then you better have more than one patient try it at a time. Unless of course your drug is so good at curing patients who are absolutely terminal (like ibrutinib was). But most meds are not that much of a blockbuster. Thus you don’t know if a single patient survived if it’s because this medicine was very effective or if it was just as good as any other available treatment regimen.
>This is how it already works - this is what a clinical trial is.
That's not at all how clinical trials work. If and when someone decides to advance a drug to the next stage of development, it's done because they think there's a sufficiently profitable market at the end of the road. So lots of extremely promising drugs stall out if the numbers don't make sense. Furthermore, the trial enrollment process itself is cumbersome and often your only entrypoint is email "[email protected]" and hope you can a response and can navigate their arbitrary screening process. Speaking of screening: trials are full of exclusion criteria. They can get copy-pasted between trial protocols and often have no real reason excluding you. What's the washout period for your last line of chemo? Ever had any kind of immunotherapy? &c &c From the point of view of whoever is trying to get a drug approved it's better to be safe than sorry wrt any kind of uncertainty in a patient's medical status or treatment history. But that means that there might be a drug that looks extremely promising from previous trials but the only current trial excludes you for a small reason.
The problem with clinical trials is that they are looking for specific patients, though. And they will exclude patients that introduce confounding variables. Some studies exclude patients that are closest to death because they will interfere with the trial as well.
> They don't [get evidence of efficacy], though. They need a control group.
This is false. Control groups, preferably in double-blind studies, are an essential part of best practices. However, the bar for what constitutes evidence is much lower than that.
Evidence is any fact that makes a relevant conclusion more or less likely. Evidence need not be conclusive in and of itself. Over time, the cumulative effects of less-strong evidence can form the basis for a reasonable inductive conclusion. Sorry to be trite, but as an example, a person can have evidence that their romantic partner loves them without the use of an identical twin.
Insisting on gold standard medical evidence before allowing someone to undergo life-saving treatment—when the preponderance of the evidence that does exist supports that treatment—is, to be blunt, the very definition of depraved heart murder. It demonstrates a wanton disregard toward human life in the face of the known likelihood that it will result in some number of deaths.
More broadly, however, it is an extremely poor policy, in that this meme both weakens society's critical thinking generally and—among those who see through the elitist charade—significantly weakens the respect of those scientists who make the claim.
> Control groups, preferably in double-blind studies, are an essential part of best practices.
I wouldn't go that far. I think that a paradigm built upon a broader evidence base -- e.g. with far more extensive postmarketing surveillance of drugs -- would be more ethical, of more benefit to society, and ultimately more effective.
Double-blinded studies can miss, can be confounded by placebo effects, can easily be confounded by crooked researchers, and are frequently just plain unethical from first principles. I don't think that they should be considered "essential."
I think all your criticisms of randomized controlled trials (apart from the ethical one) apply to your suggested approach, no?
RCTs are the gold standard for demonstrating efficacy precisely because of the controls. I agree that they're not going to give you enough info about potential side-effects - a thorough monitoring program is definitely necessary for that.
But without RCTs, showing that a drug works would be much harder. How would you weed out the snake oil?
RCTs were not required and were rarely performed prior to 1962, and yet that period from roughly 1940 to 1960 is still today widely known as "the Golden Age of drug development." How did they know, back then, that new drugs work? Simple: They have sufficiently powerful (in a statistical sense) effects in a patient population. Very powerful effects require a very small population -- this applies to ALL drugs that are curative -- whereas moderate effects would require a large population. Reasonably sized safety trials and extensive postmarketing surveillance are entirely sufficient.
> How would you weed out the snake oil?
The current paradigm is "better 10,000 people die of neglect than 1 person die of snake oil or quackery." I'm okay with a little bit of snake oil if it means that more drugs are being introduced more quickly, and I trust practitioners and patients to, more often than not, determine the therapeutic regimens that are right for them.
I see! My intuition was that it would be hard to evaluate efficacy without RCTs - I assumed low effect sizes, which I guess is not necessarily true for many drugs.
Agreed that there should be a faster way to trial drugs and get them to market. Though I've heard that the "slowness" of the FDA is actually overstated; not sure how true that claim is.
Those were also small molecule drugs which are relatively simple to produce compared to modern large molecule drugs (biologics such as modern insulin, mrna vaccines, humira, many cancer drugs, etc.)
You might be surprised that the modern processes for true biologicals like peptides and mRNA are actually simpler than the chemical processes for classic drugs. E.g. solid state peptide synthesis. Once you can make a 10 peptide one, you can make a 1000 unit one in an identical way with more iteration.
Compare that to establishing a bespoke chemical process for each drug, or messing with purification of natural biologicals, or fusing antibodies to protein or peptides to grow inside a whole cell.
They should be actually cheap. They aren't because new.
Humira is quite a bit harder to manufacture cleanly than a peptide or mRNA, and still harder than a plain solid state synthesized and folded protein. It requires a lot of post translational modification, which means cellular machinery, which means a whole bacterial cell you now have to grow, feed and remove from the product.
The people who have already died to-date of your terminal illness aren't a good enough control group to tell that your cure works? This is the absurdity of frequentist statistics. If you can't tell the difference between recovering from the treatment and recovering by natural luck, then either the treatment is barely effective at all, or the doctor had no basis to tell you in the first place that you had a terminal illness.
That is correct, the people that have already died are unable to act as effective controls. To establish the medicine causes the disease to go away, you have to use something like a randomized trial. Otherwise, causation is not established. It’s a very difficult problem and a lot of smart people have try to come up with better systems.
With all due respect, why? This seems fallaciously wrong.
It is rather akin to saying that I can't tell whether seatbelts reduce car-crash fatalities simply by comparing collision fatalities from 1967 seatbelt-lacking car models with otherwise-identical 1968 seatbelt-equipped models; I would need to actually randomly and secretly sabotage a certain percentage of the seatbelts from 1968-model cars, while deliberately ignoring everything I know about prior fatality statistics. Even if every automobile collision without seatbelts was fatal, and every collision after seatbelts became mandatory was survived, even if nothing about the cars or driving conditions had changed, you're implying that it wouldn't provide any evidence of the efficacy of seatbelts, unless we conducted a new trial from scratch with a randomized control group as though we had no prior data?
What is the actual difference, in terms of causal evidence, between someone who died of the same disease before the study officially begins, and someone in the "control group" who dies after the study begins?
Almost nothing about medicine operates nearly as cleanly as that example; that's one of the major challenges with medical science. The seatbelt analogy breaks down in this context because the hypothetical of "Even if every automobile collision without seatbelts was fatal, and every collision after seatbelts became mandatory was survived, even if nothing about the cars or driving conditions had changed" doesn't reflect medical reality (and in the rare instances where it does, i.e. when improvement for the test group becomes obvious, immediate, and apparently side-effect free, it can become morally imperative to discontinue the control group).
In medicine, you get confounding factors like change in standard of care year-to-year and facility-to-facility. You almost never get the kind of "clean-room" trials you get in mechanical engineering (and, indeed, the processes to create them would often themselves be unethical). So the best you usually get is trying to reason around (to torture the analogy a bit) "Well we introduced seatbelts three years ago and relative to the year prior to that, fatalities went way down... Oh wait, but COVID also happened the same year we introduced seatbelts and everyone stopped driving" if you compare a current trial to past non-trial standard care as the control group.
Isn't this just the standard challenge of separation of signal and noise that exists in all disciplines, though? It seems that it would be better dealt with by general Bayesian methods than by strict adherence to a controlled-trial process that requires ignoring or even discarding large quantities of data (including every patient who ever had the disease prior to the start of the trial, which for rare diseases, might even be the majority of the data!)
Bayesian approaches alone are not enough to establish causality. I say this as someone using bayesian approaches in a causal analysis. Causality is intricately linked to the processes that underly how data is generated. Two different processes can lead to the same probabilistic results.
You don't necessarily need to establish causality in one grand step; it can still be effective to accumulate evidence of causality, even if this evidence cannot distinguish between other competing non-causal explanations.
> It seems that it would be better dealt with by general Bayesian methods than by strict adherence to a controlled-trial process that requires ignoring or even discarding large quantities of data
What is the Bayesian prior on "After receiving the COVID vaccination, patient died when they were struck by lightning?"
This is what I mean when I say the 'clean room' practices that are standard for most other fields of science would be considered unethical for medical trials. You can't, generally, lock people in a box for months to eliminate other confounding factors when trialing medicine; the best approach we have, therefore, is to minimize confounding factors by minimizing variables in space, time, &c. Yes, if we could lab-rat humans, keep them in controlled environments perpetually, we could use the previous in-a-box state as a comparison to the current state, probably. That's obviously a non-starter.
(There are some categories of medical research where time-variance has been used; it's just not generally an acceptable signal source, IIUC, for drug and therapy trials. If for no other reason than not nearly enough information has been captured about unrelated past patients to be compared to a current trial because relevant information to monitor was not known when monitoring past patients).
> What is the Bayesian prior on "After receiving the COVID vaccination, patient died when they were struck by lightning?"
Mathematically not difficult to deal with, which is why you intuitively know the answer already, because it's approximately the same way your brain works naturally. The chance of dying when you are struck by lightning is about 10%,[1] which completely overwhelms the chance of the vaccine coincidentally killing you at the same moment by several orders of magnitude; while you could use the this datapoint if you want, Bayesian analysis will still confirm that it has approximately zero evidence supporting (or refuting) that the vaccine causes death, so Bayes' theorem effectively drops that patient out of the analysis. Which is the same thing you'd probably have done intuitively, unless some kind of formal controlled-test study rule prevented you from doing what would normally be the correct procedure. (Or worse, forced you to subject an unvaccinated control-group patient to a lightning strike, just to maintain control...?)
But, since your patient died from the lightning strike rather than surviving it, there is a small signal present that points towards "the vaccine increased the chance of the lightning strike being fatal", which is a postulate that deserves a high initial skepticism until someone can propose a mechanism. But if you notice that multiple people die from lightning strikes after getting vaccinated, such that their lightning-strike survival rate is below the unvaccinated population's, you can start to update on that, and if it keeps happening, it becomes worth your time to investigate a potential mechanistic explanation. Like, the vaccination causing people to be less sweaty somehow.
... or the unvaccinated population tends to cluster, for sociopolitical reasons, in a region of the country, where lightning strikes are unusually powerful. IRL confounding factors abound.
Sweatiness is a good example of the sort of thing that would be hard to determine by comparing to past sample groups, since it's a rarely recorded piece of data.
Sure, so you add up those terms in the usual way, which you can do by straightforward factorization and summation, until your sum has diminished to probabilities small enough to round off.
This mathematical procedure is usually called "applied common sense", and if you are attempting to argue that it's impossible, you would in the process be proving that nobody can ever learn anything about anything.
You're still assuming that we know all the terms. We use randomized trials with controls improving out medicines because we almost never have enough information to compare two situationally different groups. In a controlled randomized trial, we can declare what we're testing and what we're measuring up front. The data is just too messy, otherwise. Too many hidden variables and too many possible alternate explanations that weren't measured.
> As of December 6, 2020, there were 3 SAEs reported in the vaccine group: a 65-year-old participant with community acquired pneumonia 25 days after vaccination, a 72-year-old participant with arrhythmia after being struck by lightning 28 days after vaccination, and an 87- year-old participant with worsening of chronic bradycardia 45 days after vaccination. On FDA review of the narratives, none of these SAEs are assessed as related. There were no cases of severe COVID-19 reported in the study.
The process in which a treatment is administered might change the way people behave, changing outcomes. Trials with control groups are easier to justify as having avoided these issues.
If you're interested in learning more, I would recommend learning about Judea Pearl's do-calculus, specifically the backdoor criterion. Although it isn't the end-all of causal analysis, it is a very useful non-experimental approach that will help you develop an appreciation for the benefits of controlled experiments.
As far as seat belts go, that's actually been something discussed a bit in economics. The main concern is if adding seat belts changes how people drive (confounding the actual effects of seat belts). The solution was to measure the protective effect of seat belts on occupants of the passenger seat in collisions involving drunk drivers.
If a disease at a certain well defined stage has a 100% fatality rate after 6 months, why would you need a double blind trial if some patients are still alive 1 year after taking your drug ?
The causality is pretty clear and while you might not claim that it double the life expectancy it would IMHO be immoral not to fast track the approval of that drug.
There is a standard of evidence above guesswork and below controlled trials, that is still useful: you accumulate enough survival rate data of whatever specific terminal condition you're hoping to treat, and compare it to the baseline untreated survival rate, which is already known.
Any statistically significant difference there can then incentivize a drug research company to invest in the (expensive) clinical controlled trial process.
It's strange that people here are actually saying it's better to waste perfectly good research opportunities and watch patients die, than to risk having anything less than perfect evidence of efficacy.
And that is why you shouldn't be able to ask for the experimental treatment... You should be able to ask to be part of a trial of the experimental treatment.
Remember that if 10 treatments are being tried at once, you can divide the pool of participants so that only a smallish percentage are part of the control group.
...I think that's basically how it works? If the company is running a trial, you can enroll in the trial. AFAIK being in a trial is free for participants.
Of course, there are any number of reasons someone may be rejected from the trial. If that happens, the company isn't going to just freely give them the drug anyway, because it wouldn't provide scientifically valid data. Also, no one would enroll in a trial (which carries the potential of getting the placebo) if they could get the treatment outside the trial.
The drug itself might be paid for, as well as tests under the scope of the study (which might be narrow), but services to administer the drug, and tests or treatment for effects not in the scope of the study, aren't always covered. Or, in the stupidest cases, approval gets bogged down by the drug company, patient, or administering clinic/practitioner trying to get the patient's insurance to cover whatever's not covered by the trial for so long that the patient no longer qualifies (or is dead) by the time insurance relents and agrees to pay.
Can you clarify why that is evil? Are you referring to the use of control groups or the division of the test group into several overlapping trial compounds?
Having a control group, but specifically in a study only given to terminally ill patients. Leave the control group to a subsequent study after effectiveness has been established in terminally ill patients.
I understand control groups in general, those are fine. But this is specifically in trials where people are pinning their last hopes out of desperation on some medicine and some random drop of the dice seals their fate.
(a) some random drop of the dice already sealed their fate. If anything, being admitted to an experimental group and then clustered into control or not, is the DM giving a second roll of the dice.
(b) most therapies don't improve prognosis in terminal cases and some make it worse. We don't know which beforehand. In the cases where the therapy makes it worse, was it therefore immoral to not put more people in the control group?
I'd argue the lack of knowledge makes it a wash morally. If we knew beforehand whether the therapy would be effective, we wouldn't need the control group, but we also wouldn't need the study.
The FDA thinks similarly. In late 2021, they canceled trials of Paxlovid because it was immoral to have a control group. They did not then _waive the trial requirement_, meaning Paxlovid was both too proven to deny but too unproven to be administered.
They don’t exactly need a control group if the chance of living N weeks is effectively zero. They would need it for something like living N weeks and living N+1 weeks, but any kind of change from an individual expected outcome would be useful signal for maybe more rigorous testing.
It is not. There are known treatment protocols with a reasonable (25%-ish) survival rate. There is also strong evidence that some people simply develop antibodies for it, and survive it without a vaccination or any treatment at all. It is rare, but definitely possible.
They can then apply to launch formal clinical trials with a control group. This initial evidence of efficacy is of-course anecdotal evidence with more traceability.
Would you feel the same if they got an "experimental treatment" from some quack who ended up giving them a highly dangerous chemical which resulted in a slow and extremely painful death? Because that is 100% what is going to happen when you remove all regulations for experimental medicine.
Similarly, you already cannot buy all cars privately anymore. Business-to-consumer is highly regulated (that's what the Federal Motor Vehicle Safety Standards are for, but home-built cars also need to pass several inspections before they are allowed on the road.
The company would still need a blinded control group to be able to demonstrate the effectiveness of any treatment.
However patients should be allowed to enroll for free into multiple treatment trials at the same time. For a given number of people and treatments, enrolling in multiple trials simultaneously gives more science output, and more chance of success for the participants, if you assume that ineffective treatments rarely are very harmful.
You just end up with a matrix of treatments. Whatever results you observe from someone undergoing treatments X & Y goes in the "X&Y" box, not the X box and the Y box.
But you would need much more participants in all trials, to make up for all those new combos that you now have to consider.
Also: I can imagine that most of the patients would be eager to engage in all possible treatments. You might not even have a control group that did not participate in the study of the faulty drug X
Then all you get is data about the effectiveness of XY. Would it be any better if only X was tested, but it turned out that that one drug 'X' contained two separate active ingredients, each of which the pharmaceutical company thought would probably be an effective treatment?
