N=1 and group clinical trials

Advances in the investigation of the physical universe we live in.
Post Reply
User avatar
Nonc Hilaire
Posts: 6168
Joined: Sat Dec 17, 2011 1:28 am

Re: Real Life

Post by Nonc Hilaire »

Typhoon wrote:
Nonc Hilaire wrote:
Typhoon wrote:
Mr. Perfect wrote:So CS, I'm wondering your thoughts on acupuncture, it seems you would have a unique take on it. I have people that swear by it, can't say it did a thing for me.
I've yet to come across a properly designed double-blind study that finds that acupuncture is more beneficial than a placebo,
so my conclusion is that people who "swear by it" are swearing by the placebo effect.
The placebo effect is deserving of more study than many interventions. Why does it occur, and with such regularity? What are the parameters determining effectiveness?
Indeed.

I vaguely recall that few, if any, antidepressants performed better than placebos.

As always, it's a huge challenge to design, run, and analyse a proper clinical trial.
Placebos could be easily explored in a N=1 clinical format, and because individual differences are undefined it would be a more logical format as well.
“Christ has no body now but yours. Yours are the eyes through which he looks with compassion on this world. Yours are the feet with which he walks among His people to do good. Yours are the hands through which he blesses His creation.”

Teresa of Ávila
User avatar
Typhoon
Posts: 27242
Joined: Mon Dec 12, 2011 6:42 pm
Location: 関西

Re: Real Life

Post by Typhoon »

Nonc Hilaire wrote:
Typhoon wrote:
Nonc Hilaire wrote:
Typhoon wrote:
Mr. Perfect wrote:So CS, I'm wondering your thoughts on acupuncture, it seems you would have a unique take on it. I have people that swear by it, can't say it did a thing for me.
I've yet to come across a properly designed double-blind study that finds that acupuncture is more beneficial than a placebo,
so my conclusion is that people who "swear by it" are swearing by the placebo effect.
The placebo effect is deserving of more study than many interventions. Why does it occur, and with such regularity? What are the parameters determining effectiveness?
Indeed.

I vaguely recall that few, if any, antidepressants performed better than placebos.

As always, it's a huge challenge to design, run, and analyse a proper clinical trial.
Placebos could be easily explored in a N=1 clinical format, and because individual differences are undefined it would be a more logical format as well.
An N=1 clinical trial only works if [and only if] Rutherford's dictum
If your experiment needs statistics, you ought to have done a better experiment.
applies.

Historical examples in medicine would be insulin and penicillin: no treatment → certain death.
May the gods preserve and defend me from self-righteous altruists; I can defend myself from my enemies and my friends.
User avatar
Nonc Hilaire
Posts: 6168
Joined: Sat Dec 17, 2011 1:28 am

N=1 and group clinical trials

Post by Nonc Hilaire »

Typhoon seems skeptical of N=1 research in another thread, so I am starting this one.

These trials where the subject is their own control are especially good for testing ranged variables like blood pressure, blood sugar & mental illness where there is a high degree of individuality.

Take blood pressure. In an N=1 study, the subjects blood pressure would be monitored for a significant length of time producing many datapoints. Like a measurement every 15 min for several days. Then an experimental intervention like medication, acupuncture or whatever is introduced and the effects on the datastream monitored. Then the intervention is discontinued for a while and later reintroduced. The double reversal is necessary to control for any placebo effect.

If the intervention has an effect, there will be a statistically measurable difference between the baseline data and the intervention data. There is only one subject, but often there are more datapoints than in group trials

This has three big advantages. Individual variables are perfectly controlled, and it is useful for rare conditions, and it is easily affordable by individual investigators.

Group trials assume the population studied is representative of the general population, which is often not the case. It also assumes unmeasured variables eliminated by the control group, which is also not always the case. The results are necessary for large scale applications, but one usually wants a few successful clinical trials before investing in group trials.
“Christ has no body now but yours. Yours are the eyes through which he looks with compassion on this world. Yours are the feet with which he walks among His people to do good. Yours are the hands through which he blesses His creation.”

Teresa of Ávila
User avatar
Typhoon
Posts: 27242
Joined: Mon Dec 12, 2011 6:42 pm
Location: 関西

Re: N=1 and group clinical trials

Post by Typhoon »

An n = 1 trial only works for clear binary outcomes.

For example,

diabetes → insulin → survival.

For anything else, say, blood pressure, an n = 1 trial has no ability [technical term: no power] to reject

Type I errors: false positives: rejection of the null hypothesis due to a spurious accidental effect; or

Type II errors: false negative: acceptance of the null hypothesis when there is a real effect that has been missed.

The ability to avoid these to types of errors in a clinical trial depends on 1/ the experimental design and on 2/ the sample size: the larger the size, the greater the power.

A problem, especially prevalent in medical research, is the emphasis by funding organizations and journals on positive results.

[That and a lack of understanding of statistics by the people involved: pushing the data through a canned statistics software routine without understanding the underlying assumptions.]

This has resulted in the publication of too many papers that are, in fact, Type 1 errors: reporting an effect as real when in fact it is spurious and accidental.

In another thread, you linked to a report that noted that Amgen was only able to reproduce 8 out of 50 or so results of what are considered to be seminal papers in cancer research.
May the gods preserve and defend me from self-righteous altruists; I can defend myself from my enemies and my friends.
User avatar
Nonc Hilaire
Posts: 6168
Joined: Sat Dec 17, 2011 1:28 am

Re: N=1 and group clinical trials

Post by Nonc Hilaire »

Typhoon wrote:An n = 1 trial only works for clear binary outcomes.

For example,

diabetes → insulin → survival.

For anything else, say, blood pressure, an n = 1 trial has no ability [technical term: no power] to reject
No. Results can be statistically rejected by failure of the intervention to find effect, or by failure of the reversal. It is not binary - hundreds or thousands of observations are taken in every stage. Group trials are more binary.
Typhoon wrote:Type I errors: false positives: rejection of the null hypothesis due to a spurious accidental effect; or

Type II errors: false negative: acceptance of the null hypothesis when there is a real effect that has been missed.
Errors are possible in all experimental designs, but N=1 studies have fewer assumptions. The probability of errors is sharply reduced, albiet at the cost of widespread applicability.
Typhoon wrote:The ability to avoid these to types of errors in a clinical trial depends on 1/ the experimental design and on 2/ the sample size: the larger the size, the greater the power.
1) Design flaws in an N=1 design are more readily apparent. Type I errors are more visible
2) Sample size is the number of discrete observations, which determines the number of degrees of freedom. Not enough, and one cannot disprove the null hypothesis. An N=1 study has hundreds or thousands of discrete observations, so type II error is minimized.

The experimenter isn't ready to shotgun results into a general population, but if one is experimenting on basic effectiveness, dosage levels, or rare disorders this is superior and a required preliminary to group testing.

My overarching point is that an N=1 study is not a clinical anecdote. I think you may be conflating these. Many nonspecific clinical reports would be valuable if the intervention was properly studied in a proper N=1 double reversal design.
Typhoon wrote:A problem, especially prevalent in medical research, is the emphasis by funding organizations and journals on positive results.

[That and a lack of understanding of statistics by the people involved: pushing the data through a canned statistics software routine without understanding the underlying assumptions.]

This has resulted in the publication of too many papers that are, in fact, Type 1 errors: reporting an effect as real when in fact it is spurious and accidental.

In another thread, you linked to a report that noted that Amgen was only able to reproduce 8 out of 50 or so results of what are considered to be seminal papers in cancer research.
The chances of a Type 1 error sneaking through a proper N=1 trial is miniscule. I would say virtually no group clinical trial is undertaken without successful N=1 trials run first. When you get to the expensive group trials is when greed starts to corrupt the process.
“Christ has no body now but yours. Yours are the eyes through which he looks with compassion on this world. Yours are the feet with which he walks among His people to do good. Yours are the hands through which he blesses His creation.”

Teresa of Ávila
Post Reply