FutureIQ Knowledgebase

The Randomness of Science - Why Randomised Controlled Trials (RCTs) Work - FutureIQ

3,836 views Wait, is this logic right? • Feb 28, 2025
Open in YouTube
Slog Reference: RCTs

Description

Do you know what a Randomised Controlled Trial (RCT) is? How and why are such trials conducted? Today, the 28th of Feb is World Science Day and hence in true FutureIQ style, it seemed fitting to talk about one of the pillars of good science - RCTs.

In this episode, we speak about the need for RCTs and how they came about to be. What are the different groups/phases within Randomized Controlled Trials? How does the Placebo effect play a role within RCTs? Along with giving several examples, we also explain the different variables and baises that are addressed through RCTs such as Observer Bias, Confounders Bias and Selection Bias.

While RCTs are a pretty common and standardised practice within epidemiology, there are natural questions that arise about the ethics of the practice as well. How fair is it to the participants of the trial? And where does the line of morality lie in finding the balance for the greater good of the world?

More videos you may enjoy:
Barnum Effect Explained: https://youtu.be/FcSuHP113NI
Survivorship Bias Explained: https://youtu.be/QjDXyuBJ0UY

Hope you enjoyed FutureIQ by Navin Kabra and Shrikant Joshi. Do hit us up on Twitter:
@ngkabra http://twitter.com/ngkabra
@shrikant https://twitter.com/shrikant

Listen it on the podcast provider of your choice: https://tapthe.link/FutureIQRSS

Chapters:
00:00 Introduction
01:45 Why do we need RCTs?
03:55 Ad Hoc Fallacy
04:54 Scurvy and the start of Controlled Trials
07:44 Placebo Effect
10:30 So how do we find the balance?
13:29 3 Major biases solved by RCTs
15:03 Selection Bias
17:25 Confounder's Bias
19:10 Observer Bias
21:40 Double Blind
22:33 Ethics of RCTs

External sources that are cited:
- Lobotomy: https://en.wikipedia.org/wiki/Lobotomy
- Bloodletting: https://en.wikipedia.org/wiki/Bloodletting
- Data analysis shows that religious authorities develop systems of “praying for rainfall” which maximizes the chances of prayer happening just before rainfall: https://www.nber.org/papers/w31411
- The Illinois Employee Wellness Program https://www.nytimes.com/2018/08/06/upshot/employer-wellness-programs-randomized-trials.html
- Coffee and Lung-cancer (correlated, but really it is because coffee drinkers are also smokers): https://pubmed.ncbi.nlm.nih.gov/19362749/
- Lack of pirates is causing global warming: https://www.forbes.com/sites/erikaandersen/2012/03/23/true-fact-the-lack-of-pirates-is-causing-global-warming/?sh=5ac145893a67
- Website of spurious correlations: https://www.tylervigen.com/spurious-correlations
- Clever Hans, the horse who could do maths: https://www.youtube.com/watch?v=hAJlAuEo7Ac
- Parachute RCT: https://www.bmj.com/content/363/bmj.k5094

#futureiq

Related Slog Matches

RCTs

Manual

100.00

Transcript

did you know that the doctor who invented lobotomy the surgical practice of removing half the brain got a Nobel Prize for this why because he removed the brains of Nobel Prize committee or what sounds like that but in those days this was actually considered a revolutionary great medical treatment okay in 1951 20,000 people were lobotomized in the US okay and it turns out mostly gay people to K them of gainers or women okay ouch so how does this happen how does the entire world look at a barbaric practice like this and think it is a Nobel prize winning level Discovery right my question exactly how does this happen did they see that removing the brain basically reduced them to a state
of childlike Behavior well I mean one issue is that in some cases think childlike behavior is better than whatever was before more importantly there were many cases in which it didn't actually work well and those just got conveniently ignored because you want to focus on the ones that worked okay so human beings are pretty terrible at getting a good feel for whether something is working or not and that is why randomized control trials which solve this problem are one of the greatest inventions of the 20th century okay and with National Science day coming up we thought we will talk about our cities today makes absolute sense all right tell me about randomized control trials then before that let me
give a better example of why we need randomized control trials I think lobotomy was a pretty good example by itself well not really right I mean a lot of people think oh maybe they deserve it and all of that right and because it happened just to like a marginalized Community ah yeah let's take a more mainstream example okay blood letting for thousands of years there was a medical procedure called blood letting where you actually make a hole in the person and take out blood like liters of blood yeah and that is supposed to cure you of diseases why because the theory in those days was that diseases are caused by bad humor which are in your blood and if you
remove the blood now you have less bad humor you also have less blood and when you have less equal to zero then you have no blood and you die so you're trying to make a joke but really okay so how mainstream was this practice okay George Washington he had a throat infection and his doctor suggested blood letting and over a period of 10 hours they let 3.75 L of his blood and he died of course he died the human body has 5 L of blood you take 75% out of it the person is going to die my question is how how did they not test and verify it how was there no record keeping of
whether blood letting worked or not okay so here is a question right what do you define as blood letting work or not well the person lives and gets healthy after blood letting yeah so what happens most likely what happened in a lot of those cases is that you say collect statistics about you used blood letting and did people get better and you see oh majority of the people did get better so blood letting Works no what it means is that a majority of people get better for any condition whether you do anything or not right true this is called the ad hoc fallacy that's a Latin phrase the full phras is AD hawkk ero propter Hawk
meaning after this therefore because of this correct if something happened after something else you assume that this something happened because of something else correct and tolto has a great quote about this right what he says is that if you have a medical problem yeah and you are given some treatment to make that problem better and then if you actually get better no power on Earth can convince you that you didn't get better because of the medicine right yes and we've seen this in a lot of cases and we always talk about Homeopathy in this context so I'm not even going to bring it up now go on right the thing is that why somebody got
better could be so many different things right everyone is trying various different things and one of them worked and you don't know which one right let's take another lovely example okay scurvy okay so scurvy is this disease that used to affect only Sailors I remember in school we learned scurvy happens due to deficiency of vitamin C right oh you gave away the answer but yeah Sailors used to die on Long voyages after getting a disease called scurvy okay and nobody knew what is causing it nobody knew how to fix it okay over the centuries people had tried all kinds of remedies for this okay and you know how people are right they get attached to
remedies and some people would swear by this remedy and somebody would swear by a different remedy right true there was a guy called James Lind who decided to look into this in detail he studied centuries of reports about various remedies and he just saw that there is just you know too much going on here but he felt that citrus fruits probably do cure scurvy okay but he didn't want to just get attached to it okay he had a scientific mind so he came up with an experiment to find out okay on a ship he divided Sailors into two groups and one group he gave them lemons okay and the other group he did not give lemons and
at the end of The Voyage he saw that the ones who were given lemons did not get scurvy while the others did yeah because there vitamin C deficiency was covered by the lemons they didn't know anything about vitamin C and deficiency right true all he knew was that giving lemons to this group seems to work and he knew this with a high degree of confidence because he had separated them out into two groups one group got the treatment and didn't get scurvy the other group didn't get the treatment and got scurvy right so clearly the difference in the two is the treatment right all right so this is called a controlled trial okay okay this group is called the treatment
group and this group which didn't get the treatment is the control so you divide them into two two groups one get the treatment one does not get the treatment right but there is a Twist okay okay over the years people have discovered a new problem okay okay for example if you do an experiment like this this group doesn't get treatment this group is given treatment they're told that they're being given a medicine but in reality they're just given sugar water okay what do you think will happen it should not work right because it's not the treatment unfortunately almost all over the world this group actually shows better results than this group what why we don't know
for sure okay this is called the placebo effect and as far as I understand it this is because human body is a wonderful thing right if you are giving treatment to some people it makes them feel good about themselves they think that yes I can do it now and as a result there immune system works harder whereas if you give people something saying you know what this is going to make you worse they actually get worse because their immune system gives up it says oh well already I was in trouble and now you're giving me some horrible thing that will make me worse and they get worse this is by the way called a nobo
is the opposite of a placebo but in any case the point is that just by manipulating your emotions you can make people better okay this has been proved again and again and again so the point is that in modern science you can't have two groups where one is given the treatment and one isn't what you need is two groups where one group is given the treatment and the other group is given something which they think is a treatment but they don't know that they are just getting something which is a place yeah okay because uh you have to have a treatment that works better than the placebo because if the treatment works as good as the placebo then it's
not a treatment at all exactly exactly yes okay uh but here's the question what if the group that is getting the placebo is already healthy as in they don't they they are good enough to recover by themselves yes so that's the next problem that you need to solve okay because sure I mean you take one group who's getting the treatment and the other group just happen to be a group that is healthier or going to get better then you haven't proved anything right so what you have to do is you have to make sure that the two groups that you have the only difference between the two groups is that one is getting the treatment other is getting the placebo
all else has to be equal so for example you can't do this experiment where the sailors on one ship get lemons and the sailors on a different ship don't get lemons right because you never know that ship is different the food on that ship is different the captain treats them differently they sleeping schedule is different any of these things could affect it right so you have to do it with all else being equal right so it had to be the same ship it had to be from the same bunch of sailors who shared the same uh routine every single day that is why the controls uh also needed to be equal to the treatment uh
yes in in in this in these two groups but how do you achieve this equality because everybody is different everybody has a different Constitution everybody has a different Health trajectory or whatever you want to call it let me give you a story of how that came about right so in the early 1900s Ronald Fischer a great statistician he was studying agriculture okay and farmers were all very confused about various agricultural polic right which fertilizer works well or doesn't what crop rotation method is good or not good right the problem is that every Farm is so different and even for the same Farm every year is so different right so how do you get any idea of control right yeah I mean oh you
used fertilizer this year and everything is good but that could just be because the weather was good this year right or the rains were good this year how can you do this yeah okay so Fisher who was a statistician came up with a brilliant idea what he said take a large enough group okay and then you randomly assign half of them to the treatment and half of them to the placebo right now if you did true Randomness right and the numbers are large enough then whatever are all the individual difference between them should cancel out roughly and they should all be more or less the same that's the idea of Randomness right yes and then you give one of them the
treatment so this is a very very clear difference between these two groups everything else being random all else is going to be equal on an average correct across the group right of course some will be up here some will be down here but the same will be true of this group also and on on on an average for the entire group of both groups it will cancel out and you'll get like a statistical mean kind of a thing wow but still this whole thing put together huh didn't happen until 1948 1948 is the first scientific paper about streptomycin which talked about randomized controlled trial interesting because you told us that the Nobel Prize for lobotomy was awarded in 1949 the RCT
for streptomycin happened in 1948 if the RCT had extended to the lobotomy practice then maybe the Nobel Prize would not have been awarded to lobotomy absolutely right because since then R CS have become so important that we pretty much try to use RCs for any important uh Discovery so what I'm taking away from this is that rcts were invented to solve the placebo effect and they did they did but in fact they solved other important problems okay there are three major problems selection bias confounders and Observer Bias and R C solve those also okay uh what are these let's take them one by one first selection bias so there is this great study called the Illinois
workplace Wellness study okay so work Wellness programs were offered to employees okay and the program offered activities like weight loss EX exercise taii smoking session program Financial wellness program and all of this right sounds like a great workplace and then they looked at people who did this program versus people who didn't do the program and the question they asked at the end of the period what was the difference in Behavior difference in outcomes correct so the people who took the program went to gym at Double the rate of the people who did okay okay the people who took the program ended up spending $525 on an average on Healthcare okay compared to 657 for the people who
didn't so Healthcare cost has gone down correct Hospital related costs went down from 387 to 273 right okay not just that but people who took this activity huh only 7.2% of them quit their job whereas attrition in the the other group was 15% right so attrition also half so this sounds like a great workplace wellness program why didn't uh it work out what was a problem selection bias right I said that this program was offered to employees oh they chose whether to be in the program or not now you tell me who takes a wellness program people who already wanted to go to the gym join this program okay now why why did I say this was a great study
okay yeah why did you say it was a great study because it seems not they picked 5,000 employees and then they offered this program only to 3,300 of them okay a about half of that 3300 took the program the other half who were offered didn't take the program but there were still 1,700 people who were not offered the program at all okay now the 1700 who were not offered the program at all H if you compare those outcomes those outcomes to the outcome of this whole group huh no difference all those differences that we saw disappeared yeah because yeah the control here was not the people who took the program or didn't take the program
the control here was the people who were offered and not offered exactly and that they had done randomly and that was the only way to get all else being equal because in this group which were offered all else was not equal correct right in fact all else was different because the ones who picked were very different from the ones who didn't pick and that like proves why you need randomization in your control trial and what it also means is that selection of that group is very very important to do randomly because humans no matter what they do if you pick based on some criteria you are going to introduce bias of some sort correct so randomization
makes sure that that bias of selection doesn't exist yes that explains selection bias but then you also said confounders right yeah correct so that's another very important concept so let's take this example okay okay they did a study and they found that people who have coffee get lung cancer at a higher rate but liver doctor keeps saying coffee is good to have three cups of coffee every day he says black coffee well so yeah coffee is not bad so how do you explain the fact that it looks like you know people who drink coffee are getting lung cancer when you look closely what you notice is that people who like to smoke also drink coffee at a higher rate and
they get lung cancer at a higher rate so if you don't pay attention to the smoking it will look like extra coffee consumption is causing lung cancer I'll give you a simpler funnier example right pleas if you look at the data you will see that increased ice cream consumption increases drownings yeah I I I'm beginning to remember this now yes so the confounder there H is the fact that in summer when the heat goes up ice cream consumption increases swimming also increases that is what is causing drowning right yeah so the idea of a confounder is a common thing that explains both the variables you are studying we have done an entire episode on this check it out it's excellent
stuff you can go into a lot of detail there yeah the episode is called correlation is not causation and we've taken a lot of these examples not just confounding variables but we' also talked about something called spurious correlations which is really funny and I'll tell you why it's funny because uh Pirates cause global warming if you want to know why go check that episode out but that explains confounders the third one is Observer Bias you said yes that's a very important one okay humans are terrible at observations okay in fact I keep seeing believing is seeing so just your beliefs completely color what you see what you record and that makes a huge huge difference in experiments
right we already talked about one of those things when the patient believes that they are getting treatment their outcomes improve okayo effect but similar problems exist with the researchers right oh okay so let me take an example okay imagine I told you that there is a horse here which can count and there is a horse there which can't count okay okay and the way this horse can count is that if you hold up a sign with say 5 + 2 written on it the horse will tap his foot 7 times wow smart yeah this is a real horse called clever H okay okay now if I told you this and I told you to go and observe whether it is
counting or not and you also go and observe this horse whether it is counting or not H you know what will happen if I did that with a number of experimenters huh very likely that you will get pack results at least from some of the experimental saying that this horse can count and this cannot but he can count right like if you hold five no clever Hans was a horse who was so clever okay the horse didn't understand numbers what the horse understood was that you keep tapping your leg until people start clapping that's clever right so the problem is that The Observers they already believed that this horse can count so they were just
just waiting for him to reach the correct number and they would start clapping okay okay right and this also kind of happens in real experiments right when a doctor believes that this patient is being given this strong medicine that I'm being forced to do an RCT because of FDA laws right when the doctor believes that the doctor behaves with the patient differently takes observations differently if the patient is doing a little worse say oh no this is some temporary thing so I might not record it and all kinds of problems occur because of that yeah that makes sense so what you're saying is rcts don't fully solve Observer Bias yes you have to add one layer to the RC is
called double blind okay double blind so we already talked about how the patients don't know whether they are in the control group or the treatment group right similarly the researchers should also not know who is in the control group and who is in the treatment group right ah so for example during covid when we were trying to figure out whether the vaccine works or not half the people were being given like a generic thing and half the people were being given the vaccine and the doctors giving that didn't know which was which right that is called double blind so both of them are blind correct the patient is blind and the doctor is also blind right so what I'm taking away from
from this is that rcts with double blind is the gold standard but my question is is all science done this way now no okay I'll give you an example okay parachutes don't work but they do work there has never been an RCT proving that parachutes work which means that it could just be like blood letting that we are all just you knowen about this there is enough evidence plenty of people have taken parachutes jumped out of a plane pulled the card and their parachutes have opened and I'm assuming that this is your treatment arm and there is a separate control arm where people are jumping out with Placebo parachutes what even is a placebo okay now you see why you cannot
have an RCT for parachutes yeah this brings up ethical questions right in more gray cases right imagine that there is a drug which in early trials has shown that it cures a particular disease yeah and now you're doing a larger trial so you take a bunch of people who are going to die and you say well we are going to flip a coin to decide which ones of you get the medicine that will probably save your life and which ones of you we are going just going to let you die yeah right so there is a issue there is there is that ethical issue for sure so yeah rcts with double blind are a good measure a gold standard but they
are not the ultimate solution is what you're trying to say whenever possible you should use R CS but sometimes it is not applicable sometimes it's a little too heavy weight but mainly I want you to take away from this episode The reason why RCS had to be invented without rities there were those problems placebo effect Observer Bias confounder selection bias those happen not just in those science experiments they happen in our life all the time and we need to learn to spot them and to be able to solve them using RCT is techniques yeah and as you were mentioning all of those examples I realize that there is a lot of things in our day-to-day life where
those problems crop up like uh placebo effect we've seen very regularly in cases of Homeopathy and then that tallstar quote comes to mind which is why Homeopathy works I guess but then uh selection bias U for example a restaurant sends out an online survey asking what uh people thought of it but online surveys are typically answered only by the young people the older people are not going to sit filling out online surveys so their opinions are not considered in the reputation of the restaurant uh we spoke a lot about spurious correlations and confounder effects we've done an entire episode on it uh Observer Bias where do we see obser well all of politics is Observer
Bias right the same thing a leader does half the people think it is the worst thing ever done and half the people think it it's like the best ever right I will give you another lovely example right please astrology all of astrology Works through Observer Bias right and uh it is called the baram effect uh we have done an entire episode on it which you will love if you likeed this episode check it out yeah we started off talking about rcds uh considering that National Science day was coming up soon but then uh it's not just about rcts it's about why those rcts came into existence and all of those problems that the rcts solved are also problems we experience
in our dayto day lives which is why I hope you've taken all of these things into account and you've started thinking about your life in terms of these issues and try and figure out how you can apply rcts in your own life let us know what you did meanwhile this is Shri Kant naen future IQ