“A controlled experiment is set up so that there will be no confounding variables—no variables other than the variable under consideration that could significantly affect the results. The problem is, people’s behaviour is affected by their ideas, so the ideas in the subjects’ minds are variables. The number of possible ideas that a single mind could hold is far greater than the number of people on Earth, so it doesn’t matter how large the sample is—it can’t be large enough. No sample can be large enough to control for all the variables in any experiment involving human psychology.”
– Sarah Fitz-Claridge
From the archives: The original post was posted on 27th March, 1996
In his Mon, 11 Mar 1996 18:42:42 post, a poster summarised the interpretation of the results of what he calls a “reasonably controlled” study about the effects of television.
First, there is no such thing as “reasonably controlled”. There is either “controlled” or “not controlled”. If the study was “reasonably controlled” that means it was not controlled. Indeed, it is fundamentally impossible to control any experiment in human psychology, because one can’t even in principle control for all the variables in a study like this, whatever these sociologists might like to think.
A controlled experiment is one in which the variables are controlled! That means that things are set up so that there will be no confounding variables—no variables other than the variable under consideration (here, television-watching)—that could significantly affect the results.
The problem is, people’s behaviour is affected by their ideas, so the ideas in the subjects’ minds are variables. The number of possible ideas that a single mind could hold is far greater than the number of people on Earth, let alone the number of children in one of these Canadian towns studied, so it doesn’t matter how large the sample was—it can’t have been large enough. Nor could any sample be large enough to control for all the variables in any experiment involving human psychology.
Moreover, it is impossible in principle to control for all these variables, because if one did manage to set things up in such a way that there were no confounding variables, one would then not be looking at what one wanted to look at in the first place, since it would be a completely artificial laboratory environment.
The way to hold all the variables constant would be to use similar techniques to those psychologists employ when the subjects are rats. The experimenters would have to use genetically cloned children, so that they were genetically identical. There is no mention of genetic cloning in this study.
Secondly, they would have to grow up in identical environments, i.e., identical boxes in the lab, as with experiments on rats. The subjects in this television study were not in individual identical boxes, but different towns. They would have to prevent any interaction, because with any interaction the environments would immediately become non-identical. There was clearly interacting going on here in view of the fact that the children were in school playgrounds together. Remember that the variables include the contents of people’s minds.
Suppose the experimenters were to do all that, and they got some results: the results of that would not tell us anything, because then the experiment would not be about normal people, it would be about the effects of television upon people brought up in those circumstances. The results would not would not tell us the effect of television on people living in a normal society. So it is impossible in priciple to do such an experiment on subjects in ordinary circumstances. This is a fundamental reason that one can never do such an experiment on humans. This is not a situation that is going to change “as Psychology becomes more scientific”! This is a fundamental thing! If they controlled all the variables, they would be experimenting on people in a situation that is uninteresting. And since these experimenters did not do it that way, the variables were not controlled.
The whole thing is just impossible! Just because they went through the motions of enacting the protocols of science, does not mean that this is science.
So I’d like to know why you think this “empirical evidence” worth anything? It isn’t. As I have said before, the bits of psychology that are interesting are not scientific and the bits of psychology that are scientific are not interesting. The sooner human psychologists drop their ridiculous “physics-envy” and realise the philosophical nature of our field, the better.
My introductory comments constitute the fatal argument against this whole study, and the comments that follow are unnecessary, but let’s suppose…
“… introduction of television in the 1970s to some remote Canadian communities. Realizing this represented a wonderful opportunity, some sociologists studied the towns before and after the introduction of TV.
They had three towns:
• No TV
• One TV (received one channel of CBC only)
• All TV (got lots, including some US stations)”
They all started off which identical people did they? No.
What’s the betting that this was not even a double blind experiment?!
Someone should send the Amazing Randi in to debunk this pseudo-scientific bunkum. Next they’ll be telling us that tea-leaf-reading is scientific. Talk about disingenuous! What can one say?! And to think people take it seriously!
“The introduction of TV seemed to lead to:
– a decline in the scores on a test that required creativity (no effect on vocabulary, visio-spatial skills, and only a small drop in IQ scores).”
First, the whole idea of a test requiring creativity is a complete joke. You can’t measure creativity so the test is not valid, and you can’t attribute differences in scores on that test or the IQ test to any particular factor with any reliability whatsoever. The whole thing is ridiculous from start to finish. There could be any number of reasons for differences on tests, not least of which might be that the children did not feel inclined to jump through the stupid hoops that day.
Now maybe it is because they all watched The Simpsons the night before and were busy thinking about that—in which case, that is good, and the test was interfering with their creative rational learning process—but there is no way of knowing! The putative effect could be due to millions and millions of factors that were not controlled in the experiment at all.
“- kids became more aggressive (based on the number of pushing or taunting incidents on school playgrounds (more aggression, and aggression became more widespread)”
The definition of “aggression” is theory-laden.
First, this is saying that pushing and taunting incidents on school playgrounds are caused by some underlying factor in the mind called “aggression”. I deny that. Children are human beings. What they do to each other depends upon what happens to them, and their own ideas.
For instance, suppose, for the sake of argument, that the mechanism were the following:
Children are actually full of aggression and hatred for each other, but they are also even more full of fear of the intimidating teachers. Then television comes along and shows them a wider world, and it shows them that the teachers are actually only human beings, and it shows them that there are other towns in other places, in which teachers behave differently and where the children are allowed more freedom, and so on; and the children become resentful towards the teachers. And as a result of gaining this perspective through watching television, the children start liking each other more, but also start to fear the teacher less, and so, despite liking each other more than they had done before, they start pushing and shoving each other more because they are no longer so intimidated by the teacher, and that has the greater effect upon their actions.
Now the “experiment” is perfectly compatible with that being the explanation.
I am not for a moment saying that that is the explanation—actually I don’t believe there was any effect at all—but nevertheless, note that my little explanation there is entirely consistent with the results of your “empirical study”.
Although I don’t know what the actual explanation is for the effect (if there really was an effect at all), I think it is probably due to something very complicated. And I would guess that whatever this thing is, it is good, not bad. And the reason I guess that is that the thing which has been introduced—namely television—is good.
Remember that part in Oliver Twist where Mr Bumble blames the meat? BTW, as you probably know, this book has quite a lot of interesting insights into the nature of cruelty to children, as has Great Expectations, and other books of Charles Dickens. It is also full of biting wit, and we all love it here. Unfortunately, until I started re-reading it to my children recently, my impression was that it is a singularly dreary book devoid of interest. I certainly did not remember the wit. But that’s school for you—ruins most children’s enjoyment of good literature. Anyway, I digress…
At one point quite early in the book, Oliver is hired out to an undertaker’s shop where he works as a virtual slave. But whereas at the workhouse he used to get only gruel to eat, here, he gets meat. After a while, he is provoked into a fight with another boy, Noah Claypole, and Mrs Sowerberry sends for Mr Bumble, and complains that Oliver has gone mad. Mr Bumble says that it is not madness but the meat that is responsible, and says: ‘“You’ve over-fed him, ma’am. … If you had kept the boy on gruel, ma’am, this would never have happened.” “Dear, dear!” ejaculated Mrs Sowerberry, … “this comes of being liberal!”‘ Mr Bumble urges Mrs Sowerberry to starve Oliver for a few days then put him back on gruel and no meat for the remainder of his “apprenticeship”.
In other words, Bumble’s argument is that although meat makes the boys stronger and better workers, and therefore one is always tempted to give them meat, in fact it always makes them “vicious”, and therefore is not worth doing.
I think that television makes children “vicious” in the same way meat (or, if you vegetarians prefer, “food more nutritious than gruel”) makes them “vicious”. That is, if they are being very badly treated, then giving them something good, whether it be more nutritious food, or television, makes them stronger, and therefore more inclined to rebel. If they are kept constantly on the verge of starvation or if they are constantly on the psychological defensive, they may indeed be more docile; but how can you tell aggression that is caused by becoming less docile, from aggression that is caused by becoming more “aggressive”?
In fact, I don’t think there is such a thing as “aggressive” any more than there is such a thing as “socialistive”. This notion of “aggressive” is taken from animal psychology—it means “some tendency to act out an aggressive pattern”. Human beings don’t have such patterns. They behave with intentions and for certain purposes they have. And when a human being behaves aggressively it is because he wants to hurt the other person, not because he is “acting out an aggressive pattern”. And he wants to hurt the person for some reason that he has, which may be right, wrong, a mistake, not a mistake, etc.; but we can only discuss it in those terms.
“- harmed the acquisition of reading skills (but didn’t seem to affect people who were already readers)”
Well, it could be true in the same way that “meat” makes them “vicious”, but I don’t see how the experimenters could possibly come to that conclusion from any measurement. It might be that it did not harm the acquisition of reading skills but just made different skills happen at different ages, and that the net effect is actually to make them read better at sixteen than those deprived of television, but that because they are learning things in different orders—because they are learning preferentially the type of words that occur in advertisements but don’t appear in school tests—it looks as though their reading skills are not as good as the TV-deprived group, whereas in fact, they are better.
“The decline in the acquisition of reading skills was ascribed to time. TV takes time away from competing activities, and learning to read is a competing activity.”
That is circular. To say that television is harmful, and to base that on the assumption that television takes time away from reading, is circular. Television would take time away from reading if it were better than reading as well, so the mere fact that it “takes time away from” reading proves nothing. What if television is actually better than these competing activities? Using your logic, one could reject anything that was genuinely better than reading—on the grounds that it takes time away from reading.
I think that all these results are false. But even if they were true, they don’t mean anything; for instance, if it were true that television reduced reading skills, that might be because watching television is better than reading. Television may or may not be better than reading, but this study cannot show that it isn’t.
See also:
- Philosophical theories are refuted by argument, not empirical tests
- Surely studies show that parents have little impact on how people turn out?
- Isn’t taking children seriously a risky experiment with children? Is there any evidence that it works? Has it been studied?
Sarah Fitz-Claridge, 1996, ‘It is impossible to control for all the variables in any experiment involving human psychology’, https://takingchildrenseriously.com/it-is-impossible-to-control-for-all-the-variables-in-any-experiment-involving-human-psychology/