Summary

• Science is not a cartoon
• Science works by predicting variation
• Science advances by exploiting human nature
• Some science experiments have multiple outcome variables and multiple predictors

Science Is Not A Cartoon

The cartoon version of “science” — that definition that teachers who had no idea what they were talking about gave you — runs along these lines:

Science is a method for understanding Truth. To understand Truth, a great scientist thinks deeply, and using the tomes he has read along with his powers of reflection, generates a Theory. Then, with great care, an elaborate contraption is created test the Theory. If the test works, the Hypothesis is Proven, and it becomes a Fact. Otherwise, the Theory is Wrong, and the cycle begins again.

Of course, that’s ridiculous. That’s not science. That’s what children think of as science.

Science Works by Predicting Variation

Here’s a better definition of science:

Science is a method for predicting variation. To better predict variation, scientists construct Theories, which are mental models that allow brute facts to be put in some sort of framework. For instance, the theory of Gravity explains the brute fact of an apple on a tree in one moment, and the same apple on the great in the next, into a narrative. Theories are operationalized using hypotheses, which generate specific predictions. So gravity on Earth can be operationalized as predicting that gravity acts like an acceleration that forces all object to the ground at a rate of 32.2 feet per second per second. Run enough experiments and you will begin to see this simple hypothesis mis-predict events, which will force you to generate other hypotheses. Eventually you will have a set of hypotheses which predict events enough to be useful to you.

I say this because of a recent post by Diane Ravitch (who was recently fired from the Brookings Institution), criticizing a Gates Foundation grant to measure attentiveness through measurement of human the electrical system.

If you know what science is, and how it works, your immediate thoughts should be.

Scientists desire to predict variation in educational outcomes. These scientists doubtless have Theories of education, which are mental models that allow brute facts to be put in some sort of framework. These theories are probably operationalized using hypotheses, which have generated specific predictions. There probably is error in the these predictions, which are leading to follow-up hypothesis. These scientists must think by adding information on attentiveness measured through the electrical system, they can reduce error, and predict educational outcomes better.

If you know nothing about science, such as Diane Ravitch, your reaction differs, you’ll write a nonsensical post with only one declarative sentence: “Shades of Brave New World.”

Science Advances By Exploiting Human Nature

Now, given that, try to understand the study, as the historian Mark Safranski did, in this way:

 

Let’s start from the assumption that this GSR bracelet study is actually a scientific study without hidden agendas.

But before the end of his first sentence, Mark (who unlike Diane, is attempting to seriously engage in this issue) is already lost on irrelevant tangents.

Why would science be free of “hidden agendas”? Why would scientists be some cold automatons driven by computer programs with no feelings, emotions, hopes dreams, or goals? Science advances through Academia. This is done by rewarding professors for obeying the interests of peer-reviewed grant funding agencies.:

Professors, like most people, respond to the incentives of power, influence, and money.

The institution of tenure reduces uncertainty regarding money, and focuses the incentives on power and influence.

Power in academia comes from the number of bodies a professor has under him. These bodies might be apprentices (graduate students he advises), journeymen (post-docs who have a PhD and work at the lab, or staff researchers), or simple workers (lab technicians, etc).

Influence in academia comes from the extent to which one is successful in influencing one’s peers. This is typically measured in terms of influence scores, which are a product of how often the academic is cited, weighted by how important of a publication he is cited in.

The best route to both power and influence is to earn grant money. For example, consider a professor who receives grant money from a federal agency. Some of this money goes to equipment, but the majority goes to employing several graduate students to work on this large project. Likewise, with this funding, he and his team will be writing numerous articles using the latest techniques on very large data sets, and can be expected to quickly become influential in that area. Because these graduate students have him both as an employer and as an academic adviser, when they graduate with their own doctorates, they will be experts at creating ways to detect bad standardized tests (after all, it’s what they’ve been doing for years), in a few years his influence on their careers will be apparent, and they will likewise go about working on similar problems — citing him and each other as they go along.

Believing that science is free of hidden agendas is like believing that politics is free of hidden agendas: that belief is an idea that completely ignores the reality that science, like politics, takes place among human beings.

Some Science Experiments Have Multiple Outcome Variables and Multiple Predictors

Following that, Mark gives a fair summary of the research proposal, before stumbling on a subtle but important point::

Is a normal classroom setting (say 20 to low 30′s of students) recording arousal during a 40-50 minute lesson with different student and teacher behaviors a good experimental setting where variables are identified, isolated and controlled? No. There’s hundreds, maybe thousands of variables in this environment and the researchers need to separate all the “noise” from the moment of learning. To say nothing about interruptions coming from outside the classroom (ex. fire drill, students entering, leaving, PA announcements) skewing the GSR readings.
Is it a reasonable assumption that the ideal teacher state of arousal for instructing students is the same or should even correlate with student arousal levels? No. This would seem to be a separate hypothesis to be investigated.

Given that the important parts of this post are that science works through iterative experiments to predict variation, and that the social enterprise of science depends on scientists responding to incentives, I hesitate to include the following point, but Mark’s comments bring it up.

Implicit in Mark’s comment is the idea of predicting a dependent variable from an independent one, or to put it another way, basic algebra in the format.

y = mx +b

With y as the predicted variable, x as the predictor, m and b as the intercept.  Students and trainee researchers sometimes used this exact form (which they would have learned as children in elementary algebra), because this form, the simplest of all scientific forms, is also the most advanced most laymen or reporters actually grasp.

More advanced research — the kind that has hundreds to thousands of participants — uses the almost identical form.

Y = MX + B

That is, more advanced research uses matrix algebra to allow for multiple outcomes, multiple predictors, multiple slopes, and multiple intercepts.

Conclusion

If you can put together more than one declarative sentence in a comment talking about a scientific study that leads to implications you are uncomfortable with, you have a firmer grasp of the scientific method than Diane Ravitch.