Tag Archives: Mark Safranski

The Search for Academic Utility

Over the past few weeks I talked a lot about “paradigms.” Paradigms are “research programs” that focus on a few exemplars of high quality work. This allows science to make progress, and breaks up “old boy” networks by privileging results over connections. The need for progress also allows students to have better lives after they graduate. Professors, like all people, crave money, power and respect.

Thus, normal science, paradigms, research programs, and exemplars align our the need for progress, student’s need for good lives, and professor’s need for professional accomplishment. This is how academia works. Science is not a cartoon. It is a great human achievement that gets human beings to predict, control, and improve variation of the objects scientists study.

Normal Science is good because it is useful, not because it is True.

Consider three areas of work: race-based explanations for school performance, my UFO theory, and the ancient astronaut theory of Great Pyramid construction. If I had to bet, I would bet a great deal of money there’s a very strong impact of race in academic performance not explainable by income, I would bet a small amount of money the aliens at Roswell were from Japan, and I would bet against someone claiming that the logistics of Great Pyramid construction was designed by creatures from beyond the Moon.

ancient_ufo

But attempting to found an academic career on either of these theories would lead to a failure to gain tenure. The reason is all are currently outside of the normal science in educational psychology, digital humanities, and Egyptology. None of these theories are currently useful in the field, so none are pursued.

ways_of_knowing_0

Normal Science is just part of Science, which along with Inquiry are two of the ways of knowing about the world. There’s more to this world than captured in data sets. My friend Mark Safranski recent captured readers after linking to a data set, stating “ there are hidden qualitative decisions in who did the counting, how and by what yardstick.” Indeed, Normal Science has even more limitations than that.

A lot of grief is caused by considering Science the search for Truth. It may that, but Normal Science is the search for utility in an academic context.

How Science and Academia Work

Summary

  • Science is not a cartoon
  • Science works by predicting variation
  • Science advances by exploiting human nature
  • Some science experiments have multiple outcome variables and multiple predictors

Science Is Not A Cartoon

The cartoon version of “science” — that definition that teachers who had no idea what they were talking about gave you — runs along these lines:

Science is a method for understanding Truth. To understand Truth, a great scientist thinks deeply, and using the tomes he has read along with his powers of reflection, generates a Theory. Then, with great care, an elaborate contraption is created test the Theory. If the test works, the Hypothesis is Proven, and it becomes a Fact. Otherwise, the Theory is Wrong, and the cycle begins again.

Of course, that’s ridiculous. That’s not science. That’s what children think of as science.

Science Works by Predicting Variation

Here’s a better definition of science:

Science is a method for predicting variation. To better predict variation, scientists construct Theories, which are mental models that allow brute facts to be put in some sort of framework. For instance, the theory of Gravity explains the brute fact of an apple on a tree in one moment, and the same apple on the great in the next, into a narrative. Theories are operationalized using hypotheses, which generate specific predictions. So gravity on Earth can be operationalized as predicting that gravity acts like an acceleration that forces all object to the ground at a rate of 32.2 feet per second per second. Run enough experiments and you will begin to see this simple hypothesis mis-predict events, which will force you to generate other hypotheses. Eventually you will have a set of hypotheses which predict events enough to be useful to you.

I say this because of a recent post by Diane Ravitch (who was recently fired from the Brookings Institution), criticizing a Gates Foundation grant to measure attentiveness through measurement of human the electrical system.

If you know what science is, and how it works, your immediate thoughts should be.

Scientists desire to predict variation in educational outcomes. These scientists doubtless have Theories of education, which are mental models that allow brute facts to be put in some sort of framework. These theories are probably operationalized using hypotheses, which have generated specific predictions. There probably is error in the these predictions, which are leading to follow-up hypothesis. These scientists must think by adding information on attentiveness measured through the electrical system, they can reduce error, and predict educational outcomes better.

If you know nothing about science, such as Diane Ravitch, your reaction differs, you’ll write a nonsensical post with only one declarative sentence: “Shades of Brave New World.”

Science Advances By Exploiting Human Nature

Now, given that, try to understand the study, as the historian Mark Safranski did, in this way:

 

Let’s start from the assumption that this GSR bracelet study is actually a scientific study without hidden agendas.

But before the end of his first sentence, Mark (who unlike Diane, is attempting to seriously engage in this issue) is already lost on irrelevant tangents.

Why would science be free of “hidden agendas”? Why would scientists be some cold automatons driven by computer programs with no feelings, emotions, hopes dreams, or goals? Science advances through Academia. This is done by rewarding professors for obeying the interests of peer-reviewed grant funding agencies.:

Professors, like most people, respond to the incentives of power, influence, and money.

The institution of tenure reduces uncertainty regarding money, and focuses the incentives on power and influence.

Power in academia comes from the number of bodies a professor has under him. These bodies might be apprentices (graduate students he advises), journeymen (post-docs who have a PhD and work at the lab, or staff researchers), or simple workers (lab technicians, etc).

Influence in academia comes from the extent to which one is successful in influencing one’s peers. This is typically measured in terms of influence scores, which are a product of how often the academic is cited, weighted by how important of a publication he is cited in.

The best route to both power and influence is to earn grant money. For example, consider a professor who receives grant money from a federal agency. Some of this money goes to equipment, but the majority goes to employing several graduate students to work on this large project. Likewise, with this funding, he and his team will be writing numerous articles using the latest techniques on very large data sets, and can be expected to quickly become influential in that area. Because these graduate students have him both as an employer and as an academic adviser, when they graduate with their own doctorates, they will be experts at creating ways to detect bad standardized tests (after all, it’s what they’ve been doing for years), in a few years his influence on their careers will be apparent, and they will likewise go about working on similar problems — citing him and each other as they go along.

Believing that science is free of hidden agendas is like believing that politics is free of hidden agendas: that belief is an idea that completely ignores the reality that science, like politics, takes place among human beings.

Some Science Experiments Have Multiple Outcome Variables and Multiple Predictors

Following that, Mark gives a fair summary of the research proposal, before stumbling on a subtle but important point::

Is a normal classroom setting (say 20 to low 30’s of students) recording arousal during a 40-50 minute lesson with different student and teacher behaviors a good experimental setting where variables are identified, isolated and controlled? No. There’s hundreds, maybe thousands of variables in this environment and the researchers need to separate all the “noise” from the moment of learning. To say nothing about interruptions coming from outside the classroom (ex. fire drill, students entering, leaving, PA announcements) skewing the GSR readings.
Is it a reasonable assumption that the ideal teacher state of arousal for instructing students is the same or should even correlate with student arousal levels? No. This would seem to be a separate hypothesis to be investigated.

Given that the important parts of this post are that science works through iterative experiments to predict variation, and that the social enterprise of science depends on scientists responding to incentives, I hesitate to include the following point, but Mark’s comments bring it up.

Implicit in Mark’s comment is the idea of predicting a dependent variable from an independent one, or to put it another way, basic algebra in the format.

y = mx +b

With y as the predicted variable, x as the predictor, m and b as the intercept.  Students and trainee researchers sometimes used this exact form (which they would have learned as children in elementary algebra), because this form, the simplest of all scientific forms, is also the most advanced most laymen or reporters actually grasp.

More advanced research — the kind that has hundreds to thousands of participants — uses the almost identical form.

Y = MX + B

That is, more advanced research uses matrix algebra to allow for multiple outcomes, multiple predictors, multiple slopes, and multiple intercepts.

Conclusion

If you can put together more than one declarative sentence in a comment talking about a scientific study that leads to implications you are uncomfortable with, you have a firmer grasp of the scientific method than Diane Ravitch.