How Science Works

Diane Ravitch is a labor agitator with a focus on the education sector. She is interesting to read if you care about union politics as it applies to education from a teacher’s (as opposed to student’s, parent’s, or nation’s perspective). As such following Diane Ravitch on twitter has the same sort of twisted excitement as, say, following a NAMBLA report on age-of-consent laws: they are definitely interested in the subject area, but for all the wrong reasons.

(I’m not sure if the effect of an individual child from a terrible education is worse than the effect of rape at a young age, though I suspect it is.)

In any case, Ravitch recently linked to a blog post, titled “Regents agree to give NY students data to limited corporation run by Gates and operated by Murdoch’s Wireless Gen.” The thrust of the article is that the not-for-profit Gates Foundation is providing funds to use test data to build more effective teacher assessments. The Gates Foundation wants to improve teacher quality, so that makes sense. And Ravitch is a labor agitator, so of course she is against this. So far, so good. Presumably drill-fitters would be against measuring the productivity of drill-fittings to a similar degree.

My friend Mark Safranski was curious about the story thouge. Mark mentioned, I don’t know if their data analysis is valid or reliable. Sounds like they don’t have a model yet but have a contract, and asked Why would this be figured out on the fly?.

The reason a model would be “figured out on the fly” is that this is how science works.

There is a cartoon version of science along these lines

Science is a method for understanding Truth. To understand Truth, a great scientist thinks deeply, and using the tomes he has read along with his powers of reflection, generates a Theory. Then, with great care, an elaborate contraption is created test the Theory. If the test works, the Hypothesis is Proven, and it becomes a Fact. Otherwise, the Theory is Wrong, and the cycle begins again.

This is a ridiculous view of how science works, and that Americans believe it is on prima facia evidence that our education system is deeply broken.

Science is a method for predicting variation. To better predict variation, scientists construct Theories, which are mental models that allow brute facts to be put in some sort of framework. For instance, the theory of Gravity explains the brute fact of an apple on a tree in one moment, and the same apple on the great in the next, into a narrative. Theories are opertaionalized using hypotheses, which generate specific predictions. So gravity on Earth can be operationalized as predicting that gravity acts like an acceleration that forces all object to the ground at a rate of 32.2 feet per second per second. Run enough experiments and you will begin to see this simple hypothesis mis-predict events, which will force you to generate other hypotheses. Eventually you will have a set of hypotheses which predict events enough to be useful to you.

Many individuals who hold the first view, on hearing the news about building a database of student test scores, would be confused. What is the Theory being tested? Why won’t the Great Scientist tell us? How will we easily know if he has discovered Truth, or he is Wrong?

Because this is not how science works, people who think this way believe in Pseudoscience.

Instead, actual scientists would be approach the work in a different manner. These scientists want to know what types of education best prepare students for life outside of school. They look at outcomes, such as health, income, years in college, degrees earned, social class, criminal convictions, and so on. To recognize that to the extent that ‘success’ exists, it is a latent construct that is only imperfectly, and with error, reflected in any one of these measures, using a Theory. These scientists also look at how one can objectively measure student achievement & teacher quality (two different things, certainly), also using Theories. A simple hypothesis is created, and tested on some data. Run enough studies, and the scientists will begin to see how this simple hypothesis mis-predicts events, which forces them to create more hypothesis. Eventually, the scientists have a set of hypotheses which predict student success enough to be useful to policy makers.

It is this process that Diane Ravitch and other labor agitators are deeply opposed to. A world where teacher quality can be assessed is one where bad teachers might be forced to become good teachers or be fired. This breaks worker solidarity, and means that some teachers will work harder, when any good labor organization wants to make working conditions easier for its members.

I believe in student welfare — and having a strong nation — more than I believe in labor agitation, so I support education reform, and oppose pseudoscience.