The Detroit Push-poll

Courtesy of Gas 2.0 and Hybrid Car Blog, I found the Peter D. Hart Research Associates poll on a bailout for auto-makers.

It is a push-poll.

The poll sensible begins with demographic information, and as expected with a poll of “landlines” is demographically skewed. The modal age group is 30-34 yera olds, with 35-44 year olds being dramatically underrepresented. The poll undercounts both blacks and hispanics, and has a shows more respondents as indicating being black than hispanic.

Question 4 begins the substance of the poll, and asks

How important do you feel the American automobile industry is to the American economy–extremely
important, very important, somewhat important, not important, or not at all important?

Sensible enough. However, consider the next two questions:

If the American automobile industry no longer had the resources to produce vehicles, how much harm would
it cause to [America’s manufacturing job sector / The American economy / America’s standing in the world / Consumer choice for America’s car buyers ] a great deal of harm, quite a bit of harm, just some harm, or very little harm?

And Question 6, which is the first of the serious of “do we bail out Detroit” questions:

Do you believe that the government should or should not provide loans to America’s automakers so they have the money to manufacture vehicles?

Question #5 is what we call a “Prime” It is a stimulus that changes what responses will be elicted. So for instance, if you ask a question emphasize harm to the American economy, and then ask about things that might reduce that harm, you of course get resposnes skewed to “doing something.”

In the same way, if Question #5 had been

If the American government began bailing out industries that were no longer able to pay their bills because they had made unwise business decisions, how much harm would
it cause to [America’s manufacturing job sector / The American economy / America’s standing in the world / Consumer choice ] a great deal of harm, quite a bit of harm, just some harm, or very little harm?

The answers would have been dramatically different.

There is an easy way to correct this problem: randomly vary the order in which questions are asked. Then you can examine what effect, if any, your prime has. If it has an effect, then you can be safe by reporting the unprimed result: if it does not have an effect, so much the better. However, the method section…

The accompanying poll results are from a survey conducted by the polling organization of Peter D. Hart Research Associates for General Motors on November 11 and 12, 2008. The survey was conducted by telephone among a cross section of 804 American adults.
The national sample for this poll was drawn in the following manner: 350 geographic points were randomly selected proportionate to the population of each region and, within each region, by size of place. Individuals were selected in accordance with a probability sample design that gives all landline telephone numbers (both listed and unlisted) an equal chance to be included. One adult, 18 years old or over, from each household was included, selected by a systematic procedure to provide a balance of respondents by sex.
The data’s margin of error is ±3.5 percentage points for 804 adults at the 95% confidence level. Sample tolerances for subgroups are larger.

… provides no indication that this was done.

The poll, titled “Study #8877: Auto Industry Survey” displays results in percentages, but does not appear to indicate how many participants kept answering questions. It people surveyed were against the bailout and recognized the biased nature of the questions, they may have ceased responding, causing the sample to skew to those who are in favor of a bailout anyway.