Has Perry Support Declined?

Some argue there has been no decline in polling support for Gov. Rick Perry among GOP primary voters. The chart above, and I, disagree. That looks like a bit of a dip at the end, from just barely over 30.1% to 27.1%.

But don’t take my word for it. Let’s check a completely independent source, RealClearPolitics.com:

The blue line is Perry. Looks to be down from about 32 to 28.4, according to RCP’s table.

So my trend estimator has taken a dip, RealClearPolitics’ moving average has also dipped.

So why then does The New York Times’ Nate Silver characterize my previous post as

One misconception is that Mr. Perry’s standing had been declining in primary polls prior to Thursday evening. This simply isn’t the case, at least not to any degree of statistical significance.

At first, I feared I had made an embarrassing mistake, but checking the arithmetic shows that the different conclusions between Silver and RCP and I have to do with the time perspective one brings to the data. And that turns out to be just a little more interesting than a mere statistical pissing match.

Silver comes to his conclusion by averaging the polls from Aug 13-Sept 6, and from Sept 7-Sept 19. Happily we both agree that RealClearPolitics.com’s list of national polls of GOP primary voters are the right polls to use, so our different conclusions don’t result from different sets of data. Silver finds the average Perry support before the first debate was 29.0 and after is 28.4 (also RCP’s current moving average.) That’s a change of just -0.6 percentage points, and we agree that is not statistically significant. (In point of fact, Silver uses a simple average, which ignores the differing sample sizes across the polls. If we correctly weight the polls to account for variations in sample size, the pre-debate average is 27.4 and the post-debate(s) average is 27.7, a gain of +0.3 and Silver would seem righter than he knew.)

So why do we differ in conclusions? Silver’s assumption is that the pre-debate polls are all homogeneous, with no time trend across them, and likewise for the post debate polls. If that were true, his approach to pre-post comparisons would be fine.

In contrast, my approach to estimating trends in polling, and the different but philosophically similar moving average approach of RealClearPolitics is that public opinion is dynamic and constantly moving. I and RealClearPolitics estimate trends that respond to these short term fluctuations in polling, rather than compare static pre-post groups. And that does lead to different conclusions about how Governor Perry has been doing recently.

Let’s look at a closeup of Perry’s polling since June 1 (there are few polls that include him before that date in any case.)

The gray line is my trend estimate based on all the polls. It is the same trend as the first chart of this post, except zoomed in to show more recent detail. The recent decline is numerically the same but looks far milder because of the different time scale, 4 months vs 30. The red lines are also trend estimates but calculated separately for the pre- and post-Perry entry data. The sharp jump after Perry enters is hard to capture instantly with any trend estimator, so the red lines provide an alternative view. As you can see, the red and gray trends nevertheless follow each other closely except right at the point Perry enters. The conclusion they offer about recent change is the same.

The critical point we see is that Perry support  was rising when he entered and his support continued to rise up to September 1. The red line shows a clear peak there, and the gray line reaches and then maintains a plateau.  The red line peak is a shade higher at 31.4%, but it falls off to 28.5% at the end, a decline of 2.9 points.  The gray line peaks at 30.1% and declines to 27.1%, a three point decline.

And there lies the difference between Silver’s analysis and mine. Perry entered the race on an upswing and continued to gain before his trend reversed direction just about September 1. Since then, he’s come down about 2.9 to 3.0 points. That isn’t a huge loss. He’s not in free fall. But he isn’t holding steady either.

What drives this are the four polls just before September 1. Two are quite high, above 35%. Another at 29% is as high as any previous poll. And one is rather low. But average them (or fit them with my trend estimate) and together they argue for a peak in Perry support. (The sample-size weighted average for the four is 31.78%, close to the peak of the red line at 31.4%.)  One could debate the two highest polls as outliers. Removing them would certainly change the trend. Sometimes outliers are indeed influential and a more conservative approach to the data would consider this but that issue wasn’t part of the difference of opinion that started this exchange since both Silver and I include these two high polls.

If we compare the last four polls before the first debate with the most recent four polls (three entirely after the 2nd debate and one that straddles the 2nd debate) we get an estimate of 27.06 (accounting for differing sample sizes), again quite close to my trend estimate of 27.1, and a decline between the two sets of four polls of 4.7 percentage points, a bit higher than my trend estimate.

My conclusion is that there has been a modest but clear decline in Perry’s support since the first debate. Whether you believe that or not depends on how you view the dynamics of polling. Accounting for short term change, as I do, clarifies the rising support for Perry before September 1 followed by an equal decline since then. If you prefer to treat the earlier polls as static and homogeneous, and likewise those after debates began, then you will agree with Silver’s conclusion that Perry is as popular with GOP voters now as he was before the debates began.

Horse races take at least two horses, so before ending let’s see Romney’s performance over the same period:

It is clear Romney has also enjoyed some dynamics, falling over 5 points but recently reversing that decline. From his low of 17.8% he has now climbed to 22.6%. During some of that time Perry’s rise was outpacing Romney’s but as of polling through September 19, the gap between Perry and Romney has closed to about 4.5 points. Perry remains the front runner.

At the end of my original post on this I suggested that the continued media focus on Gov. Perry’s debate troubles has helped mask continuing potential trouble for Gov. Romney. One reason Perry zoomed up so rapidly is that a sizable share of GOP primary voters are not satisfied with Romney and were happy to embrace a strongly conservative alternative. Perry’s problems don’t mean that lack of enthusiasm for Romney has changed. My analysis here still shows Perry in the lead and if no other strongly conservative candidate enters the race this may well remain the horse race of the year. Romney has yet to demonstrate he can win over those who were reluctant to support him four years ago even when paired against a candidate having some trouble adjusting to presidential debates.

Perry’s Lead Over Romney Falls After Debates

After two debates Texas Governor Rick Perry’s lead over Mitt Romney for the GOP nomination has fallen by a bit over 5 points. When Perry entered the race he enjoyed an immediate 15 point net bounce in his polling versus Romney. That moved Perry from 5 points behind Romney to 10 points ahead. This lead remained steady across nine polls prior to the GOP debate on September 7 at the Reagan Library sponsored by NBC and Politico. Following that debate there may have been a slight decline in Perry’s lead, but following the September 12 CNN/Tea Party Express debate in Florida, that lead clearly declined to slightly under 5 percent. That amounts to giving up 1/3 of the sharp gain after Perry entered the race, but still leaves Perry ahead of Romney and in a considerably better position than before he entered the race. After Thursday night’s third debate we will look for new polls to say if this decline continues or not.

The New York Times’ Nate Silver notes the decline (previously quite apparent here.) Silver however seems unsure of the evidence, saying “Whether Mitt Romney has gained ground on Rick Perry in the Republican primary race is questionable.” I’m unclear what is questionable about it. These are all national polls of GOP primary voters and there are 9 polls after Perry entered and before the first debate, and 7 polls after the 9/7 debate. While more polls are always desirable this is a pretty solid set of polls.

Initial media reaction to Thursday night’s debate suggests Perry again turned in a less than winning performance. So far, those negative media reactions appear to have also been the reaction of GOP voters. What remains to be seen is if the GOP’s somewhat reluctant embrace of Romney will ultimately help Perry, as Republican voters focus on a Perry-or-Romney choice. Romney had considerable trouble in 2007 and 2008 in winning over high church attendance GOP voters. Perry’s initial jump in support strongly suggests that GOP voters were not ready to embrace Romney prior to Perry’s entry. The uncertainty I think that remains is whether the problems elites see with Perry’s performance will overcome the reluctance of many GOP voters to accept Romney as the party nominee.

At the moment the focus on Perry’s claims in his book and on the campaign trail have helped divert a critique of Romney’s past weaknesses that haunted him in 2007-08. Ironically, Perry may be running interference for the criticism Romney might otherwise be suffering. And so far, there is no other credible GOP alternative.

Why Ames (apparently) doesn’t matter


(Addendum: The New York Times’ Nate Silver posted a very lengthy response to this. By all means check his arguments which should always be taken seriously. But in light of the recent trend in Bachmann support, I think I’m happy with my claim that the straw poll says little about future performance.)

After yesterday’s excitement over the Ames straw poll let’s take a look at the history and consequences. With Tim Pawlenty’s withdrawal today, it seems Ames does indeed matter, at least for negative consequences, and Michele Bachmann’s appearances on Sunday morning shows also hints at the upside for visibility and seriousness.

Yesterday, Nate Silver of the New York Times, presented his analysis of why Ames matters. His approach is to link the Ames straw poll to performance in the Iowa caucuses, and the results show that the two are certainly correlated, whether or not they are causally connected.  The results convince Silver that Ames is an important signal about a candidate’s likely success in the caucuses.

But the caucus is only the first event of the nomination process and so an alternative approach is to focus on the brass ring: the nomination itself. Is there any evidence that performance in the Ames straw poll has an effect on winning the GOP presidential nomination?

The short answer is “no”. And the longer answer is “beware of statistical analysis with very few cases”.

The chart above shows the predicted probability of winning the nomination based on national polling in the month before in Ames straw poll and whether or not the candidate won the straw poll. There is a strong and statistically significant effect of national poll standing (which is why the curves rise rapidly to near certainty for those polling well.) But the estimated effect for winning Ames is not statistically significant, and in fact is estimated to have a negative effect, if any at all. That is why the black line for winners is below the red line for losers.  The best conclusion is “no effect at all”.

In the figure, solid dots represent nominees while open circles are those who failed to gain the nomination. Straw poll winners who went on to secure the nomination (solid black dots) are Bush ’99 and Dole ’95. Straw poll losers (solid red dots) who became nominees are Bush ’87, Reagan ’79 and McCain ’07.

The straw poll winners who failed to gain the nomination are also revealing: Romney ’07, Gramm ’95 (who tied with Dole at the straw poll), Robertson ’87 and Bush ’79. The common trait is these all won the straw poll while standing below 10% support in the national polls at the time.  McCain ’07 with 16% poll support holds the record for lowest poll support of an eventual nominee, and he lost the straw poll.

Perhaps what is most revealing here is that the straw poll occasionally ratifies front runners, as it did with Bush in ’99 and Dole in ’95. But it often gives the nod to relatively weak candidates (from a national polling perspective) who command very modest public support. While attractive within the party, and effective at organizing in Iowa, these candidates had limited appeal in subsequent contests (or faced powerful opponents, as Bush ’79 learned when taking his “big mo” against Reagan after Iowa.)

If we plug in the current national polling numbers to the model for Michele Bachmann, Rick Perry and Mitt Romney, we see that Bachmann is given little chance at the nomination, at least according to the model. Her predicted probability of winning the nomination, based on 13% in national polls and a straw poll win is .023, or 2.3%. (She has fluctuated in polling recently and fallen from her high point of the last month, so the current 13% may or may not adequately reflect her potential support. Her current trend estimate is actually lower, at 8.7%. All candidates in the model are measured by the same standard–the most recent available national poll taken before the straw poll.)

Rick Perry has, of course, only just entered the race so his polls may be highly variable. The latest put him at 17%, yielding an probability of nomination of .114, or 11.4%. Perry’s polling has been rising rapidly with a trend estimate at 16.4, essentially the same as his latest poll.

Finally, Mitt Romney remains the polling front runner with 24%, giving him an estimated .363 probability of nomination. His trend has turned down recently and now stands at 19%.

Both Perry and Romney outpace McCain ’07 at this point, but stand well behind Reagan ’79 or Bush ’87, two other straw poll losers, and far behind Dole ’95 or Bush ’99 who won the straw poll while also polling near or over 50%.

It is worth point out that national polls are no guarantee of success either. Ask Giuliani ’07 or Dole ’87.

The simple conclusion then is that there is no evidence that straw poll success increases the likelihood of winning the GOP presidential nomination. Strong polling among Republicans nationally is far more powerful, even if certainly not a guarantee.

If there is a straw poll effect it seems more likely to be short term and tactical rather than long term and of strategic value. Pawlenty has given up. Bachmann is and will be in the spotlight for a while. But the short term effects show little evidence of carrying over to long term success in the nomination chase.

(I find a slight improvement in prospects of winning the Iowa caucus among straw poll winners, which is consistent with Silver’s finding based on percentage votes. But when predicting nomination success with a model similar to the one presented here, winning the Iowa caucus has no effect on nomination probability, once national poll standing is incorporated in the model.)

Non-geeks may want to stop here.

The longer answer is to notice how poor the data are for answering the question at all. The black line for winners has no cases along it from Bachmann at 13% to Dole at 48%. In contrast, the red line for losers is pretty well populated in this range.  What that means statistically is we have no information for estimating the effect of  winning the straw poll for anyone between 10% and 48% national support. The black line then is estimated based on the four winners who all stood below 10% and the two winners who were over 48%. (Bachmann doesn’t count because we don’t know the nominee yet, obviously.) It is the mathematical shape of the curve that provides the leverage for estimating the black line between 10% and 48%, and that is thin gruel indeed. The red line in contrast enjoys several cases in this range, and while it too has substantial uncertainty in outcomes it is far better grounded in data than is the curve for straw poll winners.

Given these limitations of data which the real world has provided, the best conclusion remains that we don’t have any evidence that the straw poll provides a boost in likelihood of winning the nomination, once national polls are taken into account.  Anecdotally, it is revealing to ponder the division between straw poll winners who are very low in national polls and winners who are very high in poll, with no one in between. It seems dark horses have a real shot in the straw poll, but not so much at the nomination.

P.S. And Joshua Tucker cuts to the chase without making it excessively complicated with just two bars.