By Kent R. Kroeger (NuQum.com, July 8, 2019)

Growing up in America’s farm belt, weather proverbs were commonly heard and taken seriously. The one I always remember I first heard from my grandmother: “Frogs croaking on the lake, means an umbrella one must take.” Or something like that.

She didn’t live near a lake, so I’m not sure how useful that piece of folk wisdom was for her, but it stuck with me. And, as it turns out, the proverb has some basis in fact. Frogs do croak more on hot, humid days — which is a good predictor of stormy weather.

But the way my grandmother used the proverb, or at least how my child’s mind interpreted it, I believed for years that croaking frogs caused thunderstorms. Croaking frogs, of course, do not have such power.

Years later, I would realize my grandmother offered me my first lesson in spurious correlations, and I’ve used the croaking frogs proverb in statistics classes many times since.

The lesson is one most people hear many times during their education: Correlation is not causation. Two events (x and y in the graphic below) can be statistically correlated but not be causally related, as they are both impacted by the true causal factor (z).

Frogs are affected by the same forces — temperature, humidity and air pressure — that cause thunderstorms. There is no causal relationship. To this day, I still call these relationships croaking frogs.

We may have a new example of this inferential deficiency concerning an analytic question of current importance: Did meddling by Russia’s Internet Research Agency(IRA) impact the final outcome of the 2016 election?

Four University of Tennessee researchers, Damian J. Ruck, Natalie Manaeva Rice, Joshua Borycz, and R. Alexander Bentley, have concluded, based upon a time-series analysis of IRA tweets and their diffusion within the Twittersphere during that election, that IRA Twitter activity predicted the 2016 election results. In their study released in July, they concluded:

“We find that changes in opinion poll numbers for one of the candidates were consistently preceded by corresponding changes in IRA re-tweet volume, at an optimum interval of one week before. In contrast, the opinion poll numbers did not correlate with future re-tweets or ‘likes’ of the IRA tweets. We find that the release of these tweets parallel significant political events of 2016 and that approximately every 25,000 additional IRA re-tweets predicted a one percent increase in election opinion polls for one candidate. As these tweets were part of a larger, multimedia campaign, it is plausible that the IRA was successful in influencing U.S. public opinion in 2016.”

The Washington Post’s Philip Bump wasted no time in challenging the Ruck et al. study for its failure to account for other causal factors that may have acted on both IRA Twitter activity and Trump’s public support:

“It’s important to note that the researchers focused on retweets and not overall tweets from the IRA. (In fact, they found that “we see weak evidence for an effect in the opposite direction, suggesting the possibility that IRA Twitter activity is increasing in response to Trump’s polling.”) This suggests that, if there was a meaningful correlation between Twitter activity and poll data, both were driven by some outside engagement. People becoming active on Twitter also may have happened as they were demonstrating more support for Trump. This is what’s known as a causal fork: Both the IRA retweets and Trump support may have been caused by the same external thing. If there’s a correlation here, that is. Which is . . . up for debate.”

Bump also noted that the magnitude and targeting of IRA’s Twitter and Facebook activity was not large or precise enough to plausibly move public opinion:

“It’s important to note that, on its face, the idea that 25,000 retweets could drive national political polls by a percentage point seems highly unlikely. Over the course of the 2016 election, there were 75 million tweets directly related to the election itself. If only 1 percent of those were retweeted 10 times, that means that the 25,000 retweets are fitting into a flood of 75 million original and 7.5 million retweeted tweets. It means, in other words, that the requisite 25,000 retweets make up 0.03 percent of all of that Twitter activity.”

“There’s very little evidence that Russia effectively targeted American voters with messages that powered Trump’s victory. Russia paid for a lot of Facebook ads in the populous states of New York and Texas in the last five weeks of the campaign, but its ads targeting the three states that handed Trump the election — Michigan, Pennsylvania and Wisconsin — were seen by only 1,000 people. There’s no evidence at all that Russia used Twitter to target people in particular places or demographic groups, targeting that would have left fingerprints in the form of receipts for payment.”

To the credit of the University of Tennessee researchers, they acknowledged the limitations of their study when they write, “Causation is not proven by this analysis, but certain directions of causality can be ruled out when one time series does not predict the other…We take the view that IRA Twitter activity was representative of a larger, multimedia disinformation campaign.”

Ruck et al. also write that their intent was to test “prediction, not causality,” as they admitted it is unlikely that “25,000 retweets could influence one percent of the electorate in isolation.” And, most appropriately, they recognize their study cannot rule out the importance of unmeasured factors that could render their findings spurious. They write in the study’s concluding section:

“Any correlation established by an observational study could be spurious. Though our main finding has proved robust and our time series analysis excludes reverse causation, there could still be a third variable driving the relationship between IRA Twitter success and U.S. election opinion polls. We controlled for one of these — the success of Donald Trump’s personal Twitter account — but there are others that are more difficult to measure; including exposure to the U.S domestic media.”

This is where the Ruck et al. research makes its biggest analytic error. What they call the ‘third variable’ is probably a set of variables — unmeasured and uncontrolled for in the Ruck et al. study — that, had they been included in the study, would likely washout the statistical significance of the IRA retweets.

By the Mueller investigation’s own estimate, IRA spent $100,000 between 2015 and 2017, with only $46,000 dedicated to Russian-linked Facebook ads purchased prior to the 2016 election. According to freelance journalist Aaron Maté, “That amounts to about 0.05 percent of the $81 million spent on Facebook ads by the Clinton and Trump campaigns combined — which is itself a tiny fraction of the estimated $2 billion spent by the candidates and their supporting PACS.”

There is, however, an obvious candidate for the honor of being the “larger, multimedia disinformation campaign” Ruck et al. consider as the more likely driving force behind the “manipulation” of the 2016 electorate. That third variable is the Trump campaign’s social media campaign, powered by Cambridge Analytica’s massive data warehouse, which included data harvested from over 50 million Facebook user profiles.

Unlike IRA’s use of Twitter and Facebook, where the hard evidence shows little sophistication in both content and targeting, Cambridge Analytica engineered one of the most sophisticated Big Data-driven social media campaigns in presidential history.

In an interview with CNBC, the 2016 Trump campaign’s digital director, Brad Parscale, detailed how his team, including Cambridge Analytica, created highly targeted Facebook advertising based on scientific testing to optimize each advertisement’s click rate. “We were making hundreds of thousands of them (ads on Facebook) programmatically. … (On an) average day (we would make) 50,000 to 60,000 ads, … changing language, words, colors, changing things because certain people like a green button better than a blue button, some people like the word ‘donate’ over ‘contribute,’” Parscale told CNBC.

Just on scale, IRA’s efforts pale in comparison to Parscale/Cambridge Analytica’s. Add to that the much higher level of campaign sophistication by Parscale/Cambridge Analytica, and it begs the question, how could any serious research on the impact of Russian meddling in 2016 not include measures of the Trump’s campaigns social media efforts (and Hillary Clinton’s as well for that matter)?

Ignoring Cambridge Analytica’s social media campaign, Ruck et al. have given us a croaking frog-level analysis. It is as if the Ruck et al. research team, while sitting in a small row boat on a lake, experienced a large wake and attributed it to an 8-meter motor boat passing by, ignoring the fact that a 100-meter, 6,700 gross tonnage yacht passed by at the exact same time.

While their forthrightness on their study’s flaws is admirable, Ruck et al. have not done the measurements and work necessary to release any meaningful results on a subject as politically volatile as Russia’s influence on the 2016 election. As it stands today, their study offers little to the conversation.

• K.R.K.

Comments and criticisms can be sent to: kroeger98@yahoo.com