Polling accuracy has been frequently discussed after most polls underestimated Trump's performance in 2016 and 2020. Some have argued that this is a result of systemic issues that are difficult to address, such as non-response bias. Others have suggested that while it's possible there are systemic issues the bias is unlikely to be consistent over time due to pollsters adjusting after big misses.
Trafalgar Group is a polling firm that does polling for state and national elections. They have been criticized by some political forecasters as inaccurate or biased toward conservatives. However, after the 2020 election FiveThirtyEight upgraded Trafalgar from a C- pollster rating to an A-, and in a podcast episode Nate Silver apologized to Trafalgar for prior off the cuff remarks.
If there is a systemic polling bias that underweights the chances of Republican candidates, one possible test might be to compare the performance of the FiveThirtyEight polling average in 2022 to the performance of a polling firm which some view as right-biased, like Trafalgar Group.
Will Trafalgar Group outperform the 538 polling average in the 2022 congressional and gubernatorial elections?
Resolution will be determined by comparing the performance of the most recent poll conducted by the Trafalgar Group for each of the Senate, House, and Gubernatorial elections in the 2022 elections taking place on November 8, 2022, to the 538 polling average on the day after the final day of the corresponding Trafalgar poll. The difference between the top two candidates by final vote share will be compared to the difference between those candidates in the FiveThirtyEight polling average and final Trafalgar Polls. The average absolute value of the difference between actual result and polling will be calculated for each. If the Trafalgar Group average difference is closer to zero than that of FiveThirtyEight then this resolves positively.
Races where Trafalgar Group does not produce a poll within one month of the election date will not be included in the average. In the event the average difference for FiveThirtyEight and Trafalgar Group is the same this resolves ambiguously.
Resolution Example Using Fake Numbers:
Suppose for three races the final results were as follows, where positive values mean the Democrat was ahead of the Republican in the final results and polls. TG stands for Trafalgar Group.
|Race||Final Result||TG Final Poll||TG Error||538 Polling Avg||538 Error|
Based on the above the average error of the Trafalgar Group would be 1.77 and the average error of the FiveThirtyEight polling average would be 0.90. If the example were all of the elections being counted this question would resolve negatively since the FiveThirtyEight error is closer to zero.