ARBITRON: TWIN HOUSTON PANELS PRODUCE SIMILAR RESULTS Paul Heine – Billboard Arbitron has declared victory in its twin panel study of the Portable People Meter. Randomly selected PPM audience panels in ongoing Houston trials “produce ratings that reliably reflect media use in the market,” the company announced Thursday.
Broadcasters had expressed concern about converting from a diary-based sample that turns over every week to a panel where respondents remain in the sample for much longer periods of time. “Some broadcasters wondered if the use of panels would create a skewed view of radio listening in the market since the panel sample stays in place for a long time once installed, while in the diary system there is a new sample every week,” senior VP, PPM marketing Jay Guyther said.
Radio, via the Arbitron Advisory Council, asked for a side-by-side comparison of the ratings produced by two separate randomly selected panels.
As it selected sample in Houston, households were randomly assigned by Arbitron's sample supplier to either panel A or panel B to establish two independent panels.
Comparing the ratings produced by the two “replicates,” Arbitron found:
No statistically significant differences in sample composition based on race/ethnicity, language preference, and presence of children, employment status and other variables.
No significant differences in the compliance profiles of the panels, including factors such as daily undocking and docking times and hours the meter was used each day.
No significant differences in overall radio listening levels.
However, there was evidence of the importance of language weighting within the Hispanic sample, which will become part of the PPM standard sample balancing process starting in Winter 2006.
Arbitron says the panels produced similar station-level ratings. Comparing 2,654 station-level AQH ratings between the two panels, the company found that 77% were within 0.2 rating point, 93% were within 0.5 rating point and 98% were within 1.0 rating point.
Overall, 3% of the differences in station-level ratings were statistically significant. That's less than the 5% Arbitron expected with a 95% confidence level.
”This is an important finding because it says that observed ratings differences between the panels are most likely due to sampling variation and other random effects and not large systematic differences between the two samples,” Arbitron said in a press release.
The test also showed that PPM produces reliable station ratings at sample sizes much smaller than the one PPM for every three diaries Arbitron plans to use. (Should Arbitron commercialize PPM in Houston next year, it would install a sample of 2,000-person age 6 and older in the DMA.) But since PPM respondents remain in the sample far longer than diary respondents, the smaller number of panelists actually increases the number of observations taken from each respondent, thus improving ratings reliability. |