Research surveys in market research are typically weighted to bring the sample distribution into line with the known population distribution. Weighting is designed to help overcome sampling bias and therefore improve the accuracy of the results.
However, if the variation in the weighting factors results in extreme weighting, and statistical significance tests are then carried out, this can lead to an overestimation of statistical significance. In Harmoni, the option to use the effective base is available to resolve this overestimation.
The effective base is available in Harmoni via personalization keys.Keys that can be applied at the user, site (all users), or company (all users) level.
In this article
1. What is extreme weighting?
Extreme weighting is when weighting factors vary a lot, that is, some respondents are greatly weighted up and others are consequently weighted down. The results may become less accurate when extreme weighting occurs.
Let's assume we have a sample of 1000 respondents to represent males and females in the population. But the sample is heavily skewed with 900 of the respondents being female and 100 male. To bring the sample into line with the population, the 100 males are weighted up to 500 (a factor of 5) and the females down to 500 (a factor of 0.55). So each male respondent has almost 10 times the influence of each female respondent in the final result.
2. Effective Base
In statistics, effective base is used as a safeguard against making statistical conclusions from a sample that has been drastically adjusted using weights to match target values. Using the effective base is considered a more conservative approach, but it provides accurate statistical results for weighted data.
The effective base indicates how much statistical power is lost by weighting. The closer the effective base is to the unweighted base, the better the weighting is.
Effective base = (sum of weighted base) squared divided by the sum of the squared weighted base.
Currently the effective base is not available as a calculation type in Harmoni, even when the effective base is set for use in statistical testing.
Where to from here?
Learn more about statistical testing in Harmoni.
- Statistical Significance Tests
- Multi-Reference Statistical Significance Tests
- Statistical Significance in Harmoni