R-WEB Samples with characteristics similar to Simpsonparadox
This web application visualizes the effect of the Simpson's paradox in the context of comparing means resulting in statements about the effectiveness of a treatment X on the value of a dependent variable Y. It is an alert to draw causal conclusions from this comparison. While the treatment seems to have a specific overall effect, the conditional regression, where Z is included as a second group variable, shows the opposite direction. This problem could occur when the base probability to be part of the treatment or the control group, is not equal for each subject. The subsamples of Z have different qualities and different probabilities to be in the treatment group, so the group specific effect could differ from the effect observed over all.
An example for better understanding
To test the effectiveness of a math training program, we take a sample of pupils. Half of them take part in our math program and the other pupils get no training and act as the control group. After that we measure the math ability of each participant. Then we want to draw conclusions about the effectiveness of our treatment by comparing the average score of the two groups. Observing significant higher average tests scores in the treatment group than in the control group leads to the conclusion that the treatment has a positive effect on the math ability. However, our participants differ e.g. in gender and therefore one could be interested in differential effects of the treatment. Surprisingly the group specific analysis divided by gender shows negative effects of our treatment in both groups! How could this happen? The reason is the unequal distributed groups. Maybe the boys are more interested in maths, so it is more likely that they join the treatment group, while the girls mostly dislike maths, resulting in an higher probability to be part of the control group. Because of their general higher interest the boys perform better in the test than the girls, even when the training has got a negative effect. This could be observed in the group specific analysis. Following this argumentation presented here we would recommend to pay attention when drawing causal inferences in experiments with non-randomized design!
Try to find a data constellation which results in the Simpson's Paradox
Define the sample! Try to create Simpson's Paradox
© 2008 Marie-Ann Milde and Sven Hartenstein, suggestions for improvement are welcome