Parametric vs. Non-Parametric analyses


Agree or disagree:

One should always use non-parametric analyses in place of parametric; they are not weak to the violation of assumptions.


Great question. Can you define non-parametric analyses for those of us not familiar with them?


Sure I can! Although to be honest, if you’re not familiar with them, you won’t really be able to tell me whether or not they’re better than parametric tests. ; )

But it’s always nice to share and educate.

When we ask quantitative questions (e.g., age, responses to surveys on a Likert-type scale, amount of dollars spent on in-app purchases) we will often want to know about differences between groups. Some examples are:

  1. Do payers enjoy our game significantly more than nonpayers? Or do they both enjoy it equally?

  2. Are women more likely to use the chat feature in our app than men?

  3. We’ve released five different iterations of a feature to closed Betas… which version was the easiest for our players to navigate?

Most of the time, the numbers we get can be turned into averages and these averages tested against one another with a parametric test, like t or an ANOVA. These tests are powerful (able to find differences between groups when those differences exist) but they require certain assumptions to be met. If those assumptions are violated, the tests can be weakened or even uninterpretable.

In the case where we’re interested in the same types of differences, but the data are counts (“how many”) or rank-orders, we use non-parametric tests. These are the equivalents of the tests I mentioned above - there’s a non-parametric t test (the Mann-Whitney U) and a non-parametric ANOVA (the Kuskall-Wallace H). Those tests are a little weaker (they’ll sometimes miss a significant difference where one exists), but they ignore all assumptions. So they could conceivably be used for ALL tests, regardless of the data, in order to circumvent the concern of violating assumptions.

My colleague suggests that for this reason, all tests performed should be non-parametric, because they will never be wrong, regardless of the shape of your data. I’m not sure how I feel about this, especially the idea of blanket-applying the word “always” when parametric tests get the job done well (even better) for most data and are often robust against the violation of assumptions.

… And now I realize that I should have just spent this time perusing the Google results for “should I use nonparametric tests?” LOL. But I hope I’ve at least been educational.



It’s always good to post here regardless. Someone in the future that knows their way around parametric analysis may drop in with some new knowledge, but if not, we’ve all learned something from you, which is a huge part of why we’re here. :slight_smile: