What happens when your model errors aren't normally distributed?
If the kurtosis is high, p-values are over-stated. If fat-tailed then p-values are understated.
Why? Because the likelihood of your p-value isn't guaranteed to be normally distributed.
Normal is a nice assumption but asymptotic can take a long time to kick in. The CLT is beautiful analytically, but fortunes are made from people who assume it.
> What happens when your model errors aren't normally distributed?
Honestly...? You're screwed. At least in Bio, where most researchers haven't taken calculus, most folks will screw up the t-test or their ANOVA if you are not super careful. For non Gaussian data you better pray it's Poisson or has some other exotic name that you can at least google.
Especially with low N, you just kinda pray it's normal and then you go and try and get grant funding with those results.
Cynically, in the end, it barely matters. It's all about that grant money. Whatever way you can tease that data to get more grants, you just do that. No-one ever checks anyway (Statcheck excepted)
If the kurtosis is high, p-values are over-stated. If fat-tailed then p-values are understated.
Why? Because the likelihood of your p-value isn't guaranteed to be normally distributed.
Normal is a nice assumption but asymptotic can take a long time to kick in. The CLT is beautiful analytically, but fortunes are made from people who assume it.