Jump to content

Statistics question: Should statistics lecturers give the formal definition of p-value


chuck norris 42

Recommended Posts

Sorry for the narrow technical topic but I know there are few people with a strong stats background on the board.

I would claim a statistics lecturer should not discuss p-value unless students understand how to work out the distribution of the test statistic. If  students
that do not have a complete understanding of the distribution of the test statistic, only a rough understanding of p-value  is possible. I think you should just roughly explain that a probability
can measure evidence ,

for example if you have a sample  of 10,000 companies and only 1  has a managers that is a teenager then the proportion of managers that are teenagers is only 1/10000=0.0001
So if you know someone is a manager you doubt they are a teenager, This argument can be understood without explaining what the probability actually means, trying to explain what the probability actually means could only cause trouble.

Link to comment
Share on other sites

It depends on how many managers there are on average for those 10,000 companies. If each company has 10 managers, and theres only 1 teenage manager at one company, than the proportion of managers that are teens is an even more miniscule 1/100000=0.00001 a mere tenth of the amount from your example.

We dont know the proportion of managers that are teenagers from your example because youve only gave the number of companies not the number of  managers.

Link to comment
Share on other sites

It depends on who is being taught. P-values are generally not necessary at lower levels, but if they're going to be introduced at all, then yes, the formal definition should be given together with enough examples to make it understandable.

Link to comment
Share on other sites

Generally speaking, I greatly prefer understanding the theory or fundamentals behind a specific task or concept if possible.  I realize that something's cannot be taught in depth immediately, but often find that instead of teaching how to rotely perform arithmetic, it's helpful to understand the actual math behind it.

I'm not strong in prob/stat but have found, generally at least, that attempting to explain the underlying fundamentals of a given problem is instructive.  I've never thought that the approach "you don't need to understand how or why this works yet, just do this" to be teaching much, but if it's necessary to teach some other skill or concept in the meantime, I guess you have to.

I guess Tom Wolfe and Ken Kesey would say I'm the not math teacher 

 

Link to comment
Share on other sites

Both? Depends? If you're teaching a class you expect to be using p-values, include an extensive explanation of what they mean and dedicate a percentage of the class work, exercises and probably the final exam/assignment to making sure they get it in-depth. Social-sciences graduate-level methodologies class made up students who need to be designing their thesis project's research methodologies or what have you. If you've got, say, an undergrad methods or stats class in the humanities and you just need to walk people through what those letters and asterisks that turn up on the ocassionaly paper they read mean, probably you don't need so much of the background.

I don't think it's somehow inappropriate to offer a basic definition of p-values and leave it there, since they do get used a lot and there's a category of people out there who would benefit from being able to broadly understand what that  p=0.05 or 0.01 is even supposed to represent without necessarily needing to understand it in-depth. It's a literacy more than a numeracy question on some level.

Link to comment
Share on other sites

If we're talking about a class on statistics, absolutely.  If you're talking about a class that requires a tangential understanding of p-values, then you could probably get away with a less formal definition provided you explain that is what you are doing and point them to a more formal definition should they want to pursue it.  

Link to comment
Share on other sites

p-values are a pretty specific topic. So I assume it's not some highschool math class, but rather a statistics introductory course for psychology, political science or biology or something similar at college level?

Yes. You kinda have to introduce the p-value at some point, and what it means. At least on a conceptual level. Basically what Datepalm said. 

And as always kids, remember the asteriks only tell half the story. Effect sizes are important, too. ;)

 

Link to comment
Share on other sites

I teach introductory statistics to psychology students and in the field at the moment they have to understand how to use p values.  Understanding how to interpret p values is probably the most important thing I have to get across to my students - because the majority of psychology journal articles include them and the students need to know how to analyse data and report them for their own research.  There are only a small minority of psychologists/journals who are taking on board critiques by Bayesians, and as Notone says, the main shift recently has been on making sure students also know about effect sizes and confidence intervals.  It is essential that students have a basic grasp of probability, but students are required to have proficiency at this level of maths in order to get onto a psychology degree in the UK, so I do not have to do more than remind them of it.  However I do not go into details of the p distributions or how calculations of p values are done (which can get excessively complicated, as I found when I was trying to discover how it was done for a chi-square test, which is a very simple test!).  I have to choose what I think is most important for the students to spend their time on (and what they are motivated by), and that is not the maths behind it.

So basically we focus on the p value as a part of null-hypothesis significance testing, as a method of generalising from a sample to population, and that the p value is the calculated value of obtaining the test statistic (or one more extreme) by chance for that particular sample size.  That is the level of explanation that we remain at - as there are so many measurement and interpretation issues that students need to spend their time critically evaluating when doing psychology research, that there would not be time to look at how p values are calculated - one can do well in psychology even by just assuming this is some kind of 'magic'!  I doubt many psychologists (including me) really understand that much of the maths.  By mentioning various methods and controversies I hope I convey to students that use of p values is not a definitive method.  It's quite exciting following the replication crisis which has hit social psychology particularly badly and may partly be due to use of significance testing, and I do a lecture on fraud and p-hacking... but of course that is partly about the psychology of researchers...

Sophie

Link to comment
Share on other sites

1 hour ago, Sophelia said:

I teach introductory statistics to psychology students and in the field at the moment they have to understand how to use p values.  Understanding how to interpret p values is probably the most important thing I have to get across to my students - because the majority of psychology journal articles include them and the students need to know how to analyse data and report them for their own research.  There are only a small minority of psychologists/journals who are taking on board critiques by Bayesians, and as Notone says, the main shift recently has been on making sure students also know about effect sizes and confidence intervals.  It is essential that students have a basic grasp of probability, but students are required to have proficiency at this level of maths in order to get onto a psychology degree in the UK, so I do not have to do more than remind them of it.  However I do not go into details of the p distributions or how calculations of p values are done (which can get excessively complicated, as I found when I was trying to discover how it was done for a chi-square test, which is a very simple test!).  I have to choose what I think is most important for the students to spend their time on (and what they are motivated by), and that is not the maths behind it.

So basically we focus on the p value as a part of null-hypothesis significance testing, as a method of generalising from a sample to population, and that the p value is the calculated value of obtaining the test statistic (or one more extreme) by chance for that particular sample size.  That is the level of explanation that we remain at - as there are so many measurement and interpretation issues that students need to spend their time critically evaluating when doing psychology research, that there would not be time to look at how p values are calculated - one can do well in psychology even by just assuming this is some kind of 'magic'!  I doubt many psychologists (including me) really understand that much of the maths.  By mentioning various methods and controversies I hope I convey to students that use of p values is not a definitive method.  It's quite exciting following the replication crisis which has hit social psychology particularly badly and may partly be due to use of significance testing, and I do a lecture on fraud and p-hacking... but of course that is partly about the psychology of researchers...

Sophie

You forgot to drop the mic

Link to comment
Share on other sites

6 hours ago, Sophelia said:

I do a lecture on fraud and p-hacking... but of course that is partly about the psychology of researchers...

Sophie

I will try giving the formal definition as well as quick discussion of p hacking (It is easy to find John Oliver's quick discussion) However I will mainly make sure they can interpret  computer output. 

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...