Quiz 1 - Precision & Recall

Quiz 1 - Precision & Recall

par Leon Christian Muller,
Nombre de réponses : 4

Hello,

Regarding today's quizz, I've got a question about the computation of recall & precision metrics (on the reviews subject).

When computing such metrics, we want to have a dominant class to be determined as TRUE and one as FALSE. For example, if we had classification of dogs and cats, the precision metric for dogs isn't the same as the one for cats.

In today's quizz, no argument was made in the question description about the one to use as the dominant one. So I assumed, as you were talking about the negative only, we should directly compute the metrics over the negative class, and not reverse the whole thing (which is not that difficult btw, so it's not the problem here, but only the comprehension of the question itself).

I guess it's just my mistake now, but is it possible for the next quizzes/exam to clearly state which class we should compute the metrics on?

Also, if both questions have been well understood by everyone, it's just mb then.

Thank you!


En réponse à Leon Christian Muller

Re: Quiz 1 - Precision & Recall

par Jean-Cédric Chappelier,
Very good point indeed, and... that's also something we want to test/provide you feedback on:
1. what is the main reason of using Precision and Recal? (I what situation, I mean?)
2. then, if nothing else is specified, for which class does it make (more) sense to compute these? (as, as you rightly stated, P and R won't be the same for the two classes).

Answers:
1. for ill-balanced classes
2. for the small class (otherwise the score will be high anyway. What about the score of a system classifying always in the bigest class? BTW, this is exactly a (nasty) way to cheat your results when you need to (politician?): provide P and R for the big class only ;-) )

Makes sense?
En réponse à Jean-Cédric Chappelier

Re: Quiz 1 - Precision & Recall

par Leon Christian Muller,
Yes, it does make more sense now that you state it!
I guess this one will be known forever now :(
Thank you for the explanation!
En réponse à Leon Christian Muller

Re: Quiz 1 - Precision & Recall

par Bich Ngoc Doan,
Hello, I do have a follow-up question on this, but on another problem in the quiz:

For the problem where there's booking and cancelling evaluation, the corpus contains 52% of booking requests. 52% seems pretty... well-balanced to me? Then for this problem, why do we need precision and recall? Shouldn't accuracy be enough?