question 6.1 in hands-on Information Retrieval

question 6.1 in hands-on Information Retrieval

by Federico Di Gennaro -
Number of replies: 4

Good morning,

I have not well understood why in question 6.1 of the hands-on about information retrieval the system preferred is the second one if S1 and S2 have the same values of Recall (metric I think is the most suitable for the task). 

Thanks,

Federico.

In reply to Federico Di Gennaro

Re: question 6.1 in hands-on Information Retrieval

by Jean-Cédric Chappelier -
What does it mean "to have the same recall" (in this context)? Which recall?

You are right that "recall is the most suitable for this task". In this context, this means you'll be working at the highest possible recall. At that recall, you'll prefer the system that provides the most appropriate answers; i.e. at that (high) recall, you'll choose the system with the highest precision.
Makes sense?
In reply to Federico Di Gennaro

question 6.1 in hands-on Information Retrieval

by Federico Di Gennaro -
But since the two system are evaluated and they got the same values of Recall for all the three evaluation, what would suggest me to prefer system S2 over system S1? I would have choose “cannot be decided” option.
In reply to Federico Di Gennaro

Re: question 6.1 in hands-on Information Retrieval

by Jean-Cédric Chappelier -
I told you: the one with the best precision at high recall.

What does it mean to have "the same values of recall"? How is the evaluation conducted? How do you change the recall (to plot the Precision versus Recall curves)?
--> by "iteratively stoping" the system at a given recall (if possible): asking it to return x% of the relevant documents.
In reply to Jean-Cédric Chappelier

Re: question 6.1 in hands-on Information Retrieval

by Federico Di Gennaro -
Thanks a lot for your answer, I got it!

Have a nice day,
Federico.