8 - Evaluation2.pdf - Information Retrieval Riphah Institute for Computing and Applied Sciences Dr Ayesha Kashif Sec 8.6 Outline \u2022 \u2022 \u2022 \u2022

8 - Evaluation2.pdf - Information Retrieval Riphah...

• Notes
• 10

This preview shows page 1 - 5 out of 10 pages.

4/14/2019 1 Information Retrieval Riphah Institute for Computing and Applied Sciences Dr. Ayesha Kashif Outline Evaluating search engines Benchmark documents and queries Assessment of the Binary relevance Non Binary Relevance Sec. 8.6
4/14/2019 2 Interpolating a Recall/Precision Curve: Exmp 2 0.4 0.8 1.0 0.8 0.6 0.4 0.2 0.2 1.0 0.6 Recall Precision Interpolating a Recall/Precision Curve: Exmp 2 0.4 0.8 1.0 0.8 0.6 0.4 0.2 0.2 1.0 0.6 Recall Precision
4/14/2019 3 Interpolation advantage Will help to plot the point closest to the ideal point Each point corresponds to a result for the top k ranked hits (j = 1, 2, 3, 4, . . .) Rationale for interpolation: The user is willing to look at more stuff if both precision and recall get better (as can be seen from previous graphs) Sec. 8.3 Average Recall/Precision Curve Typically average performance over a large set of queries. Compute average precision at each standard recall level across all queries. Plot average precision/recall curves to evaluate overall system performance on a document/query corpus.
4/14/2019 4 Mean Average Precision (MAP) Average Precision : Average of the precision values at the points at which each relevant document is retrieved. Ex1: (1 + 1 + 0.75 + 0.667 + 0.38 + 0)/6 = 0.633 Ex2: (1 + 0.667 + 0.6 + 0.5 + 0.556 + 0.429)/6 = 0.625 Mean Average Precision : Average of the average precision value for a set of queries. provides a single-figure measure of quality across recall levels The MAP value for a test collection is the arithmetic mean

You've reached the end of your free preview.

Want to read all 10 pages?

• Spring '19
• Dr. Ayesha Kashif

What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern