hw3.pdf - Assignment 3 Logistics You must create one...

This preview shows page 1 - 2 out of 4 pages.

Assignment 3 Logistics You must create one Jupyter notebook for the entire assignment, and upload it to Canvas by August 7th, before class begins . Introduction Suppose we have a bunch of URLs and we want to know their adult-rating (i.e., is the url P, or G, or X, or R). This task is difficult for computers, but easy for humans, and this has led to the growth of crowdsourcing: get a bunch of humans to give ratings to urls, but use automated techniques to figure out how much to trust each person’s ratings. We are going to use the data from a paper by Ipeirotis et al. available here. This details an experiment run on Amazon’s Mechanical Turk crowd- sourcing system. They ask a bunch of raters (called “turks”) to rate several urls, but they already know the answers (the true categories) for a few urls, called the “gold set”. The ratings of the turks on the gold set thus allows us to judge their accuracy. [Q1 10 points] Read in data Read in the files gold.txt and labels.txt . The gold DataFrame should have columns url and category , while the labels DataFrame should have columns turk , url and category . You will have to pick the right separator.
Image of page 1

Subscribe to view the full document.

Image of page 2
  • Summer '19
  • Accuracy and precision, DataFrame, The Turk

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern