Usually when running an anaylsis, it is helpful to use a training module to get better results from the mechanical turk workers. For simplicity, these instructions will skip that step. We will return to this step one the next page. The basic tasks that must be completed to run an analysis are:

  1. Obtain OAuth Token
  2. Upload Documents
  3. Set up HIT settings
  4. Set up Comparisons
  5. Send Comparisons to Turk
  6. Check on Progress
  7. Re-post incomplete HIT’s in a “Batch”
  8. Access completed tasks

All of these steps can also be completed through the R package. If you are not familiar with directly using API calls with software like Curl, we recommend using the R package. We assume below familiarity with the basics of http requests.

1. Obtain OAuth Token

2. Upload Documents

{ "email": <your email>, "auth_token": <your auth_token (taken from step 1)>, "documents": [ { "text": <document_1_text>, "source": <DEPRICATED FIELD> }, { "text": <document_2_text>, "source": <DEPRICATED FIELD> } ] }

{ "ids": [ <id_1>, <id_2> ] }

3. Set up HIT settings

4. Set up Comparisons

{ "email": <your email>, "auth_token": <your auth_token (taken from step 1)>, "question": <Question Text>, "ids": [ [<id1>, <id2>], [<id3>, <id4>] ] }

{ "batch_id": <batch_id> }

5. Send Comparisons to Turk

{ "email": <your email>, "auth_token": <your auth_token (taken from step 1)>, "hit_settings": <hit_id>, "batch_id": <batch_id> }

{ "batch_id": <batch_id> }

6. Check on Progress

{ "id": <batch_id>, "comparisons": { "total_count": <total_comparisons>, "submitted_count": <number_comparisons_submitted>, "completed_count": <number_comparisons_completed>, "expired_count": <number_comparisons_expired> } }

7. Re-post incomplete HIT’s in a “Batch”

{ "email": <your_email>, "auth_token": <your_auth_token< }

{ "id": <batch_id>, "comparisons": { "reposted_count": <number_tasks_reposted> } }

8. Access completed tasks

{ "url": <url_to_download_file> }