tag:blogger.com,1999:blog-4297242917419089261.post2562759387286108731..comments2024-03-09T01:05:10.754-08:00Comments on Babies Learning Language: Randomization on mechanical turkMichael Frankhttp://www.blogger.com/profile/00681533046507717821noreply@blogger.comBlogger10125tag:blogger.com,1999:blog-4297242917419089261.post-43523128151299140502017-03-21T02:42:45.556-07:002017-03-21T02:42:45.556-07:00Thank you for the suggestion :-).Thank you for the suggestion :-).Tomhttps://www.blogger.com/profile/05089288084990117892noreply@blogger.comtag:blogger.com,1999:blog-4297242917419089261.post-43901182384328397692017-03-20T11:09:31.422-07:002017-03-20T11:09:31.422-07:00If you're interested in having a stronger back...If you're interested in having a stronger backend, you might consider PsiTurk (https://psiturk.org/) - I deal with the issue you mentioned by blocking turkers from previewing beyond the instructions of my experiment unless they accept. Michael Frankhttps://www.blogger.com/profile/00681533046507717821noreply@blogger.comtag:blogger.com,1999:blog-4297242917419089261.post-63353797680398560432017-03-20T07:49:33.846-07:002017-03-20T07:49:33.846-07:00I tried this approach and it has an important limi...I tried this approach and it has an important limitation. If a worker previews or refreshes the HIT, s/he might receive another treatment (due to the possibility of a worker refreshing the page, disabling the preview would not suffice). In my study, this would introduce an important bias. Therefore, I extended this approach by storing all workers and their initial experimental group assignment in a MySQL database (with an index on WorkerID). Before a worker is shown a page, I check whether s/he is stored in the DB. If yes, I present him/her the same treatment as the initial treatment. If not, I present him/her a random treatment.<br /><br />I also log every time a worker loads the page the time, his/her id, and the assigned group. This way, I can check which user refreshes or drops out.Tomhttps://www.blogger.com/profile/05089288084990117892noreply@blogger.comtag:blogger.com,1999:blog-4297242917419089261.post-51359856206866301922017-03-20T07:49:20.041-07:002017-03-20T07:49:20.041-07:00I tried this approach and it has an important limi...I tried this approach and it has an important limitation. If a worker previews or refreshes the HIT, s/he might receive another treatment (due to the possibility of a worker refreshing the page, disabling the preview would not suffice). In my study, this would introduce an important bias. Therefore, I extended this approach by storing all workers and their initial experimental group assignment in a MySQL database (with an index on WorkerID). Before a worker is shown a page, I check whether s/he is stored in the DB. If yes, I present him/her the same treatment as the initial treatment. If not, I present him/her a random treatment.<br /><br />I also log every time a worker loads the page the time, his/her id, and the assigned group. This way, I can check which user refreshes or drops out.Tomhttps://www.blogger.com/profile/05089288084990117892noreply@blogger.comtag:blogger.com,1999:blog-4297242917419089261.post-79932260639517757052017-02-09T19:13:12.364-08:002017-02-09T19:13:12.364-08:00Sorry if I'm repeating myself. I just wanted t...Sorry if I'm repeating myself. I just wanted to clarify one more question and make sure that my understanding is right. <br />(1) By multiple different assignments, do you actually mean 'multiple hits'? If this is the case, do you realize 'random assignment' through javascript? <br />(2) If you did mean 'multiple assignments', I am a little confused and hope that you can give me some guide. To my understanding, multiple assignments within one hit correspond to the same task. Even though Turk itself can randomly show these assignments, I cannot use different assignments to display different groups of tasks.<br />Sorry that I have made the question so long. If there is anything wrong with my understanding, please feel free to let me know. And really thanks for the suggestion of 'unique turker link'. huaguo wuhttps://www.blogger.com/profile/13478619222010133228noreply@blogger.comtag:blogger.com,1999:blog-4297242917419089261.post-88731008297211209402017-02-09T18:16:24.622-08:002017-02-09T18:16:24.622-08:00Really appreciate your reply, Michael.Really appreciate your reply, Michael.huaguo wuhttps://www.blogger.com/profile/13478619222010133228noreply@blogger.comtag:blogger.com,1999:blog-4297242917419089261.post-30164478613019061782017-02-02T21:50:47.709-08:002017-02-02T21:50:47.709-08:00Well, the basic HTML interface allows you to uploa...Well, the basic HTML interface allows you to upload multiple different assignments, so you could just randomly assign via that method (plus use Unique Turker https://uniqueturker.myleott.com/) - but it's less versatile. Michael Frankhttps://www.blogger.com/profile/00681533046507717821noreply@blogger.comtag:blogger.com,1999:blog-4297242917419089261.post-25045841095000782372017-02-02T18:07:13.889-08:002017-02-02T18:07:13.889-08:00I see. Thank you very much!!! So there is no way t...I see. Thank you very much!!! So there is no way to realize even randomization if I only use the template provided by Turk?huaguo wuhttps://www.blogger.com/profile/13478619222010133228noreply@blogger.comtag:blogger.com,1999:blog-4297242917419089261.post-88159772256292333322017-02-02T10:14:49.443-08:002017-02-02T10:14:49.443-08:00Yes, probably - you need to have something to run ...Yes, probably - you need to have something to run the PHP and to host the JavaScript that queries it. Michael Frankhttps://www.blogger.com/profile/00681533046507717821noreply@blogger.comtag:blogger.com,1999:blog-4297242917419089261.post-79371240743184085272017-02-02T10:04:55.947-08:002017-02-02T10:04:55.947-08:00Hi there, to conduct this randomization, do I need...Hi there, to conduct this randomization, do I need to run my experiment on an external server?huaguo wuhttps://www.blogger.com/profile/13478619222010133228noreply@blogger.com