Machine Learning Work Shop - Combining Machine and Human Intelligence in Crowdsourcing
23Machine Learning Work Shop-Session 4 - Ece Kamar - 'Combining Machine and Human Intelligence in Crowdsourcing' Crowdsourcing has been increasingly popular for gaining programmatic access to human intelligence for solving tasks that computers cannot easily perform alone. To date, computers have been employed largely in the role of platforms for recruiting and reimbursing human workers; the burden of managing crowdsourcing tasks and ensuring quality has relied on manual designs and controls. In this talk, I will show how machine learning and decision-theoretic reasoning can be used in harmony to leverage the complementary strengths of humans and computational agents to solve crowdsourcing tasks efficiently. This methodology, which we refer to as CrowdSynth, includes predictive models that perform inference about workers and tasks, and efficient algorithms for making effective decisions. We demonstrate the way CrowdSynth methodology can help to maximize the efficiency of a large-scale crowdsourcing operation with experiments on a large-scale citizen-science project called Galaxy Zoo.