CFP: Crowdsourcing and Human Computation Track

Track chairs:


We invite research contributions to the Crowdsourcing and Human Computation track at the 32nd edition of The Web Conference series (formerly known as WWW), to be hosted at Austin, TX, US, on April 30 - May 4, 2023 ( /www2023/)

Fifteen years ago, a 2007 WWW paper entitled “Internet-Scale Collection of Human-Reviewed Data” was one of several forerunners to signal a new, emerging area of research on Human Computation and Crowdsourcing (HCOMP). Growing excitement and work in this new area would eventually lead to four years of HCOMP workshops across KDD and AAAI (2009-2012), a new annual AAAI HCOMP conference (2013 onward), and a new, annual HCOMP track at the WebConference (2014 onward).

Today, the world and research landscape looks remarkably different than it did in 2007, with the Web playing a central role in orchestrating such advances. Of particular note, modern neural models have transformed AI capabilities, along with far greater ubiquity and significance of AI systems now in practical deployment around the world. As one effect of this, the commoditization and democratization of AI models today has also brought a new focus to “data-centric AI” in which AI models can succeed or fail based on the quality of underlying data and human annotations. The nature of human-AI interactions are also continually evolving in response to AI advances, posing an ever-changing frontier of new challenges for researchers and practitioners. Furthermore, the growth of AI power has brought a commensurate recognition of the need for responsible AI systems that are fair, accountable, transparent, and trustworthy – across diverse, global communities of human stakeholders who interact with or are impacted by AI systems. Given the central role of HCOMP in AI (creating reliable training and benchmark annotations, as well as enabling hybrid, human-in-the-loop systems), continuing innovation in HCOMP remains a key challenge for the further advancement of AI. HCOMP itself has made tremendous strides forward in the past fifteen years, yet many research challenges remain.

Our track invites AI, HCI, and related contributions that advances the broad spectrum of crowdsourcing and human computation (HCOMP) in the scope of the Web:

More specifically, we invite work addressing contemporary HCOMP challenges including (but not limited to) the following Web-related themes:

Authors should consult the conference’s main Research Track CFP to ensure their submissions are aligned with broader conference expectations, scope, and theme: “Web Research with Openness, Fairness and Reproducibility”. The CFP also details submission guidelines, relevant dates, and important policies. Review criteria will include considerations typical of those in past years of this track and the AAAI HCOMP conference, e.g.,

Submissions that are out of scope or unresponsive to the call above will be rejected early during the reviewing process (“desk rejected”) with minimal feedback.This includes submissions that:

In case you have doubts whether your paper fits the scope of this track, please contact the track chairs