CFP
forChallenges track

This year, for the first time, The Web Conference 2018 will include the Challenges Track.

The purpose of challenges is to showcase the maturity of the state of the art on tasks common to the Web community and adjacent academic communities, in a controlled setting of rigorous evaluation.

We are therefore calling for organisers to propose challenges to be run before and during the conference. Each challenge will be centred around a specific task or problem, attracting participants to propose solutions and compete on the basis of a clear and formal evaluation (e.g. based on a shared benchmark). A challenge submission should contain information about the task/problem addressed, its significance, what will be required from the participants to provide, and the criteria under which each participant’s solution will be evaluated.

The Web Conference Challenges is an official track of the conference. Challenge organisers will therefore request from participants to provide, in addition to their participation to the challenge, a paper describing the proposed solution and, when relevant, self-assessments related to the defined criteria for evaluation. These papers will have to be peer-reviewed by experts relevant to the challenge task, and will be published in the official satellite proceedings of the conference.

More information about the format of the challenge participant papers will be available at the time of organizing the challenges.

Challenges Track Chairs:

• Elena Cabrio (Université Côte d’Azur, Inria, CNRS, I3S, France)
• Mathieu d’Aquin (Insight Centre for Data Analytics National University of Ireland Galway, Ireland)

Contact: challengechairs (at) www2018.thewebconf.org

Guideline for submission:

Your submission should describe a challenge to be organised as part of The Web Conference 2018. It should be made as a pdf document through the easychair system (at https://easychair.org/conferences/?conf=www2018satellites, selecting the “Challenges” Track).

Each submission should include information about various aspects of the proposed challenge following the template below. The selection of challenges to be included in the conference will be based on the significance of the task, the potential impact of the challenge and the structure of the challenge, including the robustness of the evaluation method and the ability to attract participants. New editions of challenges that have already been organised in the past are encouraged.

Template for submissions to the challenges track:

1. Title
2. Overview (max 500 words): Description of the task and general overview of the way the challenge will be run. Used as abstract in the easychair submission.
3. Motivation and existing work (max 500 words): Description of the reasons why the challenge should happen at The Web Conference, including assessment of the significance of the task and of the potential impact of the challenge. This should also include a brief overview of existing work in the area of the challenge (e.g. what is the current baseline solution to the problem addressed, which research groups are doing research in this area and can be expected to participate in the challenge).
4. Challenge timeline: Simple description of the structure of the challenge, including what is being submitted by participants when (e.g. solution software, evaluation results), and what is being provided by the organisers when (e.g. benchmark data, evaluation framework).
5. Evaluation criteria (max 500 words): A clear description of the way the evaluation will be conducted, i.e. what will be measured, in what way, and for what purpose. Qualitative evaluations are acceptable, but challenge organisers will have to explain how such an evaluation can be used as a basis to assess the solutions proposed by challenge participants against each other.
6. Other organisational matters: Please indicate here if you will need specific resources for the organisation of the challenge, and if you already have plans regarding prizes or sponsors for your challenge.
7. Organisers (max 200 words per organisers): Short bio of each of the organisers in relation to the challenge.
8. Relation with other initiatives or events (if relevant, max 300 words): An indication of whether the challenge is part of a series of challenges, is related to another event (e.g. an existing workshop) or to a project.

Important dates:

• Challenge proposal submission deadline : 20 October 2017
• Challenges chosen/merged – notification sent : 27 October 2017
• Training data ready and challenges CFP sent : 01 December 2017
• Challenge papers submission deadline : 12 January 2018
• Challenge papers acceptance notification : 14 February 2018
• Challenge test data published : 14 February 2018
• Challenge papers final version due : 04 March 2018
• Challenge – proclamation of winners : During the conference

All submission deadlines are at 9:00pm HAST.