Challenges in Machine Learning:
Gaming and Education
Friday December 9, 2016, Barcelona, Spain
NIPS 2016 workshops. Centre Convencions Internacional Barcelona
We want to give a large space to discussion, by organizing two discussion sessions monitored by the organizers, who will first give a brief introduction to several selected topics: how are challenges used "in class", what makes a good classroom challenge, challenges and MOOCs, grading challenge work, involving students in challenge work. In addition, the invited speakers will be asked to reflect in their presentations upon the four main topics of discussion.
Call for abstract:
The first call-for-abstract is over, but you can bring an impromptu poster and send a ~500 word abstract as late as the day before the workshop to share late breaking news!
Topics of interest
- Novel or atypical challenge protocols, particularly relating to gaming and education.
- Novel or atypical challenge protocols to tackle complex tasks with very large datasets, multi-modal data, and data streams.
- Methods and metrics of entry evaluation, quantitative and qualitative challenges.
- Methods of data collection, "ground-truthing", and preparation including bifurcation/anonymization, data generating models.
- Teaching challenge organization.
- Hackatons and on-site challenges.
- Challenge indexing and retrieval, challenge recommenders.
- Societal of psychological studies of theories about gaming and education.
- Experimental design, size data set, data split, error bounds, statistical significance, violation of typical assumptions (e.g. i.i.d. data).
- Game theory applied to the analysis of challenge participation, competition and collaboration among participants.
- Diagnosis of data sanity, artifacts in data, data leakage.
- Re-usable challenge platforms, innovative software environments.
- Linking data and software repositories to challenges.
- Security/privacy, intellectual property, licenses.
- Cheating prevention and remedies.
- Issues raised by requiring code submission.
- Challenges requiring user interaction with the platform (active learning, reinforcement learning).
- Dissemination, fact sheets, proceedings, crowdsourced papers, indexing post-challenge publications.
- Long term impact, on-going benchmarks, metrics of impact.
- Participant rewards, stimulation of participation, advertising, sponsors.
- Profiling participants, improving participant professional and social benefits.
- Challenges as an educational tool.
- Where to venture next: opportunities for challenge organizers to organize challenges in new domains with high societal impact.
- Successful challenge leading to significant breakthrough or improvement over the state-of-the-art or unexpected interesting results.
- Rigorous study of the impact of challenges, analyzing topics and tasks lending themselves to high impact machine learning challenges.
- Challenges organized or supported by Government agencies, funding opportunities.