CiML 2018‎ > ‎

Schedule

2018 Challenges in Machine Learning workshop@NIPS
Machine Learning Challenges "in the wild"


Morning session (8:00 am-12:00 pm): New platforms and innovative designs

8:00 am (10 min) - Morning welcome and introduction 

8:10  (30 min) - Esteban Arcaute and Umut Ozertem - Facebook project on developing benchmarks of algorithms in realistic settings

8:40 (30 min) - Laura Seaman - Project Alloy – Machine Learning Challenges for Researching Human-Machine Teaming (slides)

9:10  (90 min)  - Live competition: Pommerman (slides)

10:40 - 11:00  (20 min)  Break, poster viewing

11:00  (30 min)  - Julien Hay, Bich-Liên Doan, Fabrice Popineau  - Renewal news recommendation platform (slides)

11:30 - 12:00  (30 min)  Panel discussion. Design principles and implementation issues. (3 panelists TBA + 1 moderator)

Lunch break (12:00 pm-1:00pm)

Lunch served in the room for participants.

Discussions and poster viewing.

Afternoon session (1:00 pm-6:30 pm): Robotics, RL, and interactive challenges

1:00 - Afternoon welcome and announcements (organizers)

1:10  (30 min) - Mikhail Burtsev and Varvara Logacheva - Wild evaluation of chat-bots (slides)

1:40  (30 min) - Larry Jackel - Measuring Progress in Robotics

2:10  (30 min) - Daniel Polani - Competitions to Challenge Artificial Intelligence: from the L-Game to RoboCup 

2:40  (30 min) - Antoine Marot - Learning to run a power network 

3:10 - 3:30  (20 min)  Break, poster viewing

3:30  (90 min) - Live competition: The driving AI Olympics (slides)

5:00 - 5:30  (30 min) - Panel discussion: Opportunities to organize new impactful challenges. (3 panelists TBA + 1 moderator)

5:30 - Adjourn.

6:30 - Closing banquet.


Posters

Posters dimensions are: 36 x 48 in. (91cm x 122cm); masking tape will be provided. All posters can be presented at all poster sessions.


New platforms and innovative designs:

Beyond the Leaderboard, Adriënne M. Mendrik, Stephen R. Aylward 

Grand-Challenge.org, James A. Meakin Bram van Ginneken

Corpus for AutoML Pipelines, Richard Lippmann, Swaroop Vattam, Pooya Khorrami, and Cagri Dagli

How to fail hosting data science contests with images, Evgeny Nizhibitsky, Moscow State Univ. Artur Kuzin, Dbrain, Moscow Institute of Physics and Technologies, Russia

ML Benchmark Tools Package, Ryan Turner, Uber AI Labs


New impactful challenges:

NASA Frontier Development Lab 2018A. Bell, A. Chopra, W. Fawcett, R. Talebi, D. AngerhausenA. Berea, N.A. Cabrol, C. Kempes, M. Mascaro

AutoDL challenge design and beta tests, Zhengying Liu, Olivier Bousquet, Andre Elisseeff, Isabelle Guyon, Adrien Pavao, Lisheng Sun-Hosoya, and Sebastien Treguer

TrackML, a Particle Physics Tracking Machine Learning Challenge, Jean-Roch Vlimant (Caltech), Vincenzo Innocente, Andreas Salzburger (CERN), Isabelle Guyon (ChaLearn), Sabrina Amrouche, Tobias Golling, Moritz Kiehn (Geneva University),David Rousseau∗, Yetkin Yilmaz (LAL-Orsay), Paolo Calafiura, Steven Farrell, Heather Gray (LBNL), Vladimir Vava Gligorov (LPNHE-Paris), C ́ecile Germain, Victor Estrade(LRI-Orsay),Edward Moyse (University of Massachussets), Mikhail Hushchyn, Andrey Ustyuzhanin (Yandex, HSE)

L2RPN: Learning To Run a Power Network Competition, Antoine MarotBalthazar Donon, Isabelle GuyonBenjamin Donnot.

Multi-Agent RL in MalmÖ(MARLÖ) Competition, Diego Perez-Liebana Katja Hofmann Sharada Prasanna Mohanty Noburu Kuno Andre Kramer Sam Devlin Raluca D. Gaina Daniel Ionita 


Live competitions:

Pommerman

Competition summary: Train a team of communicative agents to play Bomberman. Compete against other teams.

Organizers:
Cinjon Resnick, NYU, cinjon@nyu.edu
David Ha, Google Brain, hadavid@google.com
Denny Britz, Prediction Machines, dennybritz@gmail.com
Jakob Foerster, Oxford, jakobfoerster@gmail.com
Jason Weston, Facebook FAIR, jase@fb.com
Joan Bruna, NYU, bruna@cims.nyu.edu
Julian Togelius, NYU, julian@togelius.com
Kyunghyun Cho, NYU, kyunghyun.cho@nyu.edu
Webpage: https://www.pommerman.com/

The AI diving Olympics

Competition summary: Machine Learning (ML), deep learning, and deep reinforcement learning have shown remarkable success on a variety of tasks in the very recent past. However, the ability of these methods to supersede classical approaches on physically embodied agents is still unclear. In particular, it remains to be seen whether learning-based approached can be completely trusted to control safety-critical systems such as self-driving cars. This live competition, presented by the Duckietown Foundation, is designed to explore which approaches work best for what tasks and subtasks in a complex robotic system. The participants will need to design algorithms that implement either part or all of the management and navigation required for a fleet of self-driving miniature taxis. There will be a set of different trials that correspond to progressively more sophisticated behaviors for the cars. These vary in complexity, from the reactive task of lane following to more complex and “cognitive” behaviors, such as obstacle avoidance, point-to-point navigation, and finally coordinating a vehicle fleet while adhering to the entire set of the “rules of the road”. We will provide baseline solutions for the tasks based on conventional autonomy architectures; the participants will be free to replace any or all of the components with custom learning-based solutions.The competition will be live at NIPS, but participants will not need to be physically present—they will just need to send their source code packaged as a Docker image. There will be qualifying rounds in simulation and we will make available the use of “robotariums,” which are facilities that allow remote experimentation in a reproducible setting.

Organizers:

Andrea Censi, nuTonomy and ETH Zürich, acensi@idsc.mavt.ethz.ch
Liam Paull, Université de Montréal, paulll@iro.umontreal.ca
Jacopo Tani, ETH Zürich, tanij@ethz.ch
Scott Livingston, q@rerobots.net
Julian Zilly, ETH Zürich, jzilly@ethz.ch
Ruslan Hristov, nuTonomy, rusi@nutonomy.com
Oscar Beijbom, nuTonomy, oscar@nutonomy.com
Eryk Nice, nuTonomy, eryk.nice@nutonomy.com
Sunil Mallya, Amazon, smallya@amazon.com
Justin De Castri, Amazon, decastri@amazon.com
Hsueh-Cheng (Nick) Wang, National Chiao Tung University,  hchengwang@gmail.com
Qing-Shan Jia, Tsinghua, jiaqs@tsinghua.edu.cn
Tao Zhang, Tsinghua , taozhang@tsinghua.edu.cn
Stefano Soatto, UCLA and Amazon, soattos@amazon.com
Magnus Egerstedt, Georgia Tech, magnus.egerstedt@ece.gatech.edu
Yoshua Bengio, Université de Montréal, yoshua.bengio@umontreal.ca
Emilio Frazzoli, ETH Zürich and nuTonomy, emilio.frazzoli@idsc.mavt.ethz.ch

Webpage: https://AI-DO.duckietown.org 



Dinner information:

The speakers and organizers are invited for dinner at:

The dinner will be at 8 pm the night before the workshop, Friday December 7:

Restaurant ChinaTown Kim Fung, 1111 Rue Saint-Urbain, Montréal, QC H2Z 1Y6, Canada. (514) 878-2888 


Ċ
Isabelle Guyon,
Oct 18, 2018, 1:23 PM
Ċ
Isabelle Guyon,
Oct 18, 2018, 1:23 PM
Ċ
Isabelle Guyon,
Oct 18, 2018, 1:23 PM
Ċ
Isabelle Guyon,
Oct 18, 2018, 1:23 PM
Ċ
Isabelle Guyon,
Oct 18, 2018, 1:24 PM
Ċ
Isabelle Guyon,
Oct 19, 2018, 10:09 AM
Ċ
Isabelle Guyon,
Oct 18, 2018, 1:24 PM
Ċ
Isabelle Guyon,
Oct 18, 2018, 1:23 PM
Ċ
Isabelle Guyon,
Oct 18, 2018, 1:23 PM
Ċ
Isabelle Guyon,
Oct 19, 2018, 11:14 AM
Ċ
Hugo Jair Escalante,
Dec 14, 2018, 7:35 AM
Ċ
Hugo Jair Escalante,
Dec 14, 2018, 7:38 AM
Ċ
Hugo Jair Escalante,
Dec 14, 2018, 7:38 AM
Ċ
Hugo Jair Escalante,
Dec 14, 2018, 7:32 AM
Ċ
Hugo Jair Escalante,
Dec 14, 2018, 7:32 AM
Comments