The inaugural AutoML conference 2022 was an exciting adventure for us! With 170 attendees in the very first iteration, we assess this conference as a big success and are confirmed in our belief that it was the right time to transition from a workshop series to a full-fledged conference. In this blogpost, we will summarize our take-aways from 2022 and some changes for 2023. To this end, we asked attendees from 2022 to fill out an exit survey that is the basis for many of our decisions.
Disclaimer: we invited all participants to fill out the survey; 35 attendees participated, of whom 26 attended onsite and 11 also attended ICML. Thus, the sample size is rather small, and there also was a sampling bias. Therefore, we took the results with a grain of salt.
Overall impression of AutoML’22
The in-person attendees liked the conference very much. 96.2% of them rated the conference as good (46.2%) or very good (50%), and nobody disliked it. For the virtual attendees, the overall impression was still good, with 88,9% of them giving a positive rating. Also, both groups rated the overall time distribution between talks as fine. When asked whether they would attend AutoML’23, of all the survey participants, 8.6% said “maybe”, 31.4% said “yes”, and 60% said “yes, definitely”. We are very happy about this positive feedback. And of course we will try to make AutoML’23 even better 🙂
AutoML vs ICML
We started with the goal of becoming a premier conference in the field, with the highest quality standards. Since AutoML’22 followed directly after ICML’22, we asked participants to compare the two conferences. In terms of paper quality, participants ranked AutoML’22 comparable with ICML (in fact, very slightly better on average), but the big difference lies in reproducibility: 77,7% rate the reproducibility of AutoML papers higher than at ICML. Similarly, participants clearly ranked attending AutoML better than attending ICML (62.5% better, 31.2% similar, 6.3% worse).
We did not ask for reasons that AutoML was better than ICML in the survey, but the feedback we got from a few individuals indicates that much of this is due to the long-needed specialized conference of AutoML researchers: all the papers feel directly relevant to AutoML and the participants have strong backgrounds.
Co-Location with ICML
In 2022, AutoML was an independent conference but nevertheless co-located with ICML. Our hope was that ICML’22 participants with an interest in AutoML would also attend AutoML’22 afterwards. Surprisingly, only 25,7% attended both conferences in person. Several reasons for this we were told include: (i) being away from family for too long is an issue, (ii) being away from the office is not always feasible; (iii) people either submitted to AutoML or ICML. Furthermore, 70,4% said that AutoML should not be co-located again and should have its own location. We followed that clear advice, having the nice side effect that we can also pull the submission deadlines further apart.
About the conference location
Dozens of people reached out to us saying that they were not able to get a visa for the USA on time (some received appointments at the visa office as late as 2023!). In fact, even one of Marius’ Ph.D. students could not attend AutoML because of visa issues. Further feedback was that people did not feel safe in Baltimore. Because of these reasons, we looked for an alternative location in Europe for 2023. We found a very nice venue at the Hasso Plattner Institute (Potsdam/Berlin) where we hope that neither visa nor safety will be an issue.
Program
Overall, the talks and panel sessions at AutoML’22 were very well received and the attendees on average roughly ranked the choice of our elements as “just right”. Furthermore, 73,1% said that the focus on time for interaction “was absolutely the right choice”, and we see it as reaffirming that NeurIPS followed suit.
The experience of attending the live-streamed talks on-site was rather mixed. We also share this concern, and while we faced a “cold-start” problem with getting senior speakers to attend the inaugural AutoML conference in person, we will strive to have as many talks on-site as possible for AutoML’23.
Hybrid is the new normal
In 2022, we had a post-conference virtual poster session event. Online but also on-site attendees had another chance to present their posters and to get into discussions with other attendees. With 45 concurrent attendees at the peak, this event was also a success – especially if compared to virtual poster sessions at other major conferences. 68,8% also liked this virtual poster event.
Nevertheless, we also got some negative feedback from virtual-only attendees about not being able to directly interact at the main conference but having to wait for the virtual event. Therefore, while the virtual option was really just an afterthought for 2022 owed to COVID-19, for AutoML 2023 we are considering new technical solutions for offering a real hybrid conference experience, especially including hybrid poster sessions.
Going forward
In 2023, we’re excited to have another amazing set of program chairs: Aleksandra Faust (Google), Roman Garnett (University of Washington) and Colin White (Abacus.AI), whose research covers very complementary parts of AutoML. Aleksandra also already ran a hybrid poster session before as program co-chair of CoRL 2021. Frank will stay general chair for AutoML’23 and then step down. As in 2022, in 2023 neither general nor program chairs can submit papers to the conference’s main track in order to set up the conference to avoid any conflicts of interests. (We will, however, keep the journal track, and with the external, prior stamp of approval of papers in that track, submission to this track is fine for general and program chairs.)