Latest Research

• Machine learning for Neuroscience: Growing datasets in neuroscience require tools for jointly analyzing high-dimensional behavioral and neural recordings. We built a contrastive learning framework for estimating Consistent EmBeddings of high-dimensional Recordings using Auxiliary variables (CEBRA). Stay tuned for the code release.
• Robustness & Adaptation in Computer Vision: We demonstrated that robustness estimates of ImageNet-scale architectures can be drastically improved by adapting models with simple domain adaptation methods. Batch norm adaptation yields consistent 5-15% points gains across various model architectures. Self-learning can further improve scores, even for models pre-trained on large amounts of data. We obtain scores as low as 22% mCE on ImageNet-C, 17.4% top-1 error on ImageNet-R and 14.8% top-1 error on ImageNet-A. Check out robusta, our PyTorch library for robustness & adaptation on Github. Beyond ImageNet, we demonstrate improved ood. performance of pre-trained pose estimation networks and discuss principles, pitfalls and perspectives of pose estimation algorithms in our recent Neuron primer.
• Self-supervised Learning: wav2vec and vq-wav2vec demonstrated the effectiveness of contrastive learning for reducing the need for labeled data in speech recognition models, and a lot of progress in contrastive learning has been obtained in other fields of machine learning. We worked on understanding the effectiveness of contrastive pre-training and found that contrastive learning can invert the data-generating process.

Events

• October 2022: We updated the CEBRA pre-print with additional experiments and identifiability results. Make sure to watch the repository to stay tuned about updates on the code release.
• May 2022: I joined the FAIR team at Meta in New York as a research scientist intern! I will be working with Laurens van der Maaten and Ishan Misra on multimodal representation learning.
• April 2022: We released the pre-print of CEBRA, our contrastive representation learning method for joint behavioral and neural data. Read more at cebra.ai!

Meetings, Workshops & Talks

• Mar 2023: Poster and Workshop on CEBRA at the “Biology of Behavior Change” Symposium, Kyoto (with MW Mathis and A Mathis).
• Mar 2023: Poster “Explainable and consistent embeddings of high-dimensional recordings using auxiliary variables” at COSYNE 2023, Montreal.
• Feb 2023: Talk about CEBRA at the NeuroMEETS seminar at the Max Planck Florida Institute for Neuroscience.
• Nov 2023: Guest lecture “Modern machine learning methods for neuroscience”, Advanced data and information literacy track (ADILT), University of Konstanz
• Oct 2022: Talk on CEBRA at the MSNE project week 2022 at TU Munich.
• Oct 2022: Poster on CEBRA at the Neuro X Institute Inaugural event 2022 and the AI in Life Sciences Retreat at EPFL.
• Sep 2022: Flash Talk on CEBRA at the neuromatch conference 2022.
• Sep 2022: Poster on CEBRA at the Bernstein Conference 2022 in Berlin.
• Jun 2022: We organized a workshop on distribution shift in computer vision at ICML 2022, Baltimore. The workshop contributions are available on OpenReview and GitHub.
• Oct 2021: Talk on batch norm adaptation and workshop on implementing machine learning experiments at the MSNE Project Week 2021, TU Munich.
• Jun 2021: We presented “Contrastive Learning Inverts the Data Generating Process” at ICML 2021.
• May 2021: We talked about self-learning for adaptation at the ICLR workshop on weakly supervised learning (contributed talk).
• Dec 2020: We presented our work on batch norm adaptation at NeurIPS 2020.
• Jun 2020: ICML Workshop on Uncertainty & Robustness in Deep Learning: Oral presentation (5% acceptance), of our paper Improving robustness against common corruptions by covariate shift adaptation.
• Apr 2020: ICLR 2020, poster presentation of our paper vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations
• Feb 2020: 1st Tübingen Robust Learning Symposium (Co-Organizer)
• Jan 2020: Co-Organizer, Quantum Machine Learning Course, CdE Winter School

Software

• Code for the CEBRA toolbox introduced and used in our paper “Learnable latent embeddings for joint behavioral and neural analysis” will be released upon publication of the paper. Watch the Github repository for notifications on the release date.
• Our “Shift Happens” workshop at ICML 2022 resulted in a software toolbox with 10+ computer vision benchmarks evaluating the effects of distribution shifts.
• Code for our paper Improving robustness against common corruptions by covariate shift adaptation and Adapting ImageNet-scale models to complex distribution shifts with self-learning available as part of our robusta software package on Github.
• (vq-) wav2vec code and models released as part of fairseq.
• PyTorch toolbox for Semi-supervised Adaptive Learning Across Domains (salad) for a fair and reproducible evaluation of current domain adaptation algorithms.

Press

• wav2vec is presented at Facebook F8 2019. May 1, 2019.
• Facebook details wav2vec, an AI algorithm that uses raw audio to improve speech recognition. Venturebeat, Nov 5, 2019.