Latest Research

• Machine learning for Neuroscience: Growing datasets in neuroscience require tools for jointly analyzing high-dimensional behavioral and neural recordings. We built a contrastive learning framework for estimating Consistent EmBeddings of high-dimensional Recordings using Auxiliary variables (CEBRA). Stay tuned for the code release.
• Robustness & Adaptation in Computer Vision: We demonstrated that robustness estimates of ImageNet-scale architectures can be drastically improved by adapting models with simple domain adaptation methods. Batch norm adaptation yields consistent 5-15% points gains across various model architectures. Self-learning can further improve scores, even for models pre-trained on large amounts of data. We obtain scores as low as 22% mCE on ImageNet-C, 17.4% top-1 error on ImageNet-R and 14.8% top-1 error on ImageNet-A. Check out robusta, our PyTorch library for robustness & adaptation on Github. Beyond ImageNet, we demonstrate improved ood. performance of pre-trained pose estimation networks and discuss principles, pitfalls and perspectives of pose estimation algorithms in our recent Neuron primer.
• Self-supervised Learning: wav2vec and vq-wav2vec demonstrated the effectiveness of contrastive learning for reducing the need for labeled data in speech recognition models, and a lot of progress in contrastive learning has been obtained in other fields of machine learning. We worked on understanding the effectiveness of contrastive pre-training and found that contrastive learning can invert the data-generating process.

Events

• October 2022: We updated the CEBRA pre-print with additional experiments and identifiability results. Make sure to watch the repository to stay tuned about updates on the code release.
• May 2022: I joined the FAIR team at Meta in New York as a research scientist intern! I will be working with Laurens van der Maaten and Ishan Misra on multimodal representation learning.
• April 2022: We released the pre-print of CEBRA, our contrastive representation learning method for joint behavioral and neural data. Read more at cebra.ai!

Meetings, Workshops & Talks

• October 2022: I will present CEBRA at the MSNE project week 2022 at TU Munich.
• October 2022: I will present CEBRA at the Neuro X Institute Inaugural event 2022 and the AI in Life Sciences Retreat at EPFL.
• September 2022: I presented CEBRA at the Bernstein Conference 2022 in Berlin.
• October 2021: I discussed our work on batch norm adaptation and robustness and gave a workshop on implementing machine learning experiments at the MSNE Project Week 2021.
• June 2021: We presented “Contrastive Learning Inverts the Data Generating Process” at ICML 2021.
• May 2021: We talked about self-learning for adaptation at the ICLR workshop on weakly supervised learning (contributed talk).

Software

• Code for the CEBRA toolbox introduced and used in our paper “Learnable latent embeddings for joint behavioral and neural analysis” will be released upon publication of the paper. Watch the Github repository for notifications on the release date.
• Code for our paper Improving robustness against common corruptions by covariate shift adaptation and Adapting ImageNet-scale models to complex distribution shifts with self-learning available as part of our robusta software package on Github.
• (vq-) wav2vec code and models released as part of fairseq.
• PyTorch toolbox for Semi-supervised Adaptive Learning Across Domains (salad) for a fair and reproducible evaluation of current domain adaptation algorithms.

Press

• wav2vec is presented at Facebook F8 2019. May 1, 2019.
• Facebook details wav2vec, an AI algorithm that uses raw audio to improve speech recognition. Venturebeat, Nov 5, 2019.

I am always looking for motivated students interested in joining me in the Bethge and/or Mathis lab. If you’re interested in working with me on topics around robustness, domain adaptation, reinforcement learning and self-supervised learning at the intersection of neuroscience and machine learning, please contact me at steffen@bethgelab.org.

I am currently working with:

• Jin Hwa Lee (now PhD student at UCL) joined the Bethge & Mathis labs for a summer internship in August 2020, and continued working in the Mathis lab as a Master’s thesis student and research assistant on self-supervised representation learning for neural data analysis [Paper].
• … a few incoming Master’s students I’ll announce here soon …

In the past, I also worked with:

• Shubham Krishna (now DevOps/MLOps) started as a research assistant in the Bethge lab in April 2020. Shubham works on topics around invariant representation learning and is currently doing a Master’s thesis with Bosch AI [Paper].
• Mert Yüksekgönül (now PhD student in Stanford). Mert worked on 3D lifting approaches for analyzing behavioral data.
• Khushdeep Singh (now Research Engineer at INRIA) joined the Bethge lab as a Master thesis student in April 2020. Khushdeep worked on benchmarking and improving the robustness of reinforcement learning algorithms with self-supervised learning. [Paper]
• Xingying Chen was a summer intern in the Bethge & Mathis labs from August - November 2020, modeling adaptation paradigms in neuroscience using reinforcement learning.
• Jan Hansen-Palmus from the Bringmann group at Uni Tübingen was co-advised by Evgenia Rusak and me. He wrote his Bachelor’s thesis on constrained optimization approaches for pseudo-labeling.