Max Vladymyrov

Max Vladymyrov

Senior Research Scientist, Google DeepMind

About

At Google DeepMind, I focus on developing novel machine learning architectures to advance in-context learning and mechanistic interpretability. My goal is to create efficient, interpretable architectures that enhance AI adaptability and trustworthiness.

Prior to joining DeepMind, I spent two years at Yahoo Labs. I completed my PhD at UC Merced, focusing on large-scale dimensionality reduction problems. I hold two master's degrees in Computer Science and International Economic Relations, and a bachelor's degree in Applied Mathematics, all from Kharkiv National University in Ukraine.

Research Statement

With over a decade of experience in machine learning research, my current work focuses on in-context learning and mechanistic interpretability. This builds upon my foundation in meta-learning, nonlinear optimization, and manifold learning. I aim to develop novel, efficient architectures that enhance AI adaptability and interpretability. My ultimate goal is to create AI systems that are not only powerful, but also inherently understandable, trustworthy, and beneficial to humanity.

Publications

Topic

Venue

NeurIPS

Linear Transformers are Versatile In-Context Learners

Max Vladymyrov, Johannes von Oswald, Mark Sandler, Rong Ge

38th Annual Conference on Neural Information Processing Systems (NeurIPS 2024)

arXiv
ICML Workshop

Efficient Linear System Solver with Transformers

Max Vladymyrov, Johannes von Oswald, Nolan Miller, Mark Sandler

AI for Math Workshop, ICML 2024

OpenReview Poster
Preprint

Narrowing the Focus: Learned Optimizers for Pretrained Models

Gus Kristiansen, Mark Sandler, Andrey Zhmoginov, Nolan Miller, Anirudh Goyal, Jihwan Lee, Max Vladymyrov

arXiv:2408.09310

arXiv
ICML Workshop

Learning and Unlearning of Fabricated Knowledge in Language Models

Chen Sun, Nolan Miller, Andrey Zhmoginov, Max Vladymyrov, Mark Sandler

Mechanistic Interpretability Workshop, ICML 2024

OpenReview
ICML Workshop

Learning Fast and Slow: Representations for In-Context Weight Modulation

Andrey Zhmoginov, Jihwan Lee, Max Vladymyrov, Mark Sandler

Workshop on In-Context Learning, ICML 2024

OpenReview
TMLR

Continual HyperTransformer: A Meta-Learner for Continual Few-Shot Learning

Max Vladymyrov, Andrey Zhmoginov, Mark Sandler

Transactions on Machine Learning Research, 2024

arXiv OpenReview Poster
Preprint

Uncovering Mesa-Optimization Algorithms in Transformers

Johannes von Oswald, Eyvind Niklasson, Maximilian Schlegel, Seijin Kobayashi, Nicolas Zucchet, Nino Scherrer, Nolan Miller, Mark Sandler, Max Vladymyrov, Razvan Pascanu, João Sacramento

arXiv:2309.05858

arXiv
ICML

Transformers Learn In-Context by Gradient Descent

Johannes von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, Max Vladymyrov

International Conference on Machine Learning (ICML 2023)

arXiv Poster
Preprint

Training Trajectories, Mini-Batch Losses and the Curious Role of the Learning Rate

Mark Sandler, Andrey Zhmoginov, Max Vladymyrov, Nolan Miller

arXiv preprint arXiv:2301.02312

arXiv
CVPR

Decentralized Learning with Multi-Headed Distillation

Andrey Zhmoginov, Mark Sandler, Nolan Miller, Gus Kristiansen, Max Vladymyrov

Computer Vision and Pattern Recognition (CVPR 2023)

arXiv Poster
ICML

HyperTransformer: Model Generation for Supervised and Semi-Supervised Few-Shot Learning

Andrey Zhmoginov, Mark Sandler, Max Vladymyrov

International Conference on Machine Learning (ICML 2022)

arXiv Poster Video
ICLR

GradMax: Growing Neural Networks Using Gradient Information

Utku Evci, Bart van Merrienboer, Thomas Unterthiner, Max Vladymyrov, Fabian Pedregosa

International Conference on Learning Representations (ICLR 2022)

arXiv Poster
CVPR

Fine-Tuning Image Transformers Using Learnable Memory

Mark Sandler, Andrey Zhmoginov, Max Vladymyrov, Andrew Jackson

Computer Vision and Pattern Recognition (CVPR 2022)

arXiv Poster
JMLR

Underspecification Presents Challenges for Credibility in Modern Machine Learning

Alexander D'Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D. Hoffman, Farhad Hormozdiari, Neil Houlsby, Shaobo Hou, Ghassen Jerfel, Alan Karthikesalingam, Mario Lucic, Yian Ma, Cory McLean, Diana Mincu, Akinori Mitani, Andrea Montanari, Zachary Nado, Vivek Natarajan, Christopher Nielson, Thomas F. Osborne, Rajiv Raman, Kim Ramasamy, Rory Sayres, Jessica Schrouff, Martin Seneviratne, Shannon Sequeira, Harini Suresh, Victor Veitch, Max Vladymyrov, Xuezhi Wang, Kellie Webster, Steve Yadlowsky, Taedong Yun, Xiaohua Zhai, D. Sculley

Journal of Machine Learning Research, 2022

arXiv Video
ICML

Meta-Learning Bidirectional Update Rules

Mark Sandler, Max Vladymyrov, Andrey Zhmoginov, Nolan Miller, Andrew Jackson, Tom Madams, Blaise Agüera y Arcas

38th International Conference on Machine Learning (ICML 2021), pp. 9288-9300

arXiv Poster Slides GitHub
NeurIPS

No Pressure! Addressing the Problem of Local Minima in Manifold Learning Algorithms

Max Vladymyrov

33th Annual Conference on Neural Information Processing Systems (NeurIPS 2019), pp. 678-687

arXiv Poster Slides
IJCNN

Fast, Accurate Spectral Clustering Using Locally Linear Landmarks

Max Vladymyrov, Miguel Á. Carreira-Perpiñán

30th International Joint Conference on Neural Networks (IJCNN 2017), pp. 3870-3879

Preprint Slides
ICML

The Variational Nyström Method for Large-Scale Spectral Problems

Max Vladymyrov, Miguel Á. Carreira-Perpiñán

33th International Conference on Machine Learning (ICML 2016)

Preprint Supplementary Reviews Poster Slides
NeurIPS

A Fast, Universal Algorithm to Learn Parametric Nonlinear Embeddings

Miguel Á. Carreira-Perpiñán, Max Vladymyrov

29th Annual Conference on Neural Information Processing Systems (NIPS 2015), pp. 253-261

Conference Preprint Supplementary Reviews Poster
AISTATS

Linear-Time Training of Nonlinear Low-Dimensional Embeddings

Max Vladymyrov, Miguel Á. Carreira-Perpiñán

17th International Conference on Artificial Intelligence and Statistics (AISTATS 2014), pp. 968-977

Preprint Supplementary Poster
ECML-PKDD

Locally Linear Landmarks for Large-Scale Manifold Learning

Max Vladymyrov, Miguel Á. Carreira-Perpiñán

24th European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD 2013), pp. 256-271

Paper Preprint Poster Slides
ICML

Entropic Affinities: Properties and Efficient Numerical Computation

Max Vladymyrov, Miguel Á. Carreira-Perpiñán

30th International Conference on Machine Learning (ICML 2013), pp. 477-485

Paper Preprint Supplementary Slides Poster
ICML

Partial-Hessian Strategies for Fast Learning of Nonlinear Embeddings

Max Vladymyrov, Miguel Á. Carreira-Perpiñán

29th International Conference on Machine Learning (ICML 2012), pp. 345-352

arXiv Preprint Supplementary Slides Poster