Top AI & ML Research Papers

#1. Learning to Simulate Complex Physics with Graph Networks

Authors: Alvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, Peter W. Battaglia

Summary: Here we present a general framework for learning simulation, and provide a single model implementation that yields state-of-the-art performance across a variety of challenging physical domains, involving fluids, rigid solids, and deformable materials interacting with one another.

#2. A Primer in BERTology: What we know about how BERT works

Authors: Anna Rogers, Olga Kovaleva, Anna Rumshisky

Summary: Transformer-based models are now widely used in NLP, but we still do not understand a lot about their inner workings. This paper describes what is known to date about the famous BERT model.

#3. StyleGAN2 Distillation for Feed-forward Image Manipulation

Authors: Yuri Viazovetskyi, Vladimir Ivashkin, and Evgeny Kashin

Summary: StyleGAN2 is a state-of-the-art network in generating realistic images. Latent code optimization via back propagation is commonly used for qualitative embedding of real world images, although it is prohibitively slow for many applications.

#4. Learning to Shadow Hand-drawn Sketches

Authors: Qingyuan Zheng, Zhuoru Li, and Adam Bargteil

Summary: We present a fully automatic method to generate detailed and accurate artistic shadows from pairs of line drawing sketches and lighting directions. The generated shadows respect the hand-drawn lines and underlying 3D space and contain sophisticated and accurate details, such as self-shadowing effects.

#5. Lagrangian Neural Networks

Authors: Miles Cranmer, Sam Greydanus, Stephan Hoyer, Peter Battaglia, David Spergel, Shirley Ho

Summary: Accurate models of the world are built upon notions of its underlying symmetries. In contrast to models that learn Hamiltonians, LNNs do not require canonical coordinates, and thus perform well in situations where canonical momenta are unknown or difficult to compute.

#6. Benchmarking Graph Neural Networks

Authors: Vijay Prakash Dwivedi, Chaitanya K. Joshi, Thomas Laurent, Yoshua Bengio, Xavier Bresson

Summary: Graph neural networks (GNNs) have become the standard toolkit for analyzing and learning from data on graphs. We apply this benchmarking framework to novel medium-scale graph datasets from mathematical modeling, computer vision, chemistry and combinatorial problems to establish key operations when designing effective GNNs.

#7. MLIR: A Compiler Infrastructure for the End of Moore’s Law

Authors: Chris Lattner, Mehdi Amini, Uday Bondhugula, Albert Cohen, Andy Davis, Jacques Pienaar, River Riddle, Tatiana Shpeisman, Nicolas Vasilache

Summary: This work presents MLIR, a novel approach to building reusable and extensible compiler infrastructure. Evaluation of MLIR as a generalized infrastructure that reduces the cost of building compilers-describing diverse use-cases to show research and educational opportunities for future programming languages, compilers, execution environments, and computer architecture.

#8. TensorFlow Quantum: A Software Framework for Quantum Machine Learning

Authors: Michael Broughton, Guillaume Verdon, Trevor McCourt, Antonio J. Martinez, Jae Hyeon Yoo, Sergei V. Isakov, Philip Massey, Murphy Yuezhen Niu, Ramin Halavati, Evan Peters, Martin Leib, Andrea Skolik, Michael Streif, David Von Dollen, Jarrod R. McClean, Sergio Boixo, Dave Bacon, Alan K. Ho, Hartmut Neven, Masoud Mohseni

Summary: We introduce TensorFlow Quantum (TFQ), an open source library for the rapid prototyping of hybrid quantum-classical models for classical or quantum data. This framework offers high-level abstractions for the design and training of both discriminative and generative quantum models under TensorFlow and supports high-performance quantum circuit simulators.

#9. Second Order Optimization Made Practical

Authors: Rohan Anil, Vineet Gupta, Tomer Koren, Kevin Regan, Yoram Singer

Summary: Optimization in machine learning, both theoretical and applied, is presently dominated by first-order gradient methods such as stochastic gradient descent. Second-order optimization methods that involve second-order derivatives and/or second-order statistics of the data have become far less prevalent despite strong theoretical properties, due to their prohibitive computation, memory and communication costs. We demonstrate superior performance on very large learning problems in machine translation where our distributed implementation runs considerably faster than existing gradient-based methods.

#10. SLIDE: In Defense of Smart Algorithms Over Hardware Acceleration for Large-scale Deep learning Systems

Authors: Beidi Chen, Tharun Medini, James Farwell. Sameh Gobriel, Charlie Tai, Anshumali Shrivastava

Summary: Deep Learning (DL) algorithms are the central focus of modern machine learning systems. The progress on the algorithmic front has failed to demonstrate a direct advantage over powerful hardware such as NVIDIA-V100 GPUs. Using just a CPU, SLIDE drastically reduces the computations during both training and inference outperforming an optimized implementation of Tensorflow (TF) on the best available GPU. On the same CPU hardware, SLIDE is over 10x faster than TF.