How to teach yourself about AI

Recommended reading from easy to graduate level.

Production note: Posting has been slow due to travel and grant/teaching deadlines. In the pipeline are some upcoming posts on anti-sense oligos as therapy, neoantigen vaccines, new developments in spatial transcriptomics, benchmarking AI models of gene regulation, etc. This newsletter is meant to be much more than link roundups and book recommendations, but today I need to make due with some book recommendations. I hope these are useful.

One of the most important abilities you need for a successful career is knowing how to update your skills. As I tell my kids, this isn’t just important advice for scientists, who work in a profession that is supposed to generate new knowledge and new technology, and thus is always changing. The New York Times recently covered the “Gen X career meltdown”, describing the challenges of media professionals who entered journalism, advertising, film, etc. in the 1990’s and early 2000’s. The media business looks nothing like it did 20-30 years ago.

The same is true of much of biology. When I started grad school, a little over 20 years ago, I had no sense whatsoever of the pace of change in science. While some of the skills I learned in grad school (e.g., yeast molecular genetics) are still useful, most of them, in my graduate field of functional and structural genomics, haven’t withstood the transformative impact of next-generation sequencing, CRISPR, cryoEM, deep learning, long-read sequencing, and high-throughput imaging. What did withstand those transformations was solid training in scientific reasoning and in the ability to teach myself what I needed to learn. Back then, when the draft human genome sequence was still hot off the press, some of my fellow grad students and I decided to teach ourselves Perl and R so that we could be better participate in this trendy field called genomics.

In that spirit, I want to encourage everyone learn more about AI, including scientists whose training and work may be mostly non-computational. Especially if you’re still early in your career, you have time to build your skills and bring deep learning into your work. To be clear, if you’re serious about doing computational biology, you’ll need to do more than just read some books; you’ll need to spend some time training with real computational biologists. But it is possible to, in the words of Harvard computational biologist Sean Eddy, “ go where a question takes you, not where your training left you.” Eddy described his own background, in this great 2005 piece, “Antedisciplinary Science”:

I've been a computational biologist for about 15 years now. We're still not quite sure what “computational biology” means, but we seem to agree that it's an interdisciplinary field, requiring skills in computer science, molecular biology, statistics, mathematics, and more. I'm not qualified in any of these fields. I'm certainly not a card-carrying software developer, computer scientist, or mathematician, though I spend most of my time writing software, developing algorithms, and deriving equations. I do have formal training in molecular biology, but that was 15 years ago, and I'm sure my union card has expired. For one thing, they all seem to be using these clever, expensive kits now in my wet lab, whereas I made most of my own buffers (after walking to the lab six miles in the snow, barefoot).

For the antedsciplinary scientists (and non-scientists) out there, here are my recommendations for books to learn AI. They range from gentle and popular introductions to fantastic textbooks that rigorously cover the math. (Eddy’s piece, by the way, helped tip the scales in favor of my decision to come to Washington University in St. Louis, where Eddy was at the time. I wanted to be around people who thought like that.)

1. The Hundred-Page Machine Learning Book, Andriy Burkov

If you want to spend one weekend jump starting your AI education, this is the book. Anyone with decent high school math skills can pick this up and learn about probability, linear and logistic regression, decision trees, SVMs, basic learning algorithms, and your standard neural networks. There are some small code examples in Python. It’s well-written, well-illustrated and covers all of the basics. After reading this book you’ll know the basic vocabulary of machine learning, have a general idea of what the basic algorithms are, and know how they are applied to some standard problems. I recommend this book as your first foray into AI because it includes accessible math, rather than only vague verbal descriptions of what AI does. You can download chapters for free to try before you buy.

2. AI Snake Oil, Arvind Narayanan and Sayash Kapoor

A colleague serving on a university AI committee with me recommended this book. Read this book, by two computer scientists, to teach yourself not to be credulous about AI. The authors write a popular Substack bearing the same name of the book (see the previous link), and their big argument is this: predictive AI is wildly overrated, performing worse than advertised, while generative AI is much more useful. They are very skeptical that AGI (artificial general intelligence) is imminent, and find AGI itself to not be a very coherent concept. It’s a good book, another one you can read in a weekend.

The most important message, not emphasized enough in contemporary discussions of AI, is that we should be very skeptical about AI performance claims, because so often models that perform well on benchmark datasets used during development will fail when applied in another context. Domain shift is when a model is asked to make predictions in a setting that is systematically different from its training data, and it is one of the most difficult problems to solve in AI. But that fact doesn’t stop snake oil salesmen from overselling their models to predict everything from insurance risks to recidivism to product recommendations. As a culture, we should be much more skeptical of predictive AI.

3. Why Machines Learn, Anil Anathaswamy

AI pioneer Geoffrey Hinton blurbed this book as “a masterpiece,” and he’s right. If you want to read a popular-level book that covers the math and history of AI, in a non-technical way, this is the book to read. It’s pitched at about the level of Cornell mathematician Steven Strogatz’s book Infinite Powers or Johns Hopkins physicist Sean Carroll’s Biggest Ideas in the Universe books. Anathaswamy takes the story of AI from the perceptron in the 1950’s up to today’s deep learning models, while very gently presenting the math along the way. He’s interviewed many of the pioneers in the field, who describe what it felt like to work at various stages in machine learning history. Read this book to build your intuition for the field, learn some history, and to get a lucid layperson’s explanation of Bayes’ theorem, matrices and vectors, maximum likelihood estimation, and the other important math that makes AI work.

4. The Science of Deep Learning, Iddo Drori

Now we get to real textbooks. Iddo Drori wrote this concise textbook based on courses he taught. If you have a decent background in calculus and probability, you can dive right in to deep learning with this book. It starts with forward and backpropagation, and covers optimization, various deep learning architectures, including transformers, generative models, and reinforcement learning. If you’re busy with your day job but want to learn the math behind the critical backpropagation algorithm, or to understand how transformers work, this textbook has what you want.

Along with the clear explanations and rigorous math, there are very helpful illustrations. When it comes to math, I’m a visual learner. I can parse the math in a condensed description of backpropagation, but I can’t get an intuition about how it works without pictures. Drori’s illustrations are very helpful for building intuition. Some of the chapters are free online - see the link above.

5. Probabilistic Machine Learning, Kevin Murphy

This is a recent update of a classic textbook. In contrast to Drori’s concise book, which starts in media res with backpropagation, this book starts at the beginning with a rigorous presentation of basic probability and statistics. While in principle anyone could pick up this book and learn probability from scratch, the reality is that the notation and concepts, while clearly presented, will be hard to follow if you don’t have some college-level background in this field. However, the good news is that, if you have had some introductory, college-level calculus, probability, and statistics, this book has everything you need to learn the math of AI.

The book is free online via Github and has tons of figures and code examples, all of which can be downloaded as jupyter notebooks. For advanced, up-to-date topics, there is the 1300-page volume 2. I have a whole shelf full of math and statistics textbooks, few of which are written as clearly as this one by Kevin Murphy. If you are really serious about learning the mathematical foundations of AI, and ready to invest a substantial chunk of time, this is the book to use.

The one thing missing from these recommendations is programming for AI. There are plenty of good books out there; I like this one from the publisher Packt.

If you like recommendations for learning scientific topics like this, I suggest checking out John Baez’s How to Learn Math and Physics.