This book answers, in the form of short and entertaining vignettes, the question: "What do mathematicians really do?" Readers will learn that mathematicians use numbers in the same way that novelists use letters. The individual letters are typed while the author thinks on a much grander scale, invisible to the observer. Requiring only familiarity with the multiplication table (and that for only one vignette), the book makes accessible a variety of mathematical concepts, such as game theory, chaos, and traffic flow modelling. The author accomplishes this with a light, engaging style, and a range of real-world examples that includes everything from barbershops to President James Garfield. Mathematicians Don't Work With Numbers will be of interest to the large audience of people who have always assumed that mathematicians do, in fact, work with numbers.