Randomized algorithms for very large matrix problems have received a great deal of attention in recent years. Much of this work was motivated by problems in large-scale data analysis, largely since matrices are popular structures with which to model data drawn from a wide range of application domains, and the success of this line of work opens the possibility of performing matrix-based computations with truly massive data sets. Originating within theoretical computer science, this work was subsequently extended and applied in important ways by researchers from numerical linear algebra, statistics, applied mathematics, data analysis, and machine learning, as well as domain scientists. It provides a detailed overview, appropriate for both students and researchers from all of these areas, of recent work on the theory of randomized matrix algorithms as well as the application of those ideas to the solution of practical problems in large-scale data analysis. By focusing on ubiquitous and fundamental problems such as least-squares approximation and low-rank matrix approximation that have been at the center of recent developments, an emphasis is placed on a few simple core ideas that underlie not only recent theoretical advances but also the usefulness of these algorithmic tools in large-scale data applications.