本书主要介绍神经网络与深度学习中的基础知识、主要模型（卷积神经网络、递归神经网络等）以及在计算机视觉、自然语言处理等领域的应用。

The formalism of probabilistic graphical models provides a unifying framework for capturing complex dependencies among random variables, and building large-scale multivariate statistical models. Graphical models have become a focus of research in many statistical, computational and mathematical fields, including bioinformatics, communication theory, statistical physics, combinatorial optimization, signal and image processing, information retrieval and statistical machine learning. Many problems that arise in specific instances-including the key problems of computing marginals and modes of probability distributions-are best studied in the general setting. Working with exponential family representations, and exploiting the conjugate duality between the cumulant function and the entropy for exponential families, Graphical Models, Exponential Families and Variational Inference develops general variational representations of the problems of computing likelihoods, marginal probabilities and most probable configurations. It describes how a wide variety of algorithms- among them sum-product, cluster variational methods, expectation-propagation, mean field methods, and max-product-can all be understood in terms of exact or approximate forms of these variational representations. The variational approach provides a complementary alternative to Markov chain Monte Carlo as a general source of approximation methods for inference in large-scale statistical models.

概率图模型将概率论与图论相结合，是当前非常热门的一个机器学习研究方向。本书详细论述了有向图模型（又称贝叶斯网）和无向图模型（又称马尔可夫网）的表示、推理和学习问题，全面总结了人工智能这一前沿研究领域的最新进展。为了便于读者理解，书中包含了大量的定义、定理、证明、算法及其伪代码，穿插了大量的辅助材料，如示例（examples）、技巧专栏（skill boxes）、实例专栏（case study boxes）、概念专栏（concept boxes）等。另外，在第 2章介绍了概率论和图论的核心知识，在附录中介绍了信息论、算法复杂性、组合优化等补充材料，为学习和运用概率图模型提供了完备的基础。
本书可作为高等学校和科研单位从事人工智能、机器学习、模式识别、信号处理等方向的学生、教师和研究人员的教材和参考书。
== 序 言 ==
很高兴能够看到我们所著的《概率图模型》一书被翻译为中文出版。我们了解到这本书涵盖的课题已在中国引起了巨大的兴趣。已有众多中国读者写信向我们解释这本书对于他们的学习的重要性，并希望获得更易理解的版本。随着众多来自中国研究机构或国外研究机构的中国学者署名或共同署名的文章的发表，中国研究者已在概率图领域中扮演了非常重要的角色。这些文章对于概率图模型领域的发展起到了非常重要的作用。我们相信《概率图模型》中文版的出版将帮助许多中国读者学习并掌握这一重要课题的基础。同时，这也将进一步提高中国学者应用概率图模型思想的能力，并为这一领域的发展做出贡献。
本书的翻译工作由王飞跃研究员主导，并得到了王珏研究员及其众多助手和合作者的支持。这是一份历时 5年、具有里程碑意义的努力，我深深地感谢该团队所有为本书翻译做出贡献的人员。我尤其希望借此机会感谢王珏研究员——一位中国机器学习领域的开拓者。王珏研究员是此项翻译工作的十分重要的推动者。没有他的支持，没有他的众多杰出的机器学习领域的学生的帮助，可能这项工作到现在还没有结果。很遗憾王珏研究员于 2014年 12月死于癌症，终年 66岁，已不能看到他努力的结果。然而，他的思想活在他的学生们的工作中，与本书的出版同在。
Daphne Koller
（复杂系统管理与控制国家重点实验室王晓翻译）

Machine learning allows computational systems to adaptively improve their performance with experience accumulated from the observed data. Its techniques are widely applied in engineering, science, finance, and commerce. This book is designed for a short course on machine learning. It is a short course, not a hurried course. From over a decade of teaching this material, we have distilled what we believe to be the core topics that every student of the subject should know. We chose the title `learning from data' that faithfully describes what the subject is about, and made it a point to cover the topics in a story-like fashion. Our hope is that the reader can learn all the fundamentals of the subject by reading the book cover to cover. ---- Learning from data has distinct theoretical and practical tracks. In this book, we balance the theoretical and the practical, the mathematical and the heuristic. Our criterion for inclusion is relevance. Theory that establishes the conceptual framework for learning is included, and so are heuristics that impact the performance of real learning systems. ---- Learning from data is a very dynamic field. Some of the hot techniques and theories at times become just fads, and others gain traction and become part of the field. What we have emphasized in this book are the necessary fundamentals that give any student of learning from data a solid foundation, and enable him or her to venture out and explore further techniques and theories, or perhaps to contribute their own. ---- The authors are professors at California Institute of Technology (Caltech), Rensselaer Polytechnic Institute (RPI), and National Taiwan University (NTU), where this book is the main text for their popular courses on machine learning. The authors also consult extensively with financial and commercial companies on machine learning applications, and have led winning teams in machine learning competitions.

支持向量机是数据挖掘中的一个新方法。支持向量机能非常成功地处理回归问题(时间序列分析)和模式识别(分类问题、判别分析)等诸多问题，并可推广于预测和综合评价等领域，因此可应用于理科、工科和管理等多种学科。目前国际上支持向量机在理论研究和实际应用两方面都正处于飞速发展阶段。希望本书能促进它在我国的普及与提高。
本书对象既包括关心理论的研究工作者，也包括关心应用的实际工作者。对于有关领域的具有高等数学知识的实际工作者，略去书中的某些理论部分，仍能对支持向量机的本质有一个概括的理解，从而用它解决自己的问题。
本书适合高等院校高年级学生、研究生、教师和相关科研人员及相关领域的实际工作者使用。

During the past decade there has been an explosion in computation and information technology. With it have come vast amounts of data in a variety of fields such as medicine, biology, finance, and marketing. The challenge of understanding these data has led to the development of new tools in the field of statistics, and spawned new areas such as data mining, machine learning, and bioinformatics. Many of these tools have common underpinnings but are often expressed with different terminology. This book describes the important ideas in these areas in a common conceptual framework. While the approach is statistical, the emphasis is on concepts rather than mathematics. Many examples are given, with a liberal use of color graphics. It is a valuable resource for statisticians and anyone interested in data mining in science or industry. The book's coverage is broad, from supervised learning (prediction) to unsupervised learning. The many topics include neural networks, support vector machines, classification trees and boosting---the first comprehensive treatment of this topic in any book. This major new edition features many topics not covered in the original, including graphical models, random forests, ensemble methods, least angle regression & path algorithms for the lasso, non-negative matrix factorization, and spectral clustering. There is also a chapter on methods for "wide" data (p bigger than n), including multiple testing and false discovery rates.

The dramatic growth in practical applications for machine learning over the last ten years has been accompanied by many important developments in the underlying algorithms and techniques. For example, Bayesian methods have grown from a specialist niche to become mainstream, while graphical models have emerged as a general framework for describing and applying probabilistic techniques. The practical applicability of Bayesian methods has been greatly enhanced by the development of a range of approximate inference algorithms such as variational Bayes and expectation propagation, while new models based on kernels have had a significant impact on both algorithms and applications.
This completely new textbook reflects these recent developments while providing a comprehensive introduction to the fields of pattern recognition and machine learning. It is aimed at advanced undergraduates or first-year PhD students, as well as researchers and practitioners. No previous knowledge of pattern recognition or machine learning concepts is assumed. Familiarity with multivariate calculus and basic linear algebra is required, and some experience in the use of probabilities would be helpful though not essential as the book includes a self-contained introduction to basic probability theory.
The book is suitable for courses on machine learning, statistics, computer science, signal processing, computer vision, data mining, and bioinformatics. Extensive support is provided for course instructors, including more than 400 exercises, graded according to difficulty. Example solutions for a subset of the exercises are available from the book web site, while solutions for the remainder can be obtained by instructors from the publisher. The book is supported by a great deal of additional material, and the reader is encouraged to visit the book web site for the latest information.

Most tasks require a person or an automated system to reason--to reach conclusions based on available information. The framework of probabilistic graphical models, presented in this book, provides a general approach for this task. The approach is model-based, allowing interpretable models to be constructed and then manipulated by reasoning algorithms. These models can also be learned automatically from data, allowing the approach to be used in cases where manually constructing a model is difficult or even impossible. Because uncertainty is an inescapable aspect of most real-world applications, the book focuses on probabilistic models, which make the uncertainty explicit and provide models that are more faithful to reality. Probabilistic Graphical Models discusses a variety of models, spanning Bayesian networks, undirected Markov networks, discrete and continuous models, and extensions to deal with dynamical systems and relational data. For each class of models, the text describes the three fundamental cornerstones: representation, inference, and learning, presenting both basic concepts and advanced techniques. Finally, the book considers the use of the proposed framework for causal reasoning and decision making under uncertainty. The main text in each chapter provides the detailed technical development of the key ideas. Most chapters also include boxes with additional material: skill boxes, which describe techniques; case study boxes, which discuss empirical cases related to the approach described in the text, including applications in computer vision, robotics, natural language understanding, and computational biology; and concept boxes, which present significant concepts drawn from the material in the chapter. Instructors (and readers) can group chapters in various combinations, from core topics to more technically advanced material, to suit their particular needs.

《模式分类》(原书第2版)的第1版《模式分类与场景分析》出版于1973年，是模式识别和场景分析领域奠基性的经曲名著。在第2版中，除了保留了第1版的关于统计模式识别和结构模式识别的主要内容以外，读者将会发现新增了许多近25年来的新理论和新方法，其中包括神经网络、机器学习、数据挖掘、进化计算、不变量理论、隐马尔可夫模型、统计学习理论和支持向量机等。作者还为未来25年的模式识别的发展指明了方向。书中包含许多实例，各种不同方法的对比，丰富的图表，以及大量的课后习题和计算机练习。

《小波分析与文本文字识别》可供计算机、生物医学、地球物理、图像处理、信号处理等专业的本科生、研究生、教师及工程技术人员阅读。