The goal of learning is usually to find a model which delivers good generalization performance over an underlying distribution of the data. Consider an input space\(\mathcal{X}\) and output space\(\mathcal{Y}\). Assume the pairs\((X \times Y ) \in \mathcal{X}\times \mathcal{Y}\) are random variables whose (unknown) joint distribution isP_XY. It is our goal to find a predictor\(f : \mathcal{X}\mapsto \mathcal{Y}\) which minimizes the expected risk:

whereδ(z) = 1 ifz is true, and 0 otherwise.

However, in practice we only haven pairs of training examples (X_i,Y_i) drawn identically and independently fromP_XY. SinceP_XY is unknown, we often use the risk on the training set (called empirical risk) as a surrogate of the expected risk on the underlying distribution:

Empirical Risk...

Authors and Affiliations

1. NICTA, Australian National University, Canberra, ACT, Australia

   Xinhua Zhang

2. School of Computer Science, Australian National University, Canberra, ACT, Australia

   Xinhua Zhang

3. NICTA London Circuit, Canberra, ACT, Australia

   Xinhua Zhang

Corresponding author

Correspondence toXinhua Zhang.

Editors and Affiliations

1. The University of New South Wales, Sydney, NSW, Australia

   Claude Sammut

2. Faculty of Information Technology, Monash University, Melbourne, VIC, Australia

   Geoffrey I. Webb

Reprints and permissions

© 2017 Springer Science+Business Media New York

Policies and ethics