For each of the two models (a, c, e, g, i Model 1; b, d, f, h, j, Model 2) and each size N (a, b, N = 25; c, d, N = 50; e, f, N = 100; g, h, N = 200; i, j, N = 400), we plot the description length of the MDL model identified by the Bayesian machine scientist, averaged over 40 realizations of the training dataset D (colored symbols). For each model and N, we also plot the theoretical description length of the true generating model m * (Eq. (6); solid black line) and of the trivial model m c (Eq. (7); dotted black line). As in Fig. 3, colored vertical lines are estimates of the learnability transition point at which H(m*) = H(m^c) (Eq. (8)). Right of this point (gray region; unlearnable phase) H(m*) > H(m^c), so the true model cannot be learned, from the data alone, by any method.
