Michael Nielsen built RMNIST — MNIST with the training set cut down to 1, 5 or 10 examples per class — to ask a question he put plainly: people learn to recognise digits from a handful of examples, so what principle are we missing? He published baselines and left the problem open.

We generate noisy positions of Mercury, Venus, Earth, and Mars over 10 years with a real N-body integrator. Two models try to explain the data: Newton’s law of gravity (six coupled parameters) and a Fourier sum-of-sines regression (168 independent parameters). Both fit the data about equally well; the Bayesian held-out predictive log-likelihood prefers physics by 16 nats. That margin has nothing to do with which model fits training points better. It is, almost in its entirety, a quantitative measurement of David Deutsch’s “hard to vary” principle.

A speech for a philosophy meetup, translated from Danish by Claude. The Danish original is below.

Will scaling deep learning produce human-level generality, or do we need a new approach? You may have read the exchange between Scott Alexander and Gary Marcus, and felt that there are some good arguments on both sides, some bad ones, but few arguments that go beyond analogy and handwaving - arguments that would take what we know about deep learning and intelligence, and look at what that knowledge implies. If you haven’t read the exchange, here it is: SA, GM, SA, GM.