An assortment of my thoughts on selected machine learning topics. I mainly pose questions, instead of answering them. I guess if I did have answers I would write a paper instead.


Dual numbers

09 August 2021, 8 minute read

I found writing this next post is a real treat. It’s about dual numbers. Dual numbers are a bit strange, to say the least, and at first they seem like an abstract mathematical fancy, but as you will see they have quite a useful purpose in the realm of automatic differentiation.


Reversible optimisers

20 December 2020, 6 minute read

Reversible neural architectures have been a popular research area in the last few years, but reversibility is also built into many modern day neural optimisers, perhaps serendipitously.


On the ‘invention’ of randomness

15 December 2019, 10 minute read

Recently in AMLAB we started a Jaynes reading group. E T Jaynes’ posthumous book and general all-round cult classic Probability Theory: The Logic of Science is the focus of our study. After having lectured a Bayesian statistics course for the last two years…