Jedd Haberstro
148 followers
148 followers
Posts
Post has attachment
Learning vs Inference
This will be a short post. In machine learning, the terms "learning" and "inference" are used often and and it's not always clear what is meant. For example, is "variational inference" and neural network "inferencing" the same thing? Usually not! When the d...
Post has attachment
Why is the ELBO difficult to optimize?
The task of bayesian inference is to compute the posterior p(z | x) of the model p(x, z) = p(x|z)p(z) where z is a latent variable. This is often intractable, so a bayesian may resort to approximating it with some easier distribution q(z) — this method is c...
Post has attachment
Expectation Maximization vs Variational Bayes
I constantly find myself forgetting the details of the EM algorithm, variational bayes, and what exactly the difference is between the two. To avoid confusion in the future, I wrote the following note. Q: What is the difference between EM and Variational Ba...
Post has attachment
Frequentists vs Bayesians
A really wonderful aspect of learning about machine learning is that you can't help but learn about the field statistics as well. As a computer scientist (or really, a software engineer -- I have a hard time calling myself a computer scientist), one of the ...
Post has attachment
An intuition for Newton's method
During my lazy weekend afternoons (and all the other days of the week) I've been going through Nando de Freitas' undergraduate machine learning course on youtube . In lecture 26 he introduces gradient descent, an iterative algorithm for optimizing (potentia...
Post has attachment
On optimizing high-dimensional non-convex functions
Excuse me for the (in)completeness of this post. What follows is merely a thought, inspired by two independent statements, about a domain of science (or math, really) with which I am barely initiated. Let me give you these two statements first. In the book ...
Post has attachment
TIL: premultiplied-alpha colors
Alpha is the measure of how translucent an object is. An alpha of 0.0 means the object is entirely transparent, an alpha of 1.0 means the object is entirely opaque, and an alpha in the middle means a fraction of the total light may passthrough the object. T...