Thanks for this explanation. I couldn't wrap my mind around the "differentiable hash table" analogy, but "distribution of keys" -> "distribution of values" starts to click.
I'm not an ML expert either but I have taken graduate level courses and published papers with "machine learning" in the title, so I feel like I should be able to understand these things better. The field just moves so fast. It's a lot of work to keep up. Easy-to-digest explanations like this are underrated.
>The field just moves so fast. It's a lot of work to keep up. Easy-to-digest explanations like this are underrated.
This is really the truth. I can't possibly understand how people in this field who are talented can still keep up. I have a binder full of seminal papers that I have to cull to make room for more recent and relevant research every few months. I feel there is a lot of potential in simplifying the details of the mechanisms that drive a lot of it, but nobody has time to stop, consolidate the information and publish it. And if they did, it would just be another outdated textbook in a few years.
I'm not an ML expert either but I have taken graduate level courses and published papers with "machine learning" in the title, so I feel like I should be able to understand these things better. The field just moves so fast. It's a lot of work to keep up. Easy-to-digest explanations like this are underrated.