Introduce Krotov and Hopfield 2016

1 files changed, 8 insertions, 0 deletions

diff --git a/01-autoassociative-memory.txt b/01-autoassociative-memory.txt
index e69de29..715133a 100644
--- a/01-autoassociative-memory.txt
+++ b/01-autoassociative-memory.txt
@@ -0,0 +1,8 @@
+* Autoassociative memory
+
+is where a partial, or distorted memory produces a previously known memory. Our starting point here is the open access conference paper Krotov & Hopfield 2016, Dense Associative Memory for Pattern Recognition.
+
+Hopfield networks are a deep single layer of spiking neurons connected in a way that closely tracks some models of animal neurons, formulated as a Lyapunov function which means if we think long enough, we will converge to a specific memory. The conference paper presents the mathematics for what are called modern hopfield networks- which use rectified polynomial activation functions in the Lyapunov condition, which results in them being able to store very many memories compared to the number of neurons. This can be thought of as a polynomial resulting in very different heights quickly, for relatively small linear distances between memories. This is why rectified polynomials can be used rather than a rectified linear function, otherwise popular in deep learning.
+
+sm0l networks are desirable for us.
+