1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
|
* Autoassociative memory
is where a partial, or distorted memory produces a previously known
memory. Our starting point here is the open access conference paper
Krotov & Hopfield 2016, Dense Associative Memory for Pattern
Recognition.
Hopfield networks are a deep single layer of spiking neurons connected
in a way that closely tracks some models of animal neurons, formulated
as a Lyapunov function which means if we think long enough, we will
converge to a specific memory.
* Wait, what?
Instead of starting from the beginning or the end I'm going to start
from the middle. The following is a useful common lisp implementation
of Hopfield nets using a rectified quadratic function. Details of
the common lisp package are in the appendix.
The rest of this book will be a deconstruction of this function with
reimplementation in interlisp as a lispusers contribution.
This lisp is intended to both be useable and offputting motivating the
adoption of the interlisp programming system, and a set of logical and
stylistic changes and optimisations.
#+begin_src lisp
(defun closure-hop-net (number-bits)
(let ((potential (make-array number-bits :element-type '(integer 0 1)))
(memories (list))
(formatting-output "Potential: ~s~%Memories:~%~{~s~%~}"))
(labels ((rect-poly-2 (x) (if (< x 0) 0 (expt x 2)))
(b2s (x) (expt -1 (1+ x)))
(signed-idx (sgn memory idx)
(rect-poly-2
(+ (* sgn (b2s (aref memory idx)))
(loop for pot across potential
for mem across memory
for count from 0
summing
(cond
((= count idx) 0)
(t (* (b2s mem) (b2s pot))))))))
(local-update (idx)
(setf
(aref potential idx)
(if
(minusp
(loop for memory in memories
summing
(- (signed-idx +1 memory idx)
(signed-idx -1 memory idx))))
0 1))))
(lambda (&key push-memory pop-memory format update set-potential)
(cond
(set-potential (setf potential set-potential))
(push-memory (push push-memory memories))
(pop-memory (pop memories))
(format (when (stringp format) (setf formatting-output format))
(format nil formatting-output potential memories))
(update (if (numberp update) (local-update update)
(loop for idx below (length potential)
do (local-update idx)))))))))
#+end_src
what a mouthful. Garbage collected bit-array closures. Updates are
performed from raw memories every time, which is a niche use case.
However it is a hopfield net and does work.
* The xor example
All the common lisp 2e compilers use asdf. asdf package details for
binry-hop are in the appendix. The following was performed
interactively using the popular sbcl compiler hosted on
tilde.institute.
#+begin_src lisp
(require :asdf)
(require "binry-hop")
(in-package :hop)
(defparameter *eg* (closure-hop-net 3))
(dolist (mem '(#*000 #*110 #*101 #*011)) (funcall *eg* :push-memory mem))
(funcall *eg* :set-potential #*100)
(print (funcall *eg* :format t))
#+end_src
Outputting
#+begin_example
Potential: #*100
Memories:
#*011
#*101
#*110
#*000
#+end_example
We are using the first two bits as the XOR inputs, and the third bit
as the output. Our memories for the XOR truth table, but our input is
incorrect according to our memories.
#+begin_src lisp
(funcall *eg* :update 2)
(funcall *eg* :format t)
#+end_src
We did it!
#+begin_example
Potential: #*101
Memories:
#*011
#*101
#*110
#*000
#+end_example
A hopfield net converges to one of its memories, and stops changing
for one of its inputs. The updates can be performed breadth first or
randomly (Monte Carlo). In this case we cheated and just updated the
only bit we were interested in (the output bit).
Over the next chapters I will first explore and develop programming a
simple game in terms of hopfield nets, then rewriting, and begin
exploring once comfortably inside the interlisp programming system.
* Side note on a feature of common lisp
One intended use case for common lisp is portability. The same common
lisp code is expected to be common between very different
environments. One effect of this is that while it is easy to port
interlisp into common lisp by editing out particular interlisp
features, it is relatively difficult to adapt common lisp to the more
idiomatic interlisp. However this experience is expected to be
instructive. The above common lisp package can be used freely in lisp,
including embedding in C as it stands; but we will not try and explore
in C.
|