I added in the inter-layer neurons, hoping to solve some problems related to network transmission. Basically, because I don’t use full connections and because the neurons have to find their partners by themselves, I ended up with some non-trivial problems, if neurons are too far away on a 2D surface, they can’t find each other, so I thought I’d use other “inter-layer” neurons, to link them..
Adding inter-layer neurons did not help with long range connections but it did do an interesting thing, now it can reconstruct learned pattern using only parts of the patterns, some sort of a localized Hopfield network. Running a pattern many times eventually excites other patterns from the surrounding neighborhood, and, at the extreme, it excites all the network… all layers.. a total confusion… That’s the reason I need inhibitory neurons, to stop that uncontrolled excitation.
Anyway, this new observation, doesn’t help me at all. Over the weekend I watched some youtubes on neuroscience looking for inspiration .. very disappointing, there is an inflation of scientists trying to feed their families. So I decided to restrict my feed to well established universities, other talks promise much and deliver nothing. Now I’m watching the 12 lectures from Allen Institute: Coding and Vision 101.
So where I am at ? I can detect lines of some orientations, not all, but then I can’t link lines of the same orientations to a single neuron in the next layer because they are far apart and I build the network in the assumption the neurons link to neurons close by, don’t travel the world to find their pair. For long range links I was envisioning the use of inter-layer neurons … Even if everything would work perfectly, I still don’t see the next step, in the brain, simple cells link with multiple complex cells responding to the same type of signal… to what purpose ? How does that help in understanding the shape of an object ? Are they there just for invariance some sort of pooling to account for different positions on a surface, and basically have nothing to do with shape recognition ?