I’ve already done many simulations with the new model. There’s no invariance in sight. I also have no theories that would predict this elusive invariance, so much so that I’m not so sure anymore that this is obtained at the neuronal level. So I’ve prepared plan B. Even without invariance learn as many patterns possible, very much like the regular ML used these days. Well, that also failed. When active, neurons send a call to other neurons looking for binding partners. But how are the neurons active in the first place ? To solve this problem I linked from the beginning, neuron ij from L1 to same ij neuron from L2. That is proving to be a limiting factor now since in L2 I need to be able to form more patterns than in L1. Even if L2 has more neurons, they never get activated and they never connect to anything. In my previous post I showed a 2by2 matrix learning a 2 pixel pattern, but the cross lines were missing as a pattern because I did not have enough neurons in L2 to learn those patterns.
Anyway I need to make neurons activate “spontaneously”, till they make their first connection at least, see if that solves my problem.. Why not make fully connected layers ? Theoretically, that should work too but is a bit impractical because training takes a lot of time. Also this random activation is not as simple as it seems, because it will result very fast in a fully connected network.