After many trials and errors I’m starting to see the light… First signs of invariance, and how that might work, are here..
It’s a very small matrix with only 2 layers (in fact there are 8 layers if I count input/output and inhibitory layers), 4×4. If pattern is moved on the right side, then another set of inhibitory neurons control the output, and the response is very different, like having an entirely different pattern. I was theorizing that this is how it should behave, but I was never very sure. Also I observed clustering that I have never predicted, but in hindsight I should have. They were messing my beautiful patterns.. so I inhibited the secondary dendrites as well… all possible dendrites are now controlled by inhibitory neurons , still I see asymmetries that are troublesome.
Can this be coincidental ? Yes and no, yes because the patterns are not perfectly as predicted, no, because is close to what I was predicting, and prediction show that it should work.
Anyway some things are clear… This invariance, can only work on limited fields. Theoretically the size of the field is directly proportional to the number of hidden layers (the bigger the field, the more hidden layers). And this is where I’m going next… But there still work to be done here on this tiny set-up.