In the past I tried to structure input by inputting lines of fixed length like shown here:
but that seem tedious and had also some hidden problems.
Now, I have to have some initial activation in all layer before any links are formed with the previous layer.. They are random now:
but maybe I can make them not random, so I obtain some structure even before the fist input is seen. I have played with this idea in the past, starting a wave from a corner, but I don’t even remember why I abandoned it..
That’s because I can’t envision any way to have a structured input. In a way I need to break the symmetry so A can be bigger than B as described in my previous post. I’ll start waves from the center of the image field. Not sure if I need to do this in all layers or only in the first hidden layer. I’m thinking that if I do it in the first layer, then the second layer will already have structured input, otherwise I risk some sort of opposing training in first layer vs second layer..