Page 1 of 1

error won't decrease while in training

PostPosted: Thu Apr 28, 2011 3:32 am
by sleepy_hollow
hi. i'll bother you again

i build aneural network with backpropagation learning teacher
i insert value of error rate = 0.1 and learning rate = 0.2, with an input float input[] = new float [80] and output float output[] = new float [36]
while in training, in first iteration error value are more than 500, and when the iteration more than 400 times the error value stack in 324.0
this error value won't decrease even the ieteration reach 5000 times

so what's wrong with my neural network???

please hlep me

thanks before

Re: error won't decrease while in training

PostPosted: Thu Apr 28, 2011 8:51 am
by andrew.kirillov
sleepy_hollow wrote:so what's wrong with my neural network???

It may happen that network itself is fine, but data is wrong. What is the range of your desired output values? If it is more than the output range of network's activation function, then it will never learn.

Also experiment with sigmoid's alpha value.

Re: error won't decrease while in training

PostPosted: Fri Apr 29, 2011 1:46 am
by sleepy_hollow
andrew.kirillov wrote:It may happen that network itself is fine, but data is wrong. What is the range of your desired output values? If it is more than the output range of network's activation function, then it will never learn.

Also experiment with sigmoid's alpha value.


my output range are -1.0 to 1.0, it's more than output from output range of network activation function, so should i change it to -0.5 to 0.5?

and when i change the alfa value, it'll will learn but the error will stack again in some different value

or may be my input and output that seems wrong, because my input are from binary image that i convert to inputvektor which 1.0 for black pixel and -1.0 for white pixel

Re: error won't decrease while in training

PostPosted: Fri Apr 29, 2011 8:20 am
by andrew.kirillov
sleepy_hollow wrote:my output range are -1.0 to 1.0, it's more than output from output range of network activation function, so should i change it to -0.5 to 0.5?

Two options:
1) Change your data to [-0.5, 0.5] range;
or
2) Change your activation function so it has output in [-1, 1] range.

Re: error won't decrease while in training

PostPosted: Fri Apr 29, 2011 10:40 am
by sleepy_hollow
andrew.kirillov wrote:1) Change your data to [-0.5, 0.5] range;

i have try it and still same, error won't decrease any further when iteration reach more than 50000
is it possible if my input or desired output was the problem?
or something else?

andrew.kirillov wrote:2) Change your activation function so it has output in [-1, 1] range.

i still don't know how to change bipolar sigmoid function (i use this) so the range will change to [-1, 1]

Re: error won't decrease while in training

PostPosted: Fri Apr 29, 2011 10:53 am
by andrew.kirillov
sleepy_hollow wrote:how to change bipolar sigmoid function (i use this) so the range will change to [-1, 1]

1) See documentation and find how to specify your own function;
2) See documentation and find that BipolarSigmoidFunction has [-1, 1] range actually.

Re: error won't decrease while in training

PostPosted: Fri Apr 29, 2011 11:21 am
by sleepy_hollow
i've try it and do more experiment about alfa value, and it's work
but the teacher will converged when error value reach 36 not at my error limit value
what do your think?