AForge.NET

  :: AForge.NET Framework :: Articles :: Forums ::

Output only works with specific amount of neurons

The forum is to discuss topics from different artificial intelligence areas, like neural networks, genetic algorithms, machine learning, etc.

Output only works with specific amount of neurons

Postby SiriusA » Fri Mar 05, 2010 4:16 pm

I have 38 inputs and 3 outputs, the network is activation and backpropergation learning and there are around 300k samples.

Code: Select all
            // create multi-layer neural network
            network = new ActivationNetwork(
                new SigmoidFunction(sigmoidAlphaValue),
                numberOfInputs, neuronsInFirstLayer, 3);

            // create teacher
            teacher = new BackPropagationLearning(network);


In the learning process only one of the 3 outputs is true, so it should conform to a probability triple ex. {0.2, 0.5, 0.3} whenever i try to predict. However the output of the third/last output only works when the neurons are at either 5 or 15 (i tried going up to 20, but still only 5 and 15 worked), im not sure how this can be since as far as i have been able to read the number of neurons should not stop one of the output variables from working?


EDIT:
I have other datasets to test on, so i tried a different one and here it was 4 that worked and 5 zeroed the last output. If i however tried with 6 neurons then both 1 and 3 output was zero and only 2 showed any data ^^.
SiriusA
 
Posts: 5
Joined: Fri Dec 25, 2009 1:49 pm

Re: Output only works with specific amount of neurons

Postby andrew.kirillov » Sat Mar 06, 2010 2:34 pm

Well, I can tell you that I was creating bigger networks with the library and they worked fine.

However, it is really hard to say anything about the issue you have, since you did not mention anything about input patterns and desired output (as well as their range).

SiriusA wrote:In the learning process only one of the 3 outputs is true, so it should conform to a probability triple ex. {0.2, 0.5, 0.3} whenever i try to predict.

Where did this come from? What did you mean saying this?
With best regards,
Andrew


Interested in supporting AForge.NET Framework?
User avatar
andrew.kirillov
Site Admin, AForge.NET Developer
 
Posts: 3437
Joined: Fri Jan 23, 2009 9:12 am
Location: UK

Re: Output only works with specific amount of neurons

Postby SiriusA » Sat Mar 06, 2010 3:46 pm

There are 38 input variables. The input values are either boolean (0 or 1) or double variables normalized to between 0 and 1.
I have 1 hidden layer as shown where i can specify the amount of neurons.
There are 3 outputs.

When i train the network each set of inputs have 1 true answer in either of the 3 outputs [0 0 1] or [1 0 0] or [0 1 0]

Maybe i have missed something, but i was under the impression that the neural network would when trained like this, provide me with a probability triple, giving a percentage for the probability of either of the 3 outputs. So as per my example {0.2, 0.5, 0.3} it could also be {0.1, 0.8, 0.1} or any combination inbetween.

EDIT:

I should probably add that i use a confusion matrix to verify how many predictions are correct. So it is not that for instance O3 have zero in its value, it is just that the neural network never predicts O3 to have the highest probability compared to O1 and O2. Thus an ouput might be for instance:
O1: 0.012418404390354645
O2: 0.71712699536234259
O3: 0.31323670996054648

Where it should have put O3 as the highest. Now obviously it will not get this correct all the time, but the prediction for O3 is 0%, that is until i change the neurons in the layer and hit that magical number that works for this particually amount of traning data.

Am i doing something wrong or is it supposed to work like this?
SiriusA
 
Posts: 5
Joined: Fri Dec 25, 2009 1:49 pm

Re: Output only works with specific amount of neurons

Postby SiriusA » Sat Mar 06, 2010 5:40 pm

Well it seems i figured out my newbie mistake..

I was assuming that i did not need to run many iterations simply because my dataset was so big, this was of course not true and once i let the training run for a longer period of time it started working properly!
SiriusA
 
Posts: 5
Joined: Fri Dec 25, 2009 1:49 pm

Re: Output only works with specific amount of neurons

Postby andrew.kirillov » Sun Mar 07, 2010 11:06 am

:lol: Yep, quite common mistake for somebody, who just started with ANNs
With best regards,
Andrew


Interested in supporting AForge.NET Framework?
User avatar
andrew.kirillov
Site Admin, AForge.NET Developer
 
Posts: 3437
Joined: Fri Jan 23, 2009 9:12 am
Location: UK




Return to Artificial Intelligence

cron