AForge.NET

  :: AForge.NET Framework :: Articles :: Forums ::

Question about classification with NN

The forum is to discuss topics from different artificial intelligence areas, like neural networks, genetic algorithms, machine learning, etc.

Question about classification with NN

Postby toncho11 » Thu Jun 30, 2011 8:01 am

I am using AForge.Neuro.Learning:

I have NN with "SigmoidFunction" activation function and "BackPropagationLearning" teacher. I have n input neurons and 1 output neuron.
I have two classes: 0.3 and 0.7.

I am confused what to do when I do classification and I get a value of the output neuron such 0.54343 for example.
To which class does this feature vector belongs to (0.3 or 0.7)?

1. By absolute distance it is closer to 0.7, so it is class 0.7?

2. Should I use network with output neurons equal to the number of classes and set it like that "1 0 0 0" where 1 is the right class.

3. Maybe I should use "Soft-Max" activation function instead of sigmoid function?

Thanks,
Anton
toncho11
 
Posts: 2
Joined: Thu Jun 30, 2011 7:43 am

Re: Question about classification with NN

Postby andrew.kirillov » Thu Jun 30, 2011 8:19 am

Hello,

toncho11 wrote:To which class does this feature vector belongs to (0.3 or 0.7)?

Are you sure it belongs to any ? Maybe the vector has features from both classes, so it is a mixture?

toncho11 wrote:2. Should I use network with output neurons equal to the number of classes and set it like that "1 0 0 0" where 1 is the right class.

It works for some applications. For example, the network for character recognition will have this type of structure, i.e. 26 output neurons (for 26 A-Z letters). Single output neuron works fine only for 2 classes, where you set value 1 for one class and value 0 for another, for example. But setting 0.3 and 0.7 is something strange - where these values came from?

What about training you did? Did error level decrease to any good value?
With best regards,
Andrew


Interested in supporting AForge.NET Framework?
User avatar
andrew.kirillov
Site Admin, AForge.NET Developer
 
Posts: 3443
Joined: Fri Jan 23, 2009 9:12 am
Location: UK

Re: Question about classification with NN

Postby toncho11 » Thu Jun 30, 2011 9:38 am

Thank you for your quick response!

Some background is needed. I am processing analog -> digital signal. This signal is turned into 100ms 11 dimensions feature vectors. I know what is the class when I record the signal. I record 4 classes, do the training - NN computation. Then when I get new signal I would like to check to which class it currently belongs.

I have decided that if Sigmoid functions support only values in the range [0..1] then the easiest way to go will be to divide my classes by 10 and from 1,2,3,4,5 to produce 0.1,0.2,0.3,0.4,0.5. I have trained each input vector against 0.1,0.2 etc.

I want the NN to decide which is the closest class. Isn't this the idea of NN - to statistically choose the best class?

So it seems 'output neurons = number of classes' is better.
toncho11
 
Posts: 2
Joined: Thu Jun 30, 2011 7:43 am

Re: Question about classification with NN

Postby andrew.kirillov » Thu Jun 30, 2011 9:50 am

toncho11 wrote:I have decided that if Sigmoid functions support only values in the range [0..1] then the easiest way to go will be to divide my classes by 10 and from 1,2,3,4,5 to produce 0.1,0.2,0.3,0.4,0.5.

I think you will not get any good result with such approach. It is OK to encode 2 classes with single neuron, but not more.

toncho11 wrote:So it seems 'output neurons = number of classes' is better.

Yes, definitely the way to go.
With best regards,
Andrew


Interested in supporting AForge.NET Framework?
User avatar
andrew.kirillov
Site Admin, AForge.NET Developer
 
Posts: 3443
Joined: Fri Jan 23, 2009 9:12 am
Location: UK

Re: Question about classification with NN

Postby zameer » Thu Jul 07, 2011 9:54 am

Hello

I have NN with "SigmoidFunction" activation function and "BackPropagationLearning" teacher. I have n input neurons and 1 output neuron.
I have two classes: 0.3 and 0.7.


Since you have two classes you can have Binary Activation Functions. (e.g. Unipolar Binary / Bipolar Binary) As a result you will have only two states in the neural network output. But I think Binary activation functions are still not supported by AForge.Net, if I'm not wrong.

There for you can have your own Binary activation function class written. Hope this might be a good sugestion for future addition for the Framework .

Thank You.
zameer
 
Posts: 6
Joined: Sat Aug 01, 2009 11:08 am
Location: Sri Lanka

Re: Question about classification with NN

Postby andrew.kirillov » Thu Jul 07, 2011 10:00 am

zameer wrote:But I think Binary activation functions are still not supported by AForge.Net, if I'm not wrong.

Check this: ThresholdFunction
With best regards,
Andrew


Interested in supporting AForge.NET Framework?
User avatar
andrew.kirillov
Site Admin, AForge.NET Developer
 
Posts: 3443
Joined: Fri Jan 23, 2009 9:12 am
Location: UK




Return to Artificial Intelligence