I'm going to develop an application to recognize handwritten digits. Here I am using the MNIST dataset (http://cis.jhu.edu/~sachin/digit/digit.html). My approach to train the network is like this.
1) create a List<List<Bitmap>> from the dataset
2) convert them to double[] arrays
3) make each of the above array as input( double [][])
4) create corresponding output array ( double [][])
5) Use back propagation for training.
Note : Each bitmap is size of 28*28 pixels and corresponding double array contains 784 elements.
I'm training the digits from 0-9
Therefore each input array consist of 784 elements and each output array consist of 10 elements
Using above information I created the following method to train the network
- Code: Select all
/// <summary>
/// Train using Back Propogation
/// </summary>
/// <param name="inputData">The input data.</param>
/// <param name="outputData">The output data.</param>
/// <param name="iterations">The iterations.</param>
/// <param name="neurons">The neurons.</param>
/// <returns></returns>
public string[] BackPropogationTrain(double[][] inputData, double[][] outputData, int iterations)
{
System.Diagnostics.Debug.WriteLine("Training Started...");
bool needStopping = false;
int iterateCount = 0;
double error = 0;
string[] results = new string[2];
// initialize input and output values
double[][] input = inputData;
double[][] output = outputData;
network = new AForge.Neuro.ActivationNetwork(new AForge.Neuro.BipolarSigmoidFunction(2),
784, // 784 inputs (coz each array corresponding to an image consists of 784 elements )
784, //784 neurons in the first layer (corresponding to input)
10); //10 neurons in the second layer (corresponding to 10 digits )
network.Randomize();
bpteacher = new AForge.Neuro.Learning.BackPropagationLearning(network);
bpteacher.LearningRate = 1;
while (!needStopping)
{
error = bpteacher.RunEpoch(input, output);
if (error == 0) //If the error rate is 0
{
break;
}
else if (Math.Round(error, 2) == 0) // If the error rate is 0 to the second decimal point
{
break;
}
else if (iterateCount < iterations) //If the given iteraions are completed
{
iterateCount++;
}
else
{
needStopping = true;
}
iterateCount++;
System.Diagnostics.Debug.WriteLine("Iteration :\t" + iterateCount + " \tError Rate :\t" + error);
}
System.Diagnostics.Debug.WriteLine("Error Rate : " + error);
results[0] = error.ToString();
System.Diagnostics.Debug.WriteLine("Iterations : " + iterateCount);
results[1] = iterateCount.ToString();
System.Diagnostics.Debug.WriteLine("Training Completed...");
return results;
}
My problem is this.
* Though I train with a large amout of data, at each iteration the error rate doesn't decrease with a considerable amount. Even for 3 images for each digit, training doesn't complete for hours. I want to know if I'm doing the back propagation in a wrong manner. And whether there are any issues with my training method.
I tried to explain the question at my best, sorry for the lengthy question.
Any help would be appreciated.
Thank you very much.