Using Back Propagation for Digit Recognition

Hi all,
I'm going to develop an application to recognize handwritten digits. Here I am using the MNIST dataset (http://cis.jhu.edu/~sachin/digit/digit.html). My approach to train the network is like this.
1) create a List<List<Bitmap>> from the dataset
2) convert them to double[] arrays
3) make each of the above array as input( double [][])
4) create corresponding output array ( double [][])
5) Use back propagation for training.
Note : Each bitmap is size of 28*28 pixels and corresponding double array contains 784 elements.
I'm training the digits from 0-9
Therefore each input array consist of 784 elements and each output array consist of 10 elements
Using above information I created the following method to train the network
My problem is this.
* Though I train with a large amout of data, at each iteration the error rate doesn't decrease with a considerable amount. Even for 3 images for each digit, training doesn't complete for hours. I want to know if I'm doing the back propagation in a wrong manner. And whether there are any issues with my training method.
I tried to explain the question at my best, sorry for the lengthy question.
Any help would be appreciated.
Thank you very much.
I'm going to develop an application to recognize handwritten digits. Here I am using the MNIST dataset (http://cis.jhu.edu/~sachin/digit/digit.html). My approach to train the network is like this.
1) create a List<List<Bitmap>> from the dataset
2) convert them to double[] arrays
3) make each of the above array as input( double [][])
4) create corresponding output array ( double [][])
5) Use back propagation for training.
Note : Each bitmap is size of 28*28 pixels and corresponding double array contains 784 elements.
I'm training the digits from 0-9
Therefore each input array consist of 784 elements and each output array consist of 10 elements
Using above information I created the following method to train the network
- Code: Select all
/// <summary>
/// Train using Back Propogation
/// </summary>
/// <param name="inputData">The input data.</param>
/// <param name="outputData">The output data.</param>
/// <param name="iterations">The iterations.</param>
/// <param name="neurons">The neurons.</param>
/// <returns></returns>
public string[] BackPropogationTrain(double[][] inputData, double[][] outputData, int iterations)
{
System.Diagnostics.Debug.WriteLine("Training Started...");
bool needStopping = false;
int iterateCount = 0;
double error = 0;
string[] results = new string[2];
// initialize input and output values
double[][] input = inputData;
double[][] output = outputData;
network = new AForge.Neuro.ActivationNetwork(new AForge.Neuro.BipolarSigmoidFunction(2),
784, // 784 inputs (coz each array corresponding to an image consists of 784 elements )
784, //784 neurons in the first layer (corresponding to input)
10); //10 neurons in the second layer (corresponding to 10 digits )
network.Randomize();
bpteacher = new AForge.Neuro.Learning.BackPropagationLearning(network);
bpteacher.LearningRate = 1;
while (!needStopping)
{
error = bpteacher.RunEpoch(input, output);
if (error == 0) //If the error rate is 0
{
break;
}
else if (Math.Round(error, 2) == 0) // If the error rate is 0 to the second decimal point
{
break;
}
else if (iterateCount < iterations) //If the given iteraions are completed
{
iterateCount++;
}
else
{
needStopping = true;
}
iterateCount++;
System.Diagnostics.Debug.WriteLine("Iteration :\t" + iterateCount + " \tError Rate :\t" + error);
}
System.Diagnostics.Debug.WriteLine("Error Rate : " + error);
results[0] = error.ToString();
System.Diagnostics.Debug.WriteLine("Iterations : " + iterateCount);
results[1] = iterateCount.ToString();
System.Diagnostics.Debug.WriteLine("Training Completed...");
return results;
}
My problem is this.
* Though I train with a large amout of data, at each iteration the error rate doesn't decrease with a considerable amount. Even for 3 images for each digit, training doesn't complete for hours. I want to know if I'm doing the back propagation in a wrong manner. And whether there are any issues with my training method.
I tried to explain the question at my best, sorry for the lengthy question.
Any help would be appreciated.
Thank you very much.