Page 1 of 1

Problem with very simple neural network in PHP

PostPosted: Mon Aug 26, 2013 4:44 pm
by mat345
Hi!

I wanted to play with neural networks and had the idea to implement a very simple one in php. Although I had no experience with this subject, i wanted to manage to run it.

I want to create the following neural net:

One Input with the values of 1 or 2 , one hiddenlayer with 2 neurons, one output. If the input=1, output=0; if input=2, output=1.

- ##############--Hidden 1 ---->
--> input (1, 2) --> ###########----> Output (0, 1)
###############-- Hidden 2 ------>

The Problem I have is that my program seems to converge the output to 0 or 1 for either input. I feel that I have something wrong in the calculation of the needed change-difference of the weights and biases.

Here is my code (the weights and biases are stored in an sql-database. It also gets its training data from there):

Code: Select all
$learningrate=0.2;

if(isset($_POST["submitTRAIN"]))
{
   $result=mysql_query("SELECT * FROM traindata") or die (mysql_error());
   while($row=mysql_fetch_assoc($result))
   {
      $result2=mysql_query("SELECT * FROM network232 WHERE ID='1'") or die (mysql_error());
      $row2=mysql_fetch_assoc($result2) or die (mysql_error());
                $i1=$row["i1"]; // The input from the training dataset
       $o1=$row["o1"]; //The desired output. If the input is 1, output should be 0. if input is 2, output should be 1.
      $w1=$row2["w1"]; // weight 1 (input to hidden1)
      $w2=$row2["w2"]; // weight 2 (input to hidden2)
      $w3=$row2["w3"]; // weight 3 (hidden1 to output )
      $w4=$row2["w4"]; // weight 4 (hidden2 to output)
                $b1=$row2["b1"]; // bias hidden1
      $b2=$row2["b2"]; // bias hidden2
      $b3=$row2["b3"];// bias output
               
               
      $oh1= 1 / (1 + exp((-1)*($w1*$i1+$b1))); // output of hidden1
      $oh2= 1 / (1 + exp((-1)*($w2*$i1+$b2))); // output of hidden2
      $oo1= 1 / (1 + exp((-1)*($w3*$oh1+$w4*$oh2+$b3))); // final output of output

      $do1=($o1-$oo1)*$oo1*(1-$oo1); // delta of output
      $dh1=$oh1*(1-$oh1)*($w3*$do1); // delta of hidden1
      $dh2=$oh2*(1-$oh2)*($w4*$do1); // delta of hidden2
               
                $addw1=(-1)*$learningrate*$dh1*$i1; // whats needed to be added to weight 1
        $addw2=(-1)*$learningrate*$dh2*$i1;
       $addw3=(-1)*$learningrate*$do1*$oh1;
       $addw4=(-1)*$learningrate*$do1*$oh2;
      
      $addb1=(-1)*$learningrate*$dh1; // whats needed to be added to bias 1
       $addb2=(-1)*$learningrate*$dh2;
      $addb3=(-1)*$learningrate*$do1;
      
      
      $neuw1=$w1+$addw1; // new weight 1
       $neuw2=$w2+$addw2;
       $neuw3=$w3+$addw3;
       $neuw4=$w4+$addw4;
      
      $neub1=$b1+$addb1; // new bias 1
      $neub2=$b2+$addb2;
      $neub3=$b3+$addb3;

                 echo $o1 . "  " . $oo1 . " <br>";
      
      $result3=mysql_query("UPDATE network232 SET w1='$neuw1', w2='$neuw2', w3='$neuw3', w4='$neuw4', b1='$neub1', b2='$neub2', b3='$neub3'") or die (mysql_error());
      
   }
}



This code converges all results towards 1. If i delete all minus-signs from the following example:

turn $addw1=(-1)*$learningrate*$dh1*$i1; into $addw1=*$learningrate*$dh1*$i1;

... then it will converge to 0 for all inputs.


I know that I am very new to this field and it is most likely a little stupid error I am making. it would be very helpful if you could find it for me, because i've been trying very hard for the last couple of days.

Thank you!