Page 1 of 1

How can I implement IActivationFunction?

PostPosted: Wed Dec 14, 2016 10:56 pm
by edurazee
Suppose, Image

Network structure: 2 input neurons: x and 1 (bias), one hidden layer of m neurons, one output neuron.

Activation function of hidden neurons = custom function

Code: Select all
public double Function(double h)
        {
            double result = 0;

            result = (Math.Exp(h) - Math.Exp(-h)) / (Math.Exp(h) + Math.Exp(-h));

            return result;
        }


How can we achieve that?

Activation function of output neuron = linear.

Re: How can I implement IActivationFunction?

PostPosted: Thu Dec 15, 2016 8:03 am
by andrew.kirillov
Have a look at the code of BipolarSigmoidFunction. It will give an idea how to implement IActivationFunction interface.

Re: How can I implement IActivationFunction?

PostPosted: Thu Dec 15, 2016 12:55 pm
by edurazee
I need to create a Perceptron with 2 input neurons: x and 1 (bias), one hidden layer of m neurons, one output neuron.

Activation function of hidden neurons = custom function

Activation function of output neurons = Linear.

I have written my custom function. But, how can I make sure that the activation function of the output neurons is Linear?



My Custom Function Code.

Code: Select all
public class CustomActivationFunction : IActivationFunction
    {
        public CustomActivationFunction()
       {

       }

        public double Derivative(double h)
        {
            return (4* Math.Exp(2*h))/Math.Pow(Math.Exp(2 * h)+1, 2);
        }

        public double Derivative2(double h)
        {
            double up = 8 * Math.Exp(2 * h) * (Math.Exp(2 * h) -1);
            double down = Math.Pow(Math.Exp(2 * h) + 1, 3);

            return up / down;
        }

        public double Function(double h)
        {
            double result = 0;

            result = (Math.Exp(h) - Math.Exp(-h)) / (Math.Exp(h) + Math.Exp(-h));

            return result;
        }
    }

Re: How can I implement IActivationFunction?

PostPosted: Thu Dec 15, 2016 2:38 pm
by andrew.kirillov
edurazee wrote:But, how can I make sure that the activation function of the output neurons is Linear?

Create another custom function, which is linear. The set it to the neuron you need.