How can I implement IActivationFunction?

[edurazee]

Suppose,

Network structure: 2 input neurons: x and 1 (bias), one hidden layer of m neurons, one output neuron.

Activation function of hidden neurons = custom function

Code: Select all

public double Function(double h)
        {
            double result = 0;

            result = (Math.Exp(h) - Math.Exp(-h)) / (Math.Exp(h) + Math.Exp(-h));

            return result;
        }

How can we achieve that?

Activation function of output neuron = linear.

[andrew.kirillov]

Have a look at the code of BipolarSigmoidFunction. It will give an idea how to implement IActivationFunction interface.

[edurazee]

I need to create a Perceptron with 2 input neurons: x and 1 (bias), one hidden layer of m neurons, one output neuron.

Activation function of hidden neurons = custom function

Activation function of output neurons = Linear.

I have written my custom function. But, how can I make sure that the activation function of the output neurons is Linear?



My Custom Function Code.

Code: Select all

public class CustomActivationFunction : IActivationFunction
    {
        public CustomActivationFunction()
       {

       }

        public double Derivative(double h)
        {
            return (4* Math.Exp(2*h))/Math.Pow(Math.Exp(2 * h)+1, 2);
        }

        public double Derivative2(double h)
        {
            double up = 8 * Math.Exp(2 * h) * (Math.Exp(2 * h) -1);
            double down = Math.Pow(Math.Exp(2 * h) + 1, 3);

            return up / down;
        }

        public double Function(double h)
        {
            double result = 0;

            result = (Math.Exp(h) - Math.Exp(-h)) / (Math.Exp(h) + Math.Exp(-h));

            return result;
        }
    }

[andrew.kirillov]

But, how can I make sure that the activation function of the output neurons is Linear?

Create another custom function, which is linear. The set it to the neuron you need.