AForge.NET

  :: AForge.NET Framework :: Articles :: Forums ::

Self learning robot

The forum is to discuss topics from different artificial intelligence areas, like neural networks, genetic algorithms, machine learning, etc.

Self learning robot

Postby supagu » Mon Dec 06, 2010 11:09 pm

So I've been working on a lego robot that takes a left distance reading and a right distance reading (using ultrasound) feeds these values to a neural network and uses two output values to drive the left and right engine.

The problem I see with this is that the inputs arent like an XOR type problem where they are simply on or off, the same with the output values. Eventually I want to expand the neural network to given the left and right distances determine how long each engine should run for before taking more ultra sound readings.

I'm wondering if a distance network as opposed to an activation network will work better. Or if i should break up my input from being 2 doubles to say, cast this to an int, convert each bit of the int to a neuron such that the network (same with the output) such that the network can identify patterns better?

Or some other approach?

Currently for learning i'm trying some genetic algorithms to calculate a fitness of a network, then breed/mutate them.

Ideas welcome :)
supagu
 
Posts: 6
Joined: Mon Dec 06, 2010 10:53 pm

Re: Self learning robot

Postby andrew.kirillov » Tue Dec 07, 2010 1:25 pm

Hello,

I think you need to decide about how the robot will learn or will be trained first.

The activation network is used for supervised learning - i.e. you train the network. You provide sample inputs and desired outputs to the network and it adopts itself to provide such outputs for the given inputs.

Distance network, which is Kohonen Self Organizing map for example, is used for unsurprised learning. Here you don't provide any desired outputs to the network. You provide only some inputs and network organizes itself to find some relation in the given data - clustering.

There is another approach, which you may consider - Reinforcement Learning. A brief example is an agent operating in some environment trying to achieve some goal (or a robot navigating somewhere searching for something). Here you provide a reward for every agent's action. If agent did something bad (bumped into a wall), then you provide negative reward, like -1. If did something very bad, you may give -10 as reward, so it knows it was much worse than bumping into a well. If it does neutral action, you may provide 0 as reward. If it does something good, like found the goal, you provide positive reward: 10 for example. Using these rewards a reinforcement learning algorithm learns which actions (and in which situations) are good or bad. After learning is done, the agent uses the knowledge to find its way to target.
With best regards,
Andrew


Interested in supporting AForge.NET Framework?
User avatar
andrew.kirillov
Site Admin, AForge.NET Developer
 
Posts: 3443
Joined: Fri Jan 23, 2009 9:12 am
Location: UK

Re: Self learning robot

Postby supagu » Wed Dec 08, 2010 9:34 am

Yeah im currently using reinforced learning, I have manged to get a neural network with 2 input nodes and 2 output nodes to learn sufficiently to navigate a maze trying to stay away from the walls.
I"m trying to up the neuron count trying to get more elaborate behaviour. Im trying a middle layer with 3 neruons.

What im doing to apply the fitness is a genetic algorithm, i get the fitest brains, i'll spawn some children by picking random neruons one parent and copying to the child. I also do some where i copy the parent and apply mutations (ie. i apply some randomization to nodes thresholds/weights).

the problem is learning takes a long time the more complex the brain becomes it seems.
Are there other methods to do reinforced learning than learning via genetic algorithm?

Also, im interested in looking at self organising map, but i dont get if this would work for me, sure i get how similar nerons will cluster together, but how do i get outputs from this and how do the outputs relate to how i drive the motors of my robot?
supagu
 
Posts: 6
Joined: Mon Dec 06, 2010 10:53 pm

Re: Self learning robot

Postby andrew.kirillov » Wed Dec 08, 2010 10:09 am

supagu wrote:Are there other methods to do reinforced learning than learning via genetic algorithm?

Well, genetic algorithms are not used in reinforcement learning usually. There are other algorithm for it, like Q-Learning, SARSA, etc.

You may start from here: Reinforcement learning Wiki. Also this book was nice to read: Reinforcement Learning: An Introduction
With best regards,
Andrew


Interested in supporting AForge.NET Framework?
User avatar
andrew.kirillov
Site Admin, AForge.NET Developer
 
Posts: 3443
Joined: Fri Jan 23, 2009 9:12 am
Location: UK

Re: Self learning robot

Postby supagu » Sun Dec 12, 2010 9:56 am

I've managed to get some good results at least in simulations. I've put up my work on progress logs of my experiments if any one is interested.
If any one has any further ideas on things to try let me know :)
https://docs.google.com/viewer?a=v&pid=explorer&chrome=true&srcid=0B_nQZvJoqbFmNThlOTQ2YTItZjliOS00NGVjLWE0OGEtNTA5N2FiOTRkNzMy&hl=en&authkey=CK6N9r8N
supagu
 
Posts: 6
Joined: Mon Dec 06, 2010 10:53 pm

Re: Self learning robot

Postby zameer » Wed May 25, 2011 5:52 pm

Hi,

What about giving a try on fuzzy logic. Taking sensor inputs as fuzzy inputs, process them [define a fuzzy controller (software/hardware based)], and feed the out put values of the fuzzy controller to the left and right engines.

Refer eg:-Fuzzy logic Anti-Sway Controller for Container Crane Control

zameer
zameer
 
Posts: 6
Joined: Sat Aug 01, 2009 11:08 am
Location: Sri Lanka




Return to Artificial Intelligence