This forum uses cookies
This forum makes use of cookies to store your login information if you are registered, and your last visit if you are not. Cookies are small text documents stored on your computer; the cookies set by this forum can only be used on this website and pose no security risk. Cookies on this forum also track the specific topics you have read and when you last read them. Please confirm whether you accept or reject these cookies being set.

A cookie will be stored in your browser regardless of choice to prevent you being asked this question again. You will be able to change your cookie settings at any time using the link in the footer.

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
automatically selecting autopilot gains
#11
Is there some way we can detect the current conditions from wind sensor and inertal sensors and simplify it to a 2 or 3 dimensional space for "current conditions" ?

From here, you could tune the gains and once they are favorable press a button "save" to record these settings.

Once enough settings are recorded in enough sea states/conditions, we can interpolate the gains all the time as conditions change. If the user overrides the interpolation with either manual tuning, or some sort of autotune algorithm, this would allow a new update to be recorded to improve the settings.


This is somewhat tedious interaction and probably would take a while and different conditions until enough is know, but I think it could be interesting. Once we can tune a few boats this way and compare their settings, then a generic default settings, and try to reduce the number of changing parameters, or map the gains to a different state space with less dimensions. This would give a head start and make tuning future boats much faster, but it might even help to enter the boat type or characteristics as well (once we have lots of data)


I like this approach since it uses little processing power and is easy to understand, but I still plan on developing other pilots such as neural network approach because they have potential to do better than human steering which is not really possible with PID (unless human is tired)
Reply
#12
In fact, the lower left button 'read fom systemk' already takes the current pid settings and stores them in the current cell of the 'gainset matrix', so to speak. One could add a checkbox and a timer to do that periodically. So on openplotter, this facility chooses a preset, sends it to the pypilot; if you make it change on your pypilot, it is remembered for the current circumstance.

Parameters are expected to be not only dependent on the boat, but also on the actuator dimensions, but some persistent thread on this forum on what parameters people use could be insightful.

As for the number of input parameters, I'd stick to 2 for now until I got my setup actually operational. It's also to be seen if the other four gains would need to be part of it.
Reply
#13
From experience I think you should instead adjust P, D, and DD rather than I. Another idea is to use an overall gain and the value that equals D/P. Capturing the gains this way gives better extrapolated values for gains in new conditions, and allows transferring the table to other similar vessels better.

Eventually DD/D might be considered as well.

The limitation is still the control loop which is not optimal no matter how it is tuned since steering in waves with sail power is a non-linear problem and the basic autopilot is simple.

I think you will find a wide range of P, D values which are acceptable making it sometimes difficult to find the right gain. They can widely vary with similar steering in some cases, so by the time you determine which is best, the conditions are already changing.
Reply
#14
Good hints these are, thx; I'll add DD then.

One of my ideas with this tool is to automatically improve the gains on a continuous basis. Define a performance indicator, for instance, the ap.heading_error as you already use in autogain.py (the trick is making it a rolling average). Then, at certain intervals, take the current gainset and adjust some of them by a harmlessly small, but random factor. Measure performance for a certain time, and if performance improved, switch to the new gainset. If not, stick to the old one and repeat. This all happens within the context of one gainset. This way, the system learns by itself.

While we are brainstorming, let's touch the subject of non-linearity. To be honest, before I started looking into autopilots I thought it was more advanced than PID contol. I'm talking in general, not only pypilot. There are urban myths going around about autopilots that are in tune with the periodics of the waves and the boat, and I believed them. But reading into it, it indeed seems to be all about these PID gains. Are you aware of any wave modeling going on? If you'd have the imu.heave, you might be able to lock a phase locked loop (PLL) system to it. With some certainty this could predict an incoming wave in some well-behaved sea, if such a thing exists ;-) Would not know what to do with it yet, especially given its stochastic nature, but I do find it an interesting thought. When you're hand-steering, you anticipate waves in some way, and that is feed-forward. You can already do something with the imu.pitch I guess https://liquidsdr.org/blog/pll-simple-howto/.
Reply
#15
(2019-07-16, 07:31 PM)ironman Wrote: Good hints these are, thx; I'll add DD then.

One of my ideas with this tool is to automatically improve the gains on a continuous basis. Define a performance indicator, for instance, the ap.heading_error as you already use in autogain.py (the trick is making it a rolling average). Then, at certain intervals, take the current gainset and adjust some of them by a harmlessly small, but random factor. Measure performance for a certain time, and if performance improved, switch to the new gainset. If not, stick to the old one and repeat. This all happens within the context of one gainset. This way, the system learns by itself.

Modifying by a harmlessly small amount may improve the gains, but at a learing rate slower than changing conditions.         I don't think a continuous  improvement will work all that well.   Consider sudden wind increases.  If it can use past data, the gains can be radically changed  right away, so this is what is needed.

Quote:While we are brainstorming, let's touch the subject of non-linearity. To be honest, before I started looking into autopilots I thought it was more advanced than PID contol. I'm talking in general, not only pypilot. There are urban myths going around about autopilots that are in tune with the periodics of the waves and the boat, and I

It is a little more advanced..   There are a few non-linear gains.  I have experimented with about 12  gains in the ocean and only the 6 supported seem to have a useful effect so far.   There is so much more possible though.

Quote:believed them. But reading into it, it indeed seems to be all about these PID gains. Are you aware of any wave modeling going on? If you'd have the imu.heave, you might be able to lock a phase locked loop (PLL) system to

Theh imu.heave is basically available as imu.accel, but you might have to apply the fusionQpose quaternion to this vector to get heave and sway.

The frequency is going to be the same for all the inertial measurements.   The scope even can plot the fourier transform of any sensor.    At sea, there are also sometimes 2 distinct frequencies/periods of motion.
Quote:it. With some certainty this could predict an incoming wave in some well-behaved sea, if such a thing exists ;-)

It depends on the type of wave.   I know from passing powerboats that first  you get longer waves with less amplitude before shorter steep waves, so maybe if you know the type of wave you can do something but it's very difficult to predict.    The other approach is to use a camera so the autopilot can see the waves.

Even as a human with eyes, I sometimes duck large waves that don't make a splash, and get soaked from seemingly small ones.

Quote:Would not know what to do with it yet, especially given its stochastic nature, but I do find it an interesting thought. When you're hand-steering, you anticipate waves in some way, and that is feed-forward. You can already do something with the imu.pitch I guess https://liquidsdr.org/blog/pll-simple-howto/.

Yes.   The basic idea that isn't implemented/proven yet is to add a new gain for wave motion.

This gain would be for a value calculated from the inertial sensors.   For example take yaw rate and find the frequency and phase from the last two wave cycles.    Lowpass these values and output a sine wave with this frequency which is phase shifted an adjustable amount from the yaw rate.  

This might work if the waves are predictable but has the potential to work against you and do the wrong thing if a strange or new wave pattern is encountered.
Reply
#16
I agree with most of your remarks, although I do think that the frequency of the boat itself is by definition different than the frequency of the waves. It will be tricky to separate them if they are close. I think when you saw two frequencies in your fourier analysis, this must be it - one of the boat and one of the sea.

In fact, some seas have two frequencies - the short wave and the long swell. They say on the North Sea, every seventh wave is a big one, where the wave coincides with the swell. Haven't experienced it yet, but when I get my setup working I'll run your fourier tool. Where is that tool, btw - haven't found it in the plugin and now left the openplotter on the boat.

The quaternation is one bridge too far for me yet, but if the result is the pure vertical component, it must be the wave, given the sensor is near the boat's centre of gravity *). If one can plot that curve, and make some footage of the plot along with the steering corrections of a good helmsman (which I am not), you can work towards phase-dependent steering corrections. I expect to see that on a wavetop a boat has a tendency to turn into the wind, and a helmsman steers against that.

*) if the wave go up and your boat goes down, it's time to disengage your autopilot
Reply
#17
I've done a few few trips now on the PyPilot and I've collected some sets of gains. As suggested by Sean it seems as if D/P is constant; in fact that is my guide when adjusting the gains. However I need a calculator for this and still make typos with the number of zeros. I've trawled through other autopilot's manuals to see what user parameters are available in their settings. Based on this, I'm planning to change the described gainset prototype to become a bit more intuitive and usable front-end, given the cockpit load under which you typically have to operate it.

First, I want to select which of the P-parameters I will be using: P or PR, and the D-parameter: D or DD. Then, I want to give upper and lower limits for the P-parameter, and a factor for the D/P constant. Finally, I want to introduce a 'speed factor', which multiplies my boat speed to set the P-parameter within the given range. Then the boat speed dictates the P and D-factor, which I write back to pypilot's signalk database. Let's rename the D/P constant to a simple name, say, 'counter rudder'. I think speed factor and counter rudder are fixed to my rudder effectiveness resp. boat inertia and they should not change.

Then, when the circumstances change, there's one more overall gain that can be manually set, and that multiplies the chosen P and D parameter. Let's call this gain 'responsiveness'. At lowest value, 1, this signifies optimal use of energy; at higher values this makes the autopilot more assertive. Initially, I'd make a slider for it, or +- buttons, and set it manually.

I think eventually overall gain should in some way follow either the sea state (amplitude of imu.heave), the steadyness of the true wind (standard deviation of $WIMWV), or the wind angle/point of sail (average of $WIMWV). Which one is yet to be determined. A bit more reactive, it can also increase as the average error heading goes up.

Before I begin, any thoughts would be apprciated.

(2019-07-15, 02:57 PM)seandepagnier Wrote: [...], but I still plan on developing other pilots such as neural network approach because they have potential to do better than human steering which is not really possible with PID (unless human is tired)

Old article, but might still provide some handles:

https://pearl.plymouth.ac.uk/handle/10026.1/1100
Reply
#18
I come from the MATLAB world so I can't suggest an opensource tool, though I know some good ones exist, and don't currently have the time or the data to train a machine learning autopilot. However this is an awesome project and I have some experience with machine learning, I think a good approach would be to do it in a few steps.

1) Train a deep neural net to predict boat behaviour.

-use the last x seconds of data available to the autopilot as input
     -more time risks overfitting and will require more data, less time risks being too shortsighted
-train it to predict the next set of sensor inputs to the autopilot.

This will require a lot of data and a lot of compute time to generate but running the generated neural net is fast. Doing this first lets you guarantee you have enough data for machine learning and lets you try different hidden layer architectures without messing up your boats steering.

2) Use the boat behaviour neural net to train one of the reinforcement learning agents that use a neural net as a function approximator rather than a state matrix (probably some variant of Q-learning, markov decision process or SARSA).

-train by choosing your x seconds initial state from your real data at random, then running the neural net from there with the reinforcement learning agent controlling the rudder inputs at each time step. With enough initial states/commanded VMG combinations the reinforcement learning agent will learn how to drive your neural net boat model. The neural net model lets you run this for as long as it takes to converge, this type of learning takes a even more data than the neural net from step one, so we're using that to kind of cheat.

-The reward function for the agent would be some weighted combination of maximizing commanded VMG, minimizing rudder inputs, and minimizing course deviations. It will probably work best with one of real or apparent wind commands,  it's probably best to use that mode for all piloting but convert GPS or the worse kind of wind direction into the better type of wind direction command to do GPS mode autopilot.

3) Once the learning agent learns to steer the neural net boat model it's time to move it onto the boat, where it will continue learning online and continuously get better.

4) If and when there is a working solution, repeating the steps with a larger data set that includes more boats and boat types might make it possible to create a general agent that can drive any boat without doing stupid things, which would allow people who aren't interested in machine learning to just load that general pre-trained agent and have it learn to sail their boat specifically out on the water with little input from them.
Reply
#19
(2020-01-05, 11:05 PM)ironman Wrote: I've done a few few trips now on the PyPilot and I've collected some sets of gains. As suggested by Sean it seems as if D/P is constant; in fact that is my guide when adjusting the gains. However I need a calculator for this and still make typos with the number of zeros. I've trawled through other autopilot's manuals to see what user parameters are available in their settings. Based on this, I'm planning to change the described gainset prototype to become a bit more intuitive and usable front-end, given the cockpit load under which you typically have to operate it.

First, I want to select which of the P-parameters I will be using: P or PR, and the D-parameter: D or DD. Then, I want to give upper and lower limits for the P-parameter, and a factor for the D/P constant. Finally, I want to introduce a 'speed factor', which multiplies my boat speed to set the P-parameter within the given range. Then the boat speed dictates the P and D-factor, which I write back to pypilot's signalk database. Let's rename the D/P constant to a simple name, say, 'counter rudder'. I think speed factor and counter rudder are fixed to my rudder effectiveness resp. boat inertia and they should not change.

Then, when the circumstances change, there's one more overall gain that can be manually set, and that multiplies the chosen P and D parameter. Let's call this gain 'responsiveness'. At lowest value, 1, this signifies optimal use of energy; at higher values this makes the autopilot more assertive. Initially, I'd make a slider for it, or +- buttons, and set it manually.

I think eventually overall gain should in some way follow either the sea state (amplitude of imu.heave), the steadyness of the true wind (standard deviation of $WIMWV), or the wind angle/point of sail (average of $WIMWV). Which one is yet to be determined. A bit more reactive, it can also increase as the average error heading goes up.

Before I begin, any thoughts would be apprciated.

(2019-07-15, 02:57 PM)seandepagnier Wrote: [...], but I still plan on developing other pilots such as neural network approach because they have potential to do better than human steering which is not really possible with PID (unless human is tired)

Old article, but might still provide some handles:

https://pearl.plymouth.ac.uk/handle/10026.1/1100

Marco (and Sean)
You have here many ideas, but is https://github.com/marcobergman/gainsets 14-07-2019 your last version? I have still problems when sailing (strong and soft wind). On the motor it is working better. Perhaps gainsets can help me to experiment with the right combination of settings. One of the reason can also be that my boat is heavy (3500kg) and working with a wheel drive autohelm 3000: the motor has to twist more and the response is slower. Do you have experience with that, or an example of the best settings for such a boat?
Another question: where can I find a clear explanation of all the different gains, so that I can understand better how I have to optimize that? Thanks.
Reply
#20
Yes I made that version even before I built my pypilot, and have not reallly pursued it ever since; in fact the script is broken ever since pypilot upgraded to 2.0, due to the different internal API of pypilot ('signalk' vs pypilotServer).

There is a reason for that I have not pursued it, because it works pretty well without - for me.

Once I found out the sweet spot for my boat's gains, I only change D and DD occasionally. I have a steel longkeel chine, 4500kg+. I'm sure you have seen my recent movie of the setup performing. 

Upload your own gains and performance movie to the general forum! There's many people here who have gone through the gain hell before emerging triumphantly from it.
Reply


Forum Jump:


Users browsing this thread: 2 Guest(s)