TensorFlow, AI, Autopilot, VendeeGlobe

[seandepagnier]

A lot to unpack, and sorry if I am brief

Inputs - If you read a little about AI, you'll read the term parameters.  This is just how many inputs a model is using to solve the problem at hand.  ChatGPT4 is said to have over a trillion parameters.  In our case, a parameter is simply our wind and boat speed, wind and boat direction, GPS data, wave data, … etc.  We wouldn't need near that many for our problem.  For a specific boat (our boat) we might get away with a few tens of parameters.  In a more general case of trying to apply to all of our boats, we would likely need hundreds. 

We have to input time-series data. So at 20 updates per second, with 9 inertial sensors, this is 180 inputs per second. If we were to feed the previous 10 seconds of intertial data this is 1800 inputs. There can likely be ways to optimize/decimate this. There are also other inputs like gps, wind, rudder, motor current and many others including potentially cameras (which can add orders of magnitude more input considering video of previous 10 seconds)

Outputs – This is what spits out of the Model.  In the case of ChatGPT4, this might be a sonnet or a picture or any number of other things.  For us, in the simplest case, is where to position the rudder.  A little more complex might be sheeting of the sails.  I think I read where Sean was considering adding control line adjustment to PyPilot.

If the output is rudder position, (or potentially simply motor movement) it has to be trained either from a human or PID filter and will never do better than this. Instead I intend to train it to predict the future inertial sensor inputs and from this, on the fly it can consider all sorts of potential future corrections and see what gives the best end result. This can potentially teach it to do things that are counter intuitive, and not require supervised training.

Two Phases of AI

Using the model is the easiest – Plug in all the input parameters, run the linear-algebra steps and out pops the answer.  For our case of say several tens of parameters and the output of only controlling the rudder, even the lowest Arduino could handle this problem at several hundred hertz.  IOW, an Arduino Nano could pilot the boat just fine once the model is defined.

As mentioned above, an arduino nano is not going to be able to do this anymore since we have thousands of parameters already in the simplest model and need to evaluate it lots of times (hundreds to thousands of times per second) A raspberry pi can do this though.

Learning is where the Model is taught and eventually defined.  Teaching a model takes more memory and more time.  Whereas an Arduino can run our simplistic model, it would take a little more power to teach the model.  But, even an ESP8266 or ESP32 could probably do it and certainly any RasPi.  The model described as a matrix has many values in it.  The number of value

My basic tests show the pi zero cannot do online training with few hundred parameters, at least not using tensorflow and python. A pi 4 or pi 5 however potentially can. With more inputs it becomes marginal, and video processing it definitely cannot.

Genetic Algorithms are used to handle cases where the answer isn’t decisively known.  I’ve been more interested in exploring this case.  It is called Genetic because is attempts to model the learning procedure similar to Darwin Evolution principles.  In its simplest form, a generation of Models are generated.  For instance 80 models with all the values in the model are randomly

My intention is to be able to log all the sensor data and volunteers can upload this to a server somewhere. From here, potentially genetic algorithms could be useful to determine the optimal network model to use. I dont see GA as being useful on the fly, but instead used to optimize the model to reduce overall computation, and consider also transfer-learning possibilities benefiting from collectivized data.

[*]Power Efficiency - Watching Sean’s videos of his PyPilot on a couple of totally different boats while they automatically change course at the waypoints and handle different sea states I am struck with how little the helm needs to move.  No matter what, an AI version will be far less energy efficient of electric power. 

It depends. It may use more power or less, it may steer better or worse. Already the extended pid filter does a decent job.

[*]Learning – It should be pretty easy to create a Back Substitution version and install it on your OpenMarine RasPi.  It would start with the random values in the model (Infant Brain).  It could be completely passive and safe.  You would have to add the hardware for rudder position feedback.  It could run in the background and absorb all the incoming sensor data and take your responses to controlling the helm and start learning and modifying the Model. 

For machine learning: it is not required. rudder feedback will be optional, the way wind speed is, as it can be helpful.

[*]Sea State – If you want to make it handle different sea states, it should be obvious that you need sensors that provide some kind of data that will change based on sea states.  This might be as simple barometric pressure sensor (~$1) to measure rise and fall of the boat.  It might be accelerometers and gyros to measure rocking and rolling of the vessel. 

This is where the model gets more complex. It needs to activate certain sections depending on sea state so that it can retain information from past knowledge (or even from other boat's shared data) and train those while suppressing training that is not relevant at that point in time.

[*]Simplicity – The beauty of AI is that no first principle type logic is necessary for someone to work out.  Just adding the sensors to the mix and adjusting the model to absorb their input will automatically start to learn about how you handle the boat in various sea states.

If done wrong it will take forever to learn and not work very well.

[*]Generalizing – I hope the above examples have given you some taste of what is involved.  I don’t believe the talk about making one common, centralized model available to everyone is feasible at any level.  Let me ask the real sailors here – Do you control your boat differently based on how it is loaded – Heavy, light?  How would you propose to tell the computer this state?  Do you believe that every 40’ monohull is the same?  To be able to quantify every parameter of boats so the learning can distinguish from one 40’ boat from another would be staggeringly complex.  Naval Architects have been trying to quantify boats since day-one. 

The model needs to be able to determine similarities with other vessels if it is to take advantage of transfer learning.

[*]Trust Issues - Would I ever trust it?  Probably only while I'm watching it.  I'm more likely to trust PyPilot.  The thing about AI you don't know what edge case might crop up that never cropped up before.  Would it spike the helm hard over because of some

I dont trust most people either, especially not even myself. For cruising the best defense may be to sail so the boat will not be in danger regardless of helm action, but this eliminates the possibility of high performance. For this reason true machine learning models are in my knowledge not used even on ocean racing boats. Instead they prefer more advanced reliable algorithms. A model could flip a multihull which is bad.

I'm curious if there is anyone else on the forum that has worked with AI algorithms and has a different opinion.  I believe AI can do some staggeringly complex things for good or ill.  It just depends on how much effort and computer power you're willing to throw at the problem.  If I could do it with a RasPi, I might experiement.  If it requires a AMD 7950X with a Nvidia 4090 using about a kilowatt of power... I think I'll pass.  

it could be worth the power to potentially train and/or understand the problem better. Further, in the future trillions of operations can be done on a few watts.