TensorFlow, AI, Autopilot, VendeeGlobe

[seandepagnier]

A lot to unpack, and sorry if I am brief

Inputs - If you read a little about AI, you'll read the term parameters.  This is just how many inputs a model is using to solve the problem at hand.  ChatGPT4 is said to have over a trillion parameters.  In our case, a parameter is simply our wind and boat speed, wind and boat direction, GPS data, wave data, … etc.  We wouldn't need near that many for our problem.  For a specific boat (our boat) we might get away with a few tens of parameters.  In a more general case of trying to apply to all of our boats, we would likely need hundreds. 

We have to input time-series data. So at 20 updates per second, with 9 inertial sensors, this is 180 inputs per second. If we were to feed the previous 10 seconds of intertial data this is 1800 inputs. There can likely be ways to optimize/decimate this. There are also other inputs like gps, wind, rudder, motor current and many others including potentially cameras (which can add orders of magnitude more input considering video of previous 10 seconds)

Outputs – This is what spits out of the Model.  In the case of ChatGPT4, this might be a sonnet or a picture or any number of other things.  For us, in the simplest case, is where to position the rudder.  A little more complex might be sheeting of the sails.  I think I read where Sean was considering adding control line adjustment to PyPilot.

If the output is rudder position, (or potentially simply motor movement) it has to be trained either from a human or PID filter and will never do better than this. Instead I intend to train it to predict the future inertial sensor inputs and from this, on the fly it can consider all sorts of potential future corrections and see what gives the best end result. This can potentially teach it to do things that are counter intuitive, and not require supervised training.

Two Phases of AI

Using the model is the easiest – Plug in all the input parameters, run the linear-algebra steps and out pops the answer.  For our case of say several tens of parameters and the output of only controlling the rudder, even the lowest Arduino could handle this problem at several hundred hertz.  IOW, an Arduino Nano could pilot the boat just fine once the model is defined.

As mentioned above, an arduino nano is not going to be able to do this anymore since we have thousands of parameters already in the simplest model and need to evaluate it lots of times (hundreds to thousands of times per second) A raspberry pi can do this though.

Learning is where the Model is taught and eventually defined.  Teaching a model takes more memory and more time.  Whereas an Arduino can run our simplistic model, it would take a little more power to teach the model.  But, even an ESP8266 or ESP32 could probably do it and certainly any RasPi.  The model described as a matrix has many values in it.  The number of value

My basic tests show the pi zero cannot do online training with few hundred parameters, at least not using tensorflow and python. A pi 4 or pi 5 however potentially can. With more inputs it becomes marginal, and video processing it definitely cannot.

Genetic Algorithms are used to handle cases where the answer isn’t decisively known.  I’ve been more interested in exploring this case.  It is called Genetic because is attempts to model the learning procedure similar to Darwin Evolution principles.  In its simplest form, a generation of Models are generated.  For instance 80 models with all the values in the model are randomly

My intention is to be able to log all the sensor data and volunteers can upload this to a server somewhere. From here, potentially genetic algorithms could be useful to determine the optimal network model to use. I dont see GA as being useful on the fly, but instead used to optimize the model to reduce overall computation, and consider also transfer-learning possibilities benefiting from collectivized data.

[*]Power Efficiency - Watching Sean’s videos of his PyPilot on a couple of totally different boats while they automatically change course at the waypoints and handle different sea states I am struck with how little the helm needs to move.  No matter what, an AI version will be far less energy efficient of electric power. 

It depends. It may use more power or less, it may steer better or worse. Already the extended pid filter does a decent job.

[*]Learning – It should be pretty easy to create a Back Substitution version and install it on your OpenMarine RasPi.  It would start with the random values in the model (Infant Brain).  It could be completely passive and safe.  You would have to add the hardware for rudder position feedback.  It could run in the background and absorb all the incoming sensor data and take your responses to controlling the helm and start learning and modifying the Model. 

For machine learning: it is not required. rudder feedback will be optional, the way wind speed is, as it can be helpful.

[*]Sea State – If you want to make it handle different sea states, it should be obvious that you need sensors that provide some kind of data that will change based on sea states.  This might be as simple barometric pressure sensor (~$1) to measure rise and fall of the boat.  It might be accelerometers and gyros to measure rocking and rolling of the vessel. 

This is where the model gets more complex. It needs to activate certain sections depending on sea state so that it can retain information from past knowledge (or even from other boat's shared data) and train those while suppressing training that is not relevant at that point in time.

[*]Simplicity – The beauty of AI is that no first principle type logic is necessary for someone to work out.  Just adding the sensors to the mix and adjusting the model to absorb their input will automatically start to learn about how you handle the boat in various sea states.

If done wrong it will take forever to learn and not work very well.

[*]Generalizing – I hope the above examples have given you some taste of what is involved.  I don’t believe the talk about making one common, centralized model available to everyone is feasible at any level.  Let me ask the real sailors here – Do you control your boat differently based on how it is loaded – Heavy, light?  How would you propose to tell the computer this state?  Do you believe that every 40’ monohull is the same?  To be able to quantify every parameter of boats so the learning can distinguish from one 40’ boat from another would be staggeringly complex.  Naval Architects have been trying to quantify boats since day-one. 

The model needs to be able to determine similarities with other vessels if it is to take advantage of transfer learning.

[*]Trust Issues - Would I ever trust it?  Probably only while I'm watching it.  I'm more likely to trust PyPilot.  The thing about AI you don't know what edge case might crop up that never cropped up before.  Would it spike the helm hard over because of some

I dont trust most people either, especially not even myself. For cruising the best defense may be to sail so the boat will not be in danger regardless of helm action, but this eliminates the possibility of high performance. For this reason true machine learning models are in my knowledge not used even on ocean racing boats. Instead they prefer more advanced reliable algorithms. A model could flip a multihull which is bad.

I'm curious if there is anyone else on the forum that has worked with AI algorithms and has a different opinion.  I believe AI can do some staggeringly complex things for good or ill.  It just depends on how much effort and computer power you're willing to throw at the problem.  If I could do it with a RasPi, I might experiement.  If it requires a AMD 7950X with a Nvidia 4090 using about a kilowatt of power... I think I'll pass.  

it could be worth the power to potentially train and/or understand the problem better. Further, in the future trillions of operations can be done on a few watts.

[Hillzzz]

I would like add my 2 cents worth.  I’ll give an Artificial Intelligence-101 summary so my opinions at the end will have some meat behind them and hopefully sound intuitive… versus just an opinion.  I'd say I'm only an intermediate sailor and would defer to almost anyone here as knowing more and hope to learn from you all.  My background is more technical and I like the idea of using AI and have worked with it for a while.  I feel I have a pretty good grasp of what it can and can't do... at least at the commodity level of hardware - Arduino, RasPi, PC's.  These MPU/CPU/GPUs can do AI the same as ChatGPT4... just at a far smaller scale.  The concepts are the same.

Nuts and Bolts of AI

I’ll start out with a 10,000 foot view.  I’m not sure how non-technical this can be made, but I’ll try.  Whereas I would have trouble with naming all the different lines on a boat... I just color code mine and tell someone to pull the green one.    Similarly, I'm guessing many bailed out of Mathematics  classes at the earliest opportunity.  

Inputs - If you read a little about AI, you'll read the term parameters.  This is just how many inputs a model is using to solve the problem at hand.  ChatGPT4 is said to have over a trillion parameters.  In our case, a parameter is simply our wind and boat speed, wind and boat direction, GPS data, wave data, … etc.  We wouldn't need near that many for our problem.  For a specific boat (our boat) we might get away with a few tens of parameters.  In a more general case of trying to apply to all of our boats, we would likely need hundreds. 

The Model – The model is the “brain” of AI.  This is where mathematics comes into play.  I don’t recall where I was first introduced to Matrices, but I read that it is commonly taught around Algebra II in high school these days.  The model is nothing more than one or more matrices.  Basic Linear Algebra is used to take the Inputs do multiplication and/or addition steps on these matrices.  The values in this matrix are simply numbers.  They are most often called weights and biases.  The best way to think of this is there are hundreds of Gains. 

Outputs – This is what spits out of the Model.  In the case of ChatGPT4, this might be a sonnet or a picture or any number of other things.  For us, in the simplest case, is where to position the rudder.  A little more complex might be sheeting of the sails.  I think I read where Sean was considering adding control line adjustment to PyPilot.

Two Phases of AI

Using the model is the easiest – Plug in all the input parameters, run the linear-algebra steps and out pops the answer.  For our case of say several tens of parameters and the output of only controlling the rudder, even the lowest Arduino could handle this problem at several hundred hertz.  IOW, an Arduino Nano could pilot the boat just fine once the model is defined.

Learning is where the Model is taught and eventually defined.  Teaching a model takes more memory and more time.  Whereas an Arduino can run our simplistic model, it would take a little more power to teach the model.  But, even an ESP8266 or ESP32 could probably do it and certainly any RasPi.  The model described as a matrix has many values in it.  The number of values in this matrix is well characterized based on the number of inputs and the number of outputs.  Once characterized, its size remains fixed – for those inputs, outputs and the goals being taught.  Like an infant brain, a starting Model is fundamentally blank.  The values are usually set to some random number between 0 and 1.  The inputs (from our sensors) are set and the Using calculations takes place to get the outputs.  Now… since the brain is ignorant, the output is basically garbage.  Learning involves taking the outputs and telling the model just how bad the results are.  Adjustments are made to the weights and/or biases and the Using calculations are run again.  Hopefully a model converges on a useful solution… Eventually!

Ways of Learning

I’ll describe two ways.  I’m sure there are many more and there are many sub-classes of each of these.  Both methods attempt to achieve the same thing. 

Back Propagation – This is the quicker and more definitive way.  Its main limitation is it is based on the teacher knowing the answer decisively.  It uses more linear-algebra, and calculus to make the adjustments to the Model in a well characterized way.  The theory can be distilled down into a program that is fairly small and strait forward.  As mentioned it could fit on an ESP8266 or ESP32 for our couple hundred parameter use case.  It would be pretty easy to set up the Learning program to accept all the sensor data that the PyPilot now takes as inputs.  In the evaluation phase, it would simply be compared to where the human helmsman is currently positioning the rudder.  If anyone is interested, the “Hello World” program for AI, Back Propagation determines what a hand-written digit is.  It uses a publicly available database of pre-written digits from hundreds of people.  Here is a series of YT that explains both the Using phase and the Back Propagation phase.  These get far more technical in describing the above paragraphs. https://www.youtube.com/playlist?list=PL...x_ZCJB-3pi  In this Hello World example the AI can get the answer right around 96% of the time. 

Genetic Algorithms are used to handle cases where the answer isn’t decisively known.  I’ve been more interested in exploring this case.  It is called Genetic because is attempts to model the learning procedure similar to Darwin Evolution principles.  In its simplest form, a generation of Models are generated.  For instance 80 models with all the values in the model are randomly chosen.  Each model is run through a series of tests to evaluate its fitness for survival.  The stronger ones mate by combining their values into children models.  This goes on for many generations and hopefully convergence is achieved.  Sometimes it works, sometimes not.  Here is (more or less) the founder’s book on the strategy.  It can generally be picked up for under $10 used - https://www.abebooks.com/9780201157673/G...157675/plp

I’ll give the example I was working on.  It uses a two-wheeled robot that can go forward and backward and turn.  Its inputs are some distance sensors around its perimeter and the speed of the two motors driving it.  The outputs are the new speeds of the motors to be used.  In this example there is no higher power like a human that knows the right answer of how fast and in which direction the motors should turn.  A goal is set and the robot is evaluated by this goal.  The goal for this robot is more fit if it goes faster and stays further away from obstructions.  It fails and dies if it runs into a wall.  It fails if it spins around and goes nowhere. 
Here is a YT of the robot in a simple irregular donut type arena when it is first starting to learn.  Think – A baby trying to learn to walk.
https://www.youtube.com/watch?v=cXlDSS_dojI

Here in later generations where some things are starting to come together.  It at least gets out of the first little region.  You’ll note in these, it has learned that it can turn and/or back up after hitting a wall.
https://www.youtube.com/watch?v=1FWgiEgE75s

Here is a fairly good candidate that can cycle through.
https://www.youtube.com/watch?v=5vjrMeTcAtw

And finally, transferring it to another arena where it wasn’t trained.  This shows that it can apply what it knows in unknown areas.
https://www.youtube.com/watch?v=L0flTXwEigg

Take Aways and Opinions

1. None of this is 100% reliable.  Much of the Mathematics is based on statistics and probability.  We only run into a rock, 2% of the time. 
   
2. Back Substitution can be easily implemented for known outputs
   
3. Genetic Algorithms can solve for problems with outputs that are hard to quantify, but with goals for trends of outputs.
   

My Opinions

1. Power Efficiency - Watching Sean’s videos of his PyPilot on a couple of totally different boats while they automatically change course at the waypoints and handle different sea states I am struck with how little the helm needs to move.  No matter what, an AI version will be far less energy efficient of electric power. 
   
2. Learning – It should be pretty easy to create a Back Substitution version and install it on your OpenMarine RasPi.  It would start with the random values in the model (Infant Brain).  It could be completely passive and safe.  You would have to add the hardware for rudder position feedback.  It could run in the background and absorb all the incoming sensor data and take your responses to controlling the helm and start learning and modifying the Model. 
   
3. Teacher – It would only be as good as the helmsman. It would learn your good habits and your bad habits.  It’ll learn your lazy, near sleepy habits.  It’ll take your mistakes as gospel and incorporate them into its brain.  I think if (and I might try) to add such an AI once I get a working PyPilot version first.  I could let PyPilot teach it to start with and then if/when I think I can do better, I might step in and let it learn from me.  Would it do as good as PyPilot in the general case?  Would it do as good as me (if I am better) in those special cases?  Sounds like a very interesting science project.
   
4. Sea State – If you want to make it handle different sea states, it should be obvious that you need sensors that provide some kind of data that will change based on sea states.  This might be as simple barometric pressure sensor (~$1) to measure rise and fall of the boat.  It might be accelerometers and gyros to measure rocking and rolling of the vessel. 
   
5. Simplicity – The beauty of AI is that no first principle type logic is necessary for someone to work out.  Just adding the sensors to the mix and adjusting the model to absorb their input will automatically start to learn about how you handle the boat in various sea states.
   
6. Generalizing – I hope the above examples have given you some taste of what is involved.  I don’t believe the talk about making one common, centralized model available to everyone is feasible at any level.  Let me ask the real sailors here – Do you control your boat differently based on how it is loaded – Heavy, light?  How would you propose to tell the computer this state?  Do you believe that every 40’ monohull is the same?  To be able to quantify every parameter of boats so the learning can distinguish from one 40’ boat from another would be staggeringly complex.  Naval Architects have been trying to quantify boats since day-one.  Are there any parameters you believe you can look at in commonly available charts comparing boats (without seeing the boat) and would be able tell how to handle it merely by the numbers? I can not imagine I would ever voluntarily download and use a generic model.  The mere prospect of configuring it with my boat's parameters doesn't seem feasable.
   
7. How to Sail – Do you want it to sail battery power efficiently?  How about smoothest ride?  Fastest?  Safest?
   
8. Trust Issues - Would I ever trust it?  Probably only while I'm watching it.  I'm more likely to trust PyPilot.  The thing about AI you don't know what edge case might crop up that never cropped up before.  Would it spike the helm hard over because of some freak combination of input parameters.  
   
9. Racer – The only use case I can see where an AI auto pilot might do better than say PyPilot is a racer.  I recently read an article about a long distance race where there was a professional helmsman and various others.  I’m picturing a Pro/Am type event.  The racer stated that when off shift, down in the boat without being able to see the sea state would cringe at the helmsmen on shift's responses.  He could feel the state of the boat and knew exactly how he would have controlled it… and that the other did not do it that way.  He could feel this in the bunk!  I think a Back Propagation AI dedicated with this one race boat and taught only when the racer was on-helm (and rested) could approach his abilities and certainly be better than the off-watch helmsmen.  I believe this would be great for any Vendée Globe type boat/contestant.  I however think that the AI being trained on one boat/driver could not be applied to any other boat (no matter how close they are similar).  I would imagine at that level, two identical boats would probably be setup differently based on their human. 
   

I'm curious if there is anyone else on the forum that has worked with AI algorithms and has a different opinion.  I believe AI can do some staggeringly complex things for good or ill.  It just depends on how much effort and computer power you're willing to throw at the problem.  If I could do it with a RasPi, I might experiement.  If it requires a AMD 7950X with a Nvidia 4090 using about a kilowatt of power... I think I'll pass.  

Hi Inq, wow! thats a a lot of thought for 2 cents! really interesting thoughts and opinions that will take me a while to digest. Thanks for that. I have decided to do this project in stages. Starting with designing a waterproof box for the screen and electronics, then the object detection model then the AI software ad training. I recon its a couple of years work to get something you can trust 96% of the time but here goes....

cheers, H.

[jim321]

i just got a coral pcie board to play with, just installing drivers is hard. i had to install debian 10 just to do that.
there's been no updates on the github for years.

[Inq]

Hi Inq, wow! thats a a lot of thought for 2 cents! really interesting thoughts and opinions that will take me a while to digest. Thanks for that.

  Well... maybe 10 cents worth. 

Even on a robotics forum I found there were many misconceptions about what AI is and can do.  Before really digging into it... I had many also.  Am I free of misconceptions now?  Probably not.  The field wasn't barely known out of the lab several years ago.  ChatGPTx and AI robotics were lab rats.  Now, it has exploded on the public.  What will it accomplish in another couple of years?  I certainly don't have a crystal ball. 

I felt it important to give a baseline of what AI is (or at least my current understanding of it).  I thought it important to write a baseline primer for those that still see AI as a magic, black box.  ChatGPT4 can easily walk us down that delusion.  It was meant to put a stake in the ground so we're all talking on the same page.  Those with more knowledge about AI would come back and say, "Inq, have you checked this out?"  I'm here to learn to add OpenMarine/PyPilot to my boat.  I will be an apt pupil for that.  

I think many here have the belief that AI is the magic bullet that will make PyPilot better.  As I tried to illustrate above, I think it can, in very tight, specific cases.  Otherwise, I don't think a general, all-boat, crowd based brain is possible and certainly not desirable.  Would you want your Outremer catamaran being influenced by data from Bubba's diesel tug-boat?  Do you trust everyone's helmsmen skills on this forum to be better than yours (or PyPilot's current abilities) and let their teachings influence your AI usage on your boat... say in storm seas?  

I'm not some old fuddy-duddy, set in my ways.  I test drove a Tesla and let FSD drive me around roads that freaked me out.  It is scary impressive good.  And one day, when my children feel compelled to take my driver's license away (cause I'm a senile old coot), I can say... no problem... Tessy, take me home! 

I have decided to do this project in stages. Starting with designing a waterproof box for the screen and electronics, then the object detection model then the AI software ad training. I recon its a couple of years work to get something you can trust 96% of the time but here goes....

cheers, H.

That's some impressive goals.  The AI object detection alone is a huge one.  And that I think would be best part that would be useful to everyone even if it doesn't tie in to the autopilot!  Just alerting you to that cargo container iceberg in your path while your dozing under autopilot!

Have you asked ChatGPT to write any code?  It might spit it out next week. 

[Onno]

My 2 cents, I gets Seans system steering perfectly how I want and what is technical possible on any boat in any circumstances in less then 2 minutes!
So did you test it and did you understand why it did steer like you wanted or did not? 
If you don't understand it how are you gone teach it to AI, and how does AI know what is the wanted steering?
So for basic steering (keeping course) AI will probably work, but for 95% of my users the default settings also work.
For example this "Guy"   he installed the system took off and sailed the ocean without even knowing how to open the webinterface:
https://www.facebook.com/watch/?extid=CL...9671635966

[Hillzzz]

Hi Inq, wow! thats a a lot of thought for 2 cents! really interesting thoughts and opinions that will take me a while to digest. Thanks for that.

  Well... maybe 10 cents worth. 

Even on a robotics forum I found there were many misconceptions about what AI is and can do.  Before really digging into it... I had many also.  Am I free of misconceptions now?  Probably not.  The field wasn't barely known out of the lab several years ago.  ChatGPTx and AI robotics were lab rats.  Now, it has exploded on the public.  What will it accomplish in another couple of years?  I certainly don't have a crystal ball. 

I felt it important to give a baseline of what AI is (or at least my current understanding of it).  I thought it important to write a baseline primer for those that still see AI as a magic, black box.  ChatGPT4 can easily walk us down that delusion.  It was meant to put a stake in the ground so we're all talking on the same page.  Those with more knowledge about AI would come back and say, "Inq, have you checked this out?"  I'm here to learn to add OpenMarine/PyPilot to my boat.  I will be an apt pupil for that.  

I think many here have the belief that AI is the magic bullet that will make PyPilot better.  As I tried to illustrate above, I think it can, in very tight, specific cases.  Otherwise, I don't think a general, all-boat, crowd based brain is possible and certainly not desirable.  Would you want your Outremer catamaran being influenced by data from Bubba's diesel tug-boat?  Do you trust everyone's helmsmen skills on this forum to be better than yours (or PyPilot's current abilities) and let their teachings influence your AI usage on your boat... say in storm seas?  

I'm not some old fuddy-duddy, set in my ways.  I test drove a Tesla and let FSD drive me around roads that freaked me out.  It is scary impressive good.  And one day, when my children feel compelled to take my driver's license away (cause I'm a senile old coot), I can say... no problem... Tessy, take me home! 

I have decided to do this project in stages. Starting with designing a waterproof box for the screen and electronics, then the object detection model then the AI software ad training. I recon its a couple of years work to get something you can trust 96% of the time but here goes....

cheers, H.

That's some impressive goals.  The AI object detection alone is a huge one.  And that I think would be best part that would be useful to everyone even if it doesn't tie in to the autopilot!  Just alerting you to that cargo container iceberg in your path while your dozing under autopilot!

Have you asked ChatGPT to write any code?  It might spit it out next week.   

As a matter of fact I have... Um it's not terribly good. I had to re-write it but it was an interesting experiment. I think it depends what you ask it to write. Common stuff its probably very good at - I know a few programmers that think they will be out of a job in a year or two. But edge cases like mine where it has not been trained on similar projects. It waffles!!!!

On the subject of Tesla's self driving car taking you home. Its been done before... My brother used to run Horse and Carriage tours of the New Forest for tourists. They stopped at various pubs along they way and by the end of the trip my brother would have been bought a significant number of alcoholic drinks. So he heaved himself into the drivers seat and said "home horses" and off they went. At each junction the leading pair of horses would stop, look both ways, and cross or turn only if it was clear. He always got home safely. It just taken Tesla an age to re-invent 18th century technology 

H.

[Inq]

A lot to unpack, and sorry if I am brief

...

We have to input time-series data.  So at 20 updates per second,  with 9 inertial sensors,  this is 180 inputs per second.  If we were to feed the previous 10 seconds of intertial data this is 1800 inputs.    There can likely be ways to optimize/decimate this.  There are also other inputs like gps, wind, rudder,  motor current and many others including potentially cameras (which can add orders of magnitude more input considering video of previous 10 seconds)

I think I'll "unpack" it to one thought per post. 

I haven't dived into your code.  Like I mentioned, I don't read Python all that well. 

I assume(d) that PyPilot only needs to listen to the OpenMarine network for the re-sent Singnal-K messages and had some cyclic buffer to retain the samples necessary to do any time domain issues and or Kalman filtering.

You might take a look at a sub-class of problems using Recurrent Neural Network (RNN).  Here is a study that illustrates the power of it.  Considering the minimal input, the system was able to do some remarkable goals.  https://www.researchgate.net/publication...bile_Robot 

In a nut-shell, the ANN maintains a running historic data and if implemented as I suspect it could, it could eliminate the need to feed historic data and thus fall back to a parameter count being equal to the sensor count.  

Having never used TensorFlow, I don't know if it might already have RNN and thus be an easy migration.

[Inq]

Learning is where the Model is taught and eventually defined.  Teaching a model takes more memory and more time.  Whereas an Arduino can run our simplistic model, it would take a little more power to teach the model.  But, even an ESP8266 or ESP32 could probably do it and certainly any RasPi.  The model described as a matrix has many values in it.  The number of value

My basic tests show the pi zero cannot do online training with few hundred parameters,  at least not using tensorflow and python.  A pi 4 or pi 5 however potentially can.    With more inputs it becomes marginal, and video processing it definitely cannot.

I suspect you are right about tensoflow and/or python.  Here are some benchmarks I ran using a C++ coded version.

Model
Basic Back Propagation ANN
196 Input Parameters
12 Parameter Hidden Layer  (Matrix [196x12])
10 Outputs (Matrix [12x10])

Apologies, I tend to use Arduino generically since I use the Arduino IDE.  I only use ESP8266 and ESP32 for "Arduino" work so I have the WiFi and control via a Web Interface.  I hate wasting pins and MPU time pumping out to a wired display.

Benchmarks
ESP8266 @ 80 MHz runs at 66 Hz (This is forward and back propagation combined)
ESP8266 @ 160 MHz = 132 Hz
RasPi Pico W @ 133 MHz = 181 Hz
ESP32-S3 @240 MHz = 1580 Hz
Windows i7 3.3GHz = 43.7 KHz

The Pico, ESP32 and Windows were only using one thread.

I'll see if I can port this over to Raspberry Pi.  Shouldn't take that long.  I have a Zero-W, 3B+ and 4B-8GB handy to benchmark.

[Inq]

Using the Same Model
Basic Back Propagation ANN
196 Input Parameters
12 Parameter Hidden Layer  (Matrix [196x12])
10 Outputs (Matrix [12x10])

More Benchmarks
Raspberry Pi Zero-W = 691 Hz
Raspberry Pi 3B+ = 692 Hz
Raspberry Pi 4B - 8GB = 4158 Hz

[Hillzzz]

Learning is where the Model is taught and eventually defined.  Teaching a model takes more memory and more time.  Whereas an Arduino can run our simplistic model, it would take a little more power to teach the model.  But, even an ESP8266 or ESP32 could probably do it and certainly any RasPi.  The model described as a matrix has many values in it.  The number of value

My basic tests show the pi zero cannot do online training with few hundred parameters,  at least not using tensorflow and python.  A pi 4 or pi 5 however potentially can.    With more inputs it becomes marginal, and video processing it definitely cannot.

I suspect you are right about tensoflow and/or python.  Here are some benchmarks I ran using a C++ coded version.

Model
Basic Back Propagation ANN
196 Input Parameters
12 Parameter Hidden Layer  (Matrix [196x12])
10 Outputs (Matrix [12x10])

Apologies, I tend to use Arduino generically since I use the Arduino IDE.  I only use ESP8266 and ESP32 for "Arduino" work so I have the WiFi and control via a Web Interface.  I hate wasting pins and MPU time pumping out to a wired display.

Benchmarks
ESP8266 @ 80 MHz runs at 66 Hz (This is forward and back propagation combined)
ESP8266 @ 160 MHz = 132 Hz
RasPi Pico W @ 133 MHz = 181 Hz
ESP32-S3 @240 MHz = 1580 Hz
Windows i7 3.3GHz = 43.7 KHz

The Pico, ESP32 and Windows were only using one thread.

I'll see if I can port this over to Raspberry Pi.  Shouldn't take that long.  I have a Zero-W, 3B+ and 4B-8GB handy to benchmark.

Very interesting benchmarks. the Pi5 is quite a bit faster (up to 3 times faster) than the Pi4 but the Tensor libraries have not been updated for some time so to make everything work on a Pi5 you have to downgrade Python3 to 3.9 as the version in Bookworm doesn't work.

H.