2024-04-12, 03:39 PM
(This post was last modified: 2024-04-12, 03:46 PM by seandepagnier.)
So.. I think the battery-backed autopilot would eliminate most of the issues with slow boot speed (from power lost) as well as provide even more reliability. I will look into setting this up on my boat and begin testing so that I can advise people on how to do it. It is really a separate system and could of course be used with an esp32 system as well.
As for the icm20948 driver on micropython... It may exist, I am sure several people have implemented one, but I worked specifically on the rtimulib one and there are 2 factors:
1) I optimized the driver heavily. It samples the sensors much faster (100hz) and averages them using the fifo. Without this it would use more cpu and/or worse data output, so I would likely have to modify any driver and/or verify it to ensure the data is as good as possible.
2) RTIMULib also implements sensor fusion, using a kalman filter (although optionally other filters) This is all done in c++ which benefits pypilot from reduced cpu on the computation and reduced latency in the control loop. So sampling the raw sensors alone is not enough it needs this as well.
As for a micropython port, I think starting out, disabling certain features that are in separate processes could be appropriate like zeroconf, signalk, and initially even nmea. Honestly if it can be shown that multiple threads in micropython can run as separate freertos threads and therefore utilize separate cores, then this might be the way to go, and just implement python multiprocessing this way (avoid changing the main code and just add a special module that emulates everything needed including inter-process pipes) but the limited ram on these boards could be the bottleneck and it is unlikely to work without more memory. In fact when running pypilot_hat on micropython I did get it to work without external ram, but had to do some careful optimizing of the display driver. I have some esp32 dev boards which have 4mb PSRAM and it could be determined that this is the way to go as it should be plenty of memory.
From what I recall, without external memory, there is < 100kb of free ram for the program itself in a single python thread. it is just not going to be able to support several python processes without PSRAM. I think at minimum there should be 2 processes, one with critical control loop ensuring the best reaction time, and another with everything else. Right now pypilot uses several processes, with 2 in the critical loop (so it can begin to read from the sensors for the next cycle and they will be ready as soon as possible while it is still performing control loop computations) For example, once it completes the control loop and sends the servo command to move the rudder, there is still a bunch of other stuff it needs to do like figure out if it should drop the mode (gps lost) but also to send and receive data from the server to communicate with other processes, and during this time, the other critical control process (the one that basically just runs RTIMULIB ) can already be reading the sensor data so that there is minimal delay when the main control loop process wants it. Perhaps it could be reorganized into a single critical process though, this is something to consider but testing would be needed to verify it, and instrumentation to verify the delay. a few milliseconds might not seem like much but every millisecond counts in this control loop as any delay in reaction results in larger corrections needed which is almost always bad. To be clear, if pypilot were purely c++ (no longer pypilot) it would shave some milliseconds off and improve reaction time slightly, and use far less ram and therefore already work on esp32, but become un-portable, unmaintainable and likely to suffer from bugs that cause it to crash whenever new features are added (unstable) which to me is not worth it. Even after 20 times the development effort, opencpn crashes.. pypilot does not really crash, but sometimes prints (usually informative) information to the log about exceptions when there are bugs. An autopilot that occasionally crashes from software bugs is a bad thing indeed.
Other features of pypilot are just not going to fit without more memory like running the hat service and the web. As for the web service.. likely an entirely new code to service it would be required, but fortunately the web servers for micropython already exist and could likely still use the same java and pypilot/web.py is already only 272 lines so a re-implemented web-esp32.py would probably be most sensible here. It is unclear if a single esp32 has the processing power even with dual cores to handle everything. Currently the pi zero uses about half the cpu to run all pypilot processes, and the pi zero has 3-4x the processing power of an esp32.
Another possibility is to use 2 or more esp32 to run separate services, with one running the core pilot, another for the web service and so on. Another way could be to slow down the data handling to non-critical paths, eg: service data at 4hz instead of 20hz, to external processes and so forth.
There is an upcoming esp32-p4 that might be the most suitable for pypilot. It offers superior processing power but potentially still not enough ram, it is hard to say at this stage.
As for the icm20948 driver on micropython... It may exist, I am sure several people have implemented one, but I worked specifically on the rtimulib one and there are 2 factors:
1) I optimized the driver heavily. It samples the sensors much faster (100hz) and averages them using the fifo. Without this it would use more cpu and/or worse data output, so I would likely have to modify any driver and/or verify it to ensure the data is as good as possible.
2) RTIMULib also implements sensor fusion, using a kalman filter (although optionally other filters) This is all done in c++ which benefits pypilot from reduced cpu on the computation and reduced latency in the control loop. So sampling the raw sensors alone is not enough it needs this as well.
As for a micropython port, I think starting out, disabling certain features that are in separate processes could be appropriate like zeroconf, signalk, and initially even nmea. Honestly if it can be shown that multiple threads in micropython can run as separate freertos threads and therefore utilize separate cores, then this might be the way to go, and just implement python multiprocessing this way (avoid changing the main code and just add a special module that emulates everything needed including inter-process pipes) but the limited ram on these boards could be the bottleneck and it is unlikely to work without more memory. In fact when running pypilot_hat on micropython I did get it to work without external ram, but had to do some careful optimizing of the display driver. I have some esp32 dev boards which have 4mb PSRAM and it could be determined that this is the way to go as it should be plenty of memory.
From what I recall, without external memory, there is < 100kb of free ram for the program itself in a single python thread. it is just not going to be able to support several python processes without PSRAM. I think at minimum there should be 2 processes, one with critical control loop ensuring the best reaction time, and another with everything else. Right now pypilot uses several processes, with 2 in the critical loop (so it can begin to read from the sensors for the next cycle and they will be ready as soon as possible while it is still performing control loop computations) For example, once it completes the control loop and sends the servo command to move the rudder, there is still a bunch of other stuff it needs to do like figure out if it should drop the mode (gps lost) but also to send and receive data from the server to communicate with other processes, and during this time, the other critical control process (the one that basically just runs RTIMULIB ) can already be reading the sensor data so that there is minimal delay when the main control loop process wants it. Perhaps it could be reorganized into a single critical process though, this is something to consider but testing would be needed to verify it, and instrumentation to verify the delay. a few milliseconds might not seem like much but every millisecond counts in this control loop as any delay in reaction results in larger corrections needed which is almost always bad. To be clear, if pypilot were purely c++ (no longer pypilot) it would shave some milliseconds off and improve reaction time slightly, and use far less ram and therefore already work on esp32, but become un-portable, unmaintainable and likely to suffer from bugs that cause it to crash whenever new features are added (unstable) which to me is not worth it. Even after 20 times the development effort, opencpn crashes.. pypilot does not really crash, but sometimes prints (usually informative) information to the log about exceptions when there are bugs. An autopilot that occasionally crashes from software bugs is a bad thing indeed.
Other features of pypilot are just not going to fit without more memory like running the hat service and the web. As for the web service.. likely an entirely new code to service it would be required, but fortunately the web servers for micropython already exist and could likely still use the same java and pypilot/web.py is already only 272 lines so a re-implemented web-esp32.py would probably be most sensible here. It is unclear if a single esp32 has the processing power even with dual cores to handle everything. Currently the pi zero uses about half the cpu to run all pypilot processes, and the pi zero has 3-4x the processing power of an esp32.
Another possibility is to use 2 or more esp32 to run separate services, with one running the core pilot, another for the web service and so on. Another way could be to slow down the data handling to non-critical paths, eg: service data at 4hz instead of 20hz, to external processes and so forth.
There is an upcoming esp32-p4 that might be the most suitable for pypilot. It offers superior processing power but potentially still not enough ram, it is hard to say at this stage.