No Pypilot data into Signalk Server after OS Backup swap

[seandepagnier]

So you have a pi3, and a pi5 both with signalk servers?

then also a victron?

It might report the victron in the log but should not detect it as a signalk server this is correct right?

As for the pi3 and pi5 both with signalk servers, yes that likely confuse it.

Being able to override it in pypilot.conf makes sense to me the most, and I thought it would work but I didn't really test with multiple signalk servers and I wrote this a while ago.

So the other thing is, if it did detect a server, maybe it should make access requests to all of them? Then it needs to support sending data to multiple signalk servers and it becomes an issue if it is receiving data from multiple signalk servers how to resolve conflicts, so I never worked on this. The other option would be to submit access requests to all of them but then only connect to one.

If it did override in the config and if it doesn't detect that server makes an access request to the one it does detect, then what it overwrites the config again? Because if it does, then it might be bad if you have a momentary reboot of the main server, but if it doesn't it will be difficult to change network config or use a new server if you added one without editing the file everytime. So maybe the only solution is to use the config as a hint, but even then if the main server reboots it could switch to the secondary signalk server and not ever switch back.

So honestly without supporting connecting to multiple signalk servers at the same time I think there will always be conflicts for someone but it could probably do better than it does.

So I would ask, if it could do anything, what exactly would it do?

[aitonos]

Thanks Sean!

This is not a very typical installation, but more and more people are running Venus OS on boats, and in my experience it works flawlessly.

On my boat I currently have three Raspberry Pis running 24/7:

• Pi 5 – Main CPU and main Signal K server, running OpenPlotter v4 + MacArthur HAT, IMU and Maiana AIS.
The IMU on the Pi 5 is quite affected by EMI, so in practice I prefer using the IMU from the Navitop TinyPilot.

• Pi 3 – Running Venus OS (I believe the large edition), which includes Signal K and Node-RED.
This Pi manages the entire Victron environment, reports data to the main Signal K server, and also reports to the Victron VRM portal.

• Pi Zero – Navitop TinyPilot. It works wonderfully: very accurate IMU readings, and it connects perfectly to NMEA data from the main Signal K server via IP configuration.

So effectively I have 3 Pis and 2 Signal K servers online. One is the real “master”; the other one (on Venus OS) is not really used as a server. As far as I understand, Venus OS shares its data via MQTT, not via a Signal K stream.

I don’t think it’s necessary—or even advisable—to report to two Signal K servers at the same time. According to Signal K recommendations, there should be a single master server, so I prefer to report to only one.

The first time I installed your system, I believe both the Pi 5 and the Pi 3 were running Signal K servers, and TinyPilot selected the correct one. Possibly by chance, or maybe the Pi 3 was not running at that moment—I’m not completely sure. In any case, everything worked perfectly out of the box (which is great, by the way—wonderful piece of software, very much appreciated!).

Later, the Pi 5 system collapsed and I had to replace the NVMe card, while the Pi 3 kept running normally. When I brought the Pi 5 back online using a fresh backup (already configured to receive data from TinyPilot), it didn't receive data anymore.

I spent quite some time troubleshooting until I realized that TinyPilot was detecting the Signal K server running on the Victron system. After not receiving usable data from it, it stopped querying other computers on the network.

If you ask me, From my (limited) understanding, possible improvements or best practices could be:

• Keep automatic discovery, but perform a full network scan (either continuously or at least once) until a “usable” Signal K server grants access.
  
• Keep automatic discovery, but allow an override for a fixed IP. If no IP is configured (or the target is unreachable), fall back to automatic discovery.
  
• Provide a configuration field in the control panel to manually enter the Signal K server IP; if left blank, use automatic discovery.
  
• Document this specific scenario clearly, so users don’t go crazy trying to understand what’s happening.
  

I hope this feedback is useful. If not, no worries at all—feel free to ask me for clarification or tests if needed.

Thanks again!

[seandepagnier]

"Keep automatic discovery, but perform a full network scan (either continuously or at least once) until a “usable” Signal K server grants access."
well then it would have to request access to every signal server it finds? Then what if both grant access what does it do?

I agree about an override ip. Perhaps in this case the IP given is the hint of which server to choose if there are more than one. Maybe the web interface could have a page (linked in configuration) that lists all the signalk servers and lets you prioritize them. You can see how this is a bit of work to handle a case that I haven't run into until now! Otherwise, I think requesting access from all of them until one grants access, and then if you revoke access from the server you dont want it to use anymore it wont use it.

The complicated bit is, even without access it can still receive signalk data, it just can't publish. So without any access it has no clear idea of which server to use in that case, but maybe it is not that important.

What I recommend for now, is just disable the signalk server on the venus os if you can. Is there any reason for it to be running? It will take some time before I push significant changes to pypilot for this as it is way down my list of items, but certainly something I am aware of and will look into.

[barnaclebill22]

I created a PR for one possible solution.
https://github.com/pypilot/pypilot/pull/267
Parameters in pypilot.conf will override zeroconf. The zeroconf process still runs but you would have to erase the signalk lines in pypilot.conf and restart to get zeroconf to re-scan. I started doing this second modification and I got out of my depth with race conditions so I will leave it to someone smarter (like Claude :-)

[aitonos]

I created a PR for one possible solution.
https://github.com/pypilot/pypilot/pull/267
Parameters in pypilot.conf will override zeroconf. The zeroconf process still runs but you would have to erase the signalk lines in pypilot.conf and restart to get zeroconf to re-scan. I started doing this second modification and I got out of my depth with race conditions so I will leave it to someone smarter (like Claude :-)

Thanks for the attempt. 

I will try it, however now it's successfully providing data to the main Sk server. However, I don't know how, as there was a point that it wasn't, even when I put venus os Raspi down for several days. 

So desperate, I activated again the Imu attached to my Mac Arthur hat, which before, when TinyPilot was providing data , was disabled. 

After some days, working on another thing, I noticed that I had messages in Sk server log saying something like  "there was a new device offering heading, roll, etc" (don't remember the exact words), a message shown in a neverending cascade. That was the TinyPilot back to scene. 

So I disabled again the MacArthur Imu and, voila!  I got the data flow as before the problem was. But there is no trace in Sk server of anything. 

I just got an active deltas Ws connection with my Sk server username which don't know where did it come from. I don't even know what data is generating. Mistery! 

One day TinyPilot stopped providing its data again, but maybe due to the fact that these days I'm developing a Sk Plugin and I am restarting Sk server maybe 20 times a day. Now it's OK again. 

In the meantime of this story, I also updated twice Sk installer and sk server to the latest version. 

If I have the same problem again, I'll try your solution, but for the moment, I won't touch it again. 

Thanks a lot for your efforts.