*newbie* /dev/root full after 24hrs - Printable Version +- OpenMarine (https://forum.openmarine.net) +-- Forum: OpenPlotter (https://forum.openmarine.net/forumdisplay.php?fid=1) +--- Forum: Bug Reports (https://forum.openmarine.net/forumdisplay.php?fid=4) +--- Thread: *newbie* /dev/root full after 24hrs (/showthread.php?tid=1216) |
RE: *newbie* /dev/root full after 24hrs - PaddyB - 2018-08-06 (2018-08-06, 05:49 PM)tkurki Wrote: Could you try once more with ######## OK, just turned off the node red sending NMEA to UDP flow. Maybe worth mentioning there is a sigK node red app running as well. Code: pi@openplotter:~ $ sudo lsof -i | grep 30330 RE: *newbie* /dev/root full after 24hrs - stripydog - 2018-08-06 Here's what I think is happening. kplex has a not-quite-conventional approach to its data flow in that it doesn't use select(), poll() or similar but uses a separate thread for every input and every output. Read interfaces sit in a blocking read. If the other end of a tcp socket closes the connection the read will return an error, kplex knows the connection is closed and will terminate the interface. If the interface is bi-directional it will also terminate its "pair" in the write thread. But this is an output-only interface. What seems to be happening here is that because no data is being sent to kplex, it's not writing to any of its open sockets. Because it doesn't write to anything, it doesn't get an error to tell it the socket it is writing to is closed and so doesn't know to terminate the thread and close the socket. if I'm right, removing "direction=out" would make the socket bi-drectional and cause the read thread to terminate the connection when the client shut down. Undesirable behaviour for sure but this isn't so much a bug as a condition which wasn't anticipated (no outputs to write and repeated reconnections) *newbie* /dev/root full after 24hrs - PaddyB - 2018-08-06 Fantastic. So far anyway. Just manually edited the sigk 30330 in kplex to both and seems stable so far, though only been a few minutes. Will leave it overnight and report back in the morning. Nice one mate! Sent from my SM-T813 using Tapatalk RE: *newbie* /dev/root full after 24hrs - tkurki - 2018-08-07 Code: tjk:kplex-1.3.4 tjk$ cat kplex.conf and then Code: tjk:kplex-1.3.4 tjk$ nc localhost 30330 results in CLOSE_WAIT state. When I remove the direction=out line the connection closes properly. RE: *newbie* /dev/root full after 24hrs - tkurki - 2018-08-07 It turns out that SK server does have reconnect logic for idle tcp connections and there is no way to disable it. I don't think that makes sense for tcp connections, that are reliable by virtue of tcp. I'll get back to this here once I've done something about it. RE: *newbie* /dev/root full after 24hrs - PaddyB - 2018-08-07 with - Code: [tcp] I'm now getting - Code: pi@openplotter:~ $ sudo lsof -i | grep 30330 RE: *newbie* /dev/root full after 24hrs - Sailoog - 2018-08-07 (2018-08-06, 09:13 PM)tkurki Wrote: Sorry about that and thanks for the clarification. I thought I was linking the latest settings, as the link was to master in Github. Now I see that there are no 1.x tags in Github - have you moved the source someplace else? The old version and stable is master branch, the current version and active is beta branch. Beta branch will become master branch soon. https://github.com/sailoog/openplotter/tree/beta I have done that suggested change in data routing image, thanks. Thanks tkurki and stripydog (signal k and kplex developers) for going into this issue. RE: *newbie* /dev/root full after 24hrs - stripydog - 2018-08-07 (2018-08-06, 11:11 PM)stripydog Wrote: Undesirable behaviour for sure but this isn't so much a bug as a condition which wasn't anticipated (no outputs to write and repeated reconnections) Short of a complete rewrite (yes, yes, kplex two-dot-oh) I can think of a couple of ways to mitigate against the problem we saw here: make keepalives work where kplex is a server (rather than a client) or provide an option to output some kind of heartbeat (proprietary sentence which should be ignored by receivers). All comment/suggestionsgratefully received (but not guaranteed to be acted upon :-) RE: *newbie* /dev/root full after 24hrs - e-sailing - 2018-08-08 (2018-08-07, 09:34 PM)stripydog Wrote: Short of a complete rewrite (yes, yes, kplex two-dot-oh) I can think of a couple of ways to mitigate against the problem we saw here: make keepalives work where kplex is a server (rather than a client) or provide an option to output some kind of heartbeat (proprietary sentence which should be ignored by receivers). All comment/suggestionsgratefully received (but not guaranteed to be acted upon :-) So the easiest way is a proprietary sentence which should be ignored by receivers. Is it a good idea to do it in OpenPlotter or do you want to add one in kplex? RE: *newbie* /dev/root full after 24hrs - tkurki - 2018-08-08 (2018-08-07, 09:34 PM)stripydog Wrote:(2018-08-06, 11:11 PM)stripydog Wrote: Undesirable behaviour for sure but this isn't so much a bug as a condition which wasn't anticipated (no outputs to write and repeated reconnections) FYI I am going to remove the disconnect-on-idle behavior in SK server anyway. That should make this particular interaction go away. Outputting bogus heartbeats seems like a kludge to me - the client doesn't need them, this is kplex internal. How about a watchdog thread that closes connections that the client has closed (that are in CLOSE_WAIT), no matter what? |