2018-08-05, 07:46 AM
2018-08-05, 02:41 PM
(2018-06-15, 08:29 PM)e-sailing Wrote: [ -> ]I had the same problem run.log "accept failed for connection to signalk: Too many open files"
I think it has to do with kplex.
If I attach a gps to kplex it needs 8 open files all the time.
If I now disconnect the gps and reboot. Every half minute it adds 1 open file untill the limit of the linux system is reached. Then SignalK fires errors until the disk is full.
To test it: Find the pid of kplex
"ps aux | grep kplex"
print out the counted open files of the pid (replace pid with the process number you got from last command) with
"ls /proc/pid/fd/ | wc -l"
What are the open files? ls -l /proc/pid/fd/
Sent from my iPhone using Tapatalk
2018-08-05, 04:15 PM
(2018-08-05, 02:41 PM)tkurki Wrote: [ -> ]@ tkurki: It looks like this:(2018-06-15, 08:29 PM)e-sailing Wrote: [ -> ]I had the same problem run.log "accept failed for connection to signalk: Too many open files"
I think it has to do with kplex.
If I attach a gps to kplex it needs 8 open files all the time.
If I now disconnect the gps and reboot. Every half minute it adds 1 open file untill the limit of the linux system is reached. Then SignalK fires errors until the disk is full.
To test it: Find the pid of kplex
"ps aux | grep kplex"
print out the counted open files of the pid (replace pid with the process number you got from last command) with
"ls /proc/pid/fd/ | wc -l"
What are the open files? ls -l /proc/pid/fd/
Sent from my iPhone using Tapatalk
Code:
pi@openplotter:~ $ ps aux | grep kplex
pi 4057 0.1 0.0 26724 524 pts/0 Sl+ 14:40 0:00 kplex
pi 4143 0.0 0.0 4372 564 pts/1 S+ 14:40 0:00 grep --color=auto kplex
pi@openplotter:~ $ ls /proc/4057/fd
0 1 2 3 4 5 6
pi@openplotter:~ $ ls /proc/4057/fd
0 1 2 3 4 5 6 7
pi@openplotter:~ $ ls /proc/4057/fd
0 1 2 3 4 5 6 7
pi@openplotter:~ $ ls /proc/4057/fd
0 1 2 3 4 5 6 7 8
pi@openplotter:~ $ ls /proc/4057/fd
0 1 10 2 3 4 5 6 7 8 9
pi@openplotter:~ $
This is the same issue as http://forum.openmarine.net/showthread.php?tid=1355.
If kplex doesn't get any sentence it can't send anything. SignalK waits ca 30 seconds for information from kplex and then thinks the connection is down and doesn't close it. It just opens a new connection. This does happen until there are ca 1000 open connections.
I can't tell if it is a communication problem with kplex or signalk.
A solution could be to disable kplex.
But if there is a device, which is offline most of the time we can't disable kplex without losing functionality.
Another solution is to send a dummy sentence to keep kplex and signalk happy.
2018-08-05, 08:16 PM
I am confused - in http://forum.openmarine.net/showthread.p...41#pid6541 data routing is different from the pic in http://forum.openmarine.net/showthread.p...41#pid6541, which claims to be from the documentation??
2018-08-06, 07:02 AM
I'd like to understand this problem better.
You wrote "SignalK waits ca 30 seconds for information from kplex and then thinks the connection is down and doesn't close it. It just opens a new connection. ". I am not aware of any such logic in SK server. https://github.com/sailoog/openplotter/b...tings.json only has udp connections that have no connection - where is the connection between kplex and sk configured? tcp or udp? Which one is the server and which one the client? The diagrams I found we confusing, see my other post.
Can you run
that should list all kplex open network connections.
If would be great if people could create issues at https://github.com/SignalK/signalk-server-node/issues if they encounter problems with SK server. So many forums, can't read them all...
You wrote "SignalK waits ca 30 seconds for information from kplex and then thinks the connection is down and doesn't close it. It just opens a new connection. ". I am not aware of any such logic in SK server. https://github.com/sailoog/openplotter/b...tings.json only has udp connections that have no connection - where is the connection between kplex and sk configured? tcp or udp? Which one is the server and which one the client? The diagrams I found we confusing, see my other post.
Can you run
Code:
sudo lsof -iP | grep <kplexpid>
that should list all kplex open network connections.
If would be great if people could create issues at https://github.com/SignalK/signalk-server-node/issues if they encounter problems with SK server. So many forums, can't read them all...
2018-08-06, 08:57 AM
Had to sudo apt-get install lsof, then
kplex conf >
Did some testing , I have a barometer NMEA created in node-red going in to port 10111. This looks like it goes into SigK through TCP30330.
Just turned off the node-red nmea creation, this is where some strange things were happening before and kplex was maxing out a core. I'll do another lsof if and when it happens again. Kplex author is a friend so I'll email him, see if any of this means anything to him.
Thanks for your help!!!!!
Code:
pi@openplotter:~ $ lsof -i
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
vncserver 1030 pi 10u IPv4 17556 0t0 TCP localhost:42565->localhost:54068 (ESTABLISHED)
kplex 1424 pi 4u IPv4 15356 0t0 UDP *:10110
kplex 1424 pi 5u IPv4 15359 0t0 TCP *:30330 (LISTEN)
kplex 1424 pi 6u IPv4 19457 0t0 UDP 10.10.10.1:50000
kplex 1424 pi 7u IPv4 19459 0t0 UDP 10.10.10.1:10111
kplex 1424 pi 9u IPv4 17334 0t0 TCP 10.10.10.1:30330->10.10.10.1:60504 (ESTABLISHED)
kplex 1424 pi 11u IPv4 17376 0t0 TCP localhost:30330->localhost:56864 (ESTABLISHED)
signalk-s 13715 pi 3u IPv6 159745 0t0 TCP *:3000 (LISTEN)
kplex conf >
Code:
[udp]
name=system
direction=in
port=10110
[tcp]
name=signalk
direction=out
mode=server
port=30330
###end of defaults
###OpenPlotter GUI settings
[udp]
name=tablet
direction=in
optional=yes
address=10.10.10.1
port=50000
[udp]
name=nodered
direction=in
optional=yes
address=10.10.10.1
port=10111
[serial]
name=gps
direction=in
optional=yes
filename=/dev/ttyOP_gps
baud=4800
Did some testing , I have a barometer NMEA created in node-red going in to port 10111. This looks like it goes into SigK through TCP30330.
Just turned off the node-red nmea creation, this is where some strange things were happening before and kplex was maxing out a core. I'll do another lsof if and when it happens again. Kplex author is a friend so I'll email him, see if any of this means anything to him.
Thanks for your help!!!!!
2018-08-06, 10:43 AM
And this after a little while >
Turn the node red send nmea to kplex again and back to this >
Code:
pi@openplotter:~ $ lsof -i
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
vncserver 1030 pi 10u IPv4 17556 0t0 TCP localhost:42565->localhost:54068 (ESTABLISHED)
kplex 3363 pi 4u IPv4 437300 0t0 UDP *:10110
kplex 3363 pi 5u IPv4 437303 0t0 TCP *:30330 (LISTEN)
kplex 3363 pi 6u IPv4 437305 0t0 UDP 10.10.10.1:50000
kplex 3363 pi 7u IPv4 437307 0t0 UDP 10.10.10.1:10111
kplex 3363 pi 9u IPv4 437310 0t0 TCP *:10113 (LISTEN)
kplex 3363 pi 10u IPv4 434794 0t0 TCP 10.10.10.1:30330->10.10.10.1:54944 (CLOSE_WAIT)
kplex 3363 pi 11u IPv4 434938 0t0 TCP 10.10.10.1:30330->10.10.10.1:55516 (ESTABLISHED)
kplex 3363 pi 13u IPv4 434811 0t0 TCP 10.10.10.1:30330->10.10.10.1:54946 (CLOSE_WAIT)
kplex 3363 pi 14u IPv4 434812 0t0 TCP localhost:30330->localhost:43296 (CLOSE_WAIT)
kplex 3363 pi 15u IPv4 438396 0t0 TCP localhost:30330->localhost:43990 (CLOSE_WAIT)
kplex 3363 pi 16u IPv4 438597 0t0 TCP localhost:30330->localhost:44032 (CLOSE_WAIT)
kplex 3363 pi 17u IPv4 438652 0t0 TCP localhost:30330->localhost:44074 (CLOSE_WAIT)
kplex 3363 pi 18u IPv4 438702 0t0 TCP localhost:30330->localhost:44116 (CLOSE_WAIT)
kplex 3363 pi 19u IPv4 438751 0t0 TCP localhost:30330->localhost:44158 (CLOSE_WAIT)
kplex 3363 pi 20u IPv4 438781 0t0 TCP localhost:30330->localhost:44200 (CLOSE_WAIT)
kplex 3363 pi 21u IPv4 438855 0t0 TCP localhost:30330->localhost:44242 (CLOSE_WAIT)
kplex 3363 pi 22u IPv4 438879 0t0 TCP localhost:30330->localhost:44284 (CLOSE_WAIT)
kplex 3363 pi 23u IPv4 438913 0t0 TCP localhost:30330->localhost:44326 (CLOSE_WAIT)
kplex 3363 pi 24u IPv4 438945 0t0 TCP localhost:30330->localhost:44368 (CLOSE_WAIT)
kplex 3363 pi 25u IPv4 439005 0t0 TCP localhost:30330->localhost:44410 (CLOSE_WAIT)
kplex 3363 pi 26u IPv4 439038 0t0 TCP localhost:30330->localhost:44452 (CLOSE_WAIT)
kplex 3363 pi 27u IPv4 439100 0t0 TCP localhost:30330->localhost:44494 (CLOSE_WAIT)
kplex 3363 pi 28u IPv4 439148 0t0 TCP localhost:30330->localhost:44536 (CLOSE_WAIT)
kplex 3363 pi 29u IPv4 439193 0t0 TCP localhost:30330->localhost:44578 (CLOSE_WAIT)
kplex 3363 pi 30u IPv4 439210 0t0 TCP localhost:30330->localhost:44620 (CLOSE_WAIT)
kplex 3363 pi 31u IPv4 439265 0t0 TCP localhost:30330->localhost:44662 (CLOSE_WAIT)
kplex 3363 pi 32u IPv4 439279 0t0 TCP localhost:30330->localhost:44704 (CLOSE_WAIT)
kplex 3363 pi 33u IPv4 441346 0t0 TCP localhost:30330->localhost:44746 (CLOSE_WAIT)
kplex 3363 pi 34u IPv4 441354 0t0 TCP localhost:30330->localhost:44788 (CLOSE_WAIT)
kplex 3363 pi 35u IPv4 441390 0t0 TCP localhost:30330->localhost:44832 (CLOSE_WAIT)
kplex 3363 pi 36u IPv4 441440 0t0 TCP localhost:30330->localhost:44874 (CLOSE_WAIT)
kplex 3363 pi 37u IPv4 441480 0t0 TCP localhost:30330->localhost:44916 (CLOSE_WAIT)
kplex 3363 pi 38u IPv4 441496 0t0 TCP localhost:30330->localhost:44958 (CLOSE_WAIT)
kplex 3363 pi 39u IPv4 441528 0t0 TCP localhost:30330->localhost:45000 (CLOSE_WAIT)
kplex 3363 pi 40u IPv4 441848 0t0 TCP localhost:30330->localhost:45042 (CLOSE_WAIT)
kplex 3363 pi 41u IPv4 441891 0t0 TCP localhost:30330->localhost:45084 (CLOSE_WAIT)
kplex 3363 pi 42u IPv4 441946 0t0 TCP localhost:30330->localhost:45126 (CLOSE_WAIT)
kplex 3363 pi 43u IPv4 442005 0t0 TCP localhost:30330->localhost:45168 (CLOSE_WAIT)
kplex 3363 pi 44u IPv4 442047 0t0 TCP localhost:30330->localhost:45208 (CLOSE_WAIT)
kplex 3363 pi 45u IPv4 442061 0t0 TCP localhost:30330->localhost:45250 (CLOSE_WAIT)
kplex 3363 pi 46u IPv4 442091 0t0 TCP localhost:30330->localhost:45292 (CLOSE_WAIT)
kplex 3363 pi 47u IPv4 442168 0t0 TCP localhost:30330->localhost:45334 (CLOSE_WAIT)
kplex 3363 pi 48u IPv4 442188 0t0 TCP localhost:30330->localhost:45376 (CLOSE_WAIT)
kplex 3363 pi 49u IPv4 442237 0t0 TCP localhost:30330->localhost:45418 (CLOSE_WAIT)
kplex 3363 pi 50u IPv4 442296 0t0 TCP localhost:30330->localhost:45460 (CLOSE_WAIT)
kplex 3363 pi 51u IPv4 442326 0t0 TCP localhost:30330->localhost:45502 (CLOSE_WAIT)
kplex 3363 pi 52u IPv4 445441 0t0 TCP localhost:30330->localhost:45544 (CLOSE_WAIT)
kplex 3363 pi 53u IPv4 445514 0t0 TCP localhost:30330->localhost:45586 (CLOSE_WAIT)
kplex 3363 pi 54u IPv4 445585 0t0 TCP localhost:30330->localhost:45628 (CLOSE_WAIT)
kplex 3363 pi 55u IPv4 445621 0t0 TCP localhost:30330->localhost:45670 (CLOSE_WAIT)
kplex 3363 pi 56u IPv4 445684 0t0 TCP localhost:30330->localhost:45712 (CLOSE_WAIT)
kplex 3363 pi 57u IPv4 445702 0t0 TCP localhost:30330->localhost:45754 (CLOSE_WAIT)
kplex 3363 pi 58u IPv4 445748 0t0 TCP localhost:30330->localhost:45796 (CLOSE_WAIT)
kplex 3363 pi 59u IPv4 445769 0t0 TCP localhost:30330->localhost:45840 (CLOSE_WAIT)
kplex 3363 pi 60u IPv4 445820 0t0 TCP localhost:30330->localhost:45882 (CLOSE_WAIT)
kplex 3363 pi 61u IPv4 445903 0t0 TCP localhost:30330->localhost:45924 (CLOSE_WAIT)
kplex 3363 pi 62u IPv4 445972 0t0 TCP localhost:30330->localhost:45966 (CLOSE_WAIT)
kplex 3363 pi 63u IPv4 445999 0t0 TCP localhost:30330->localhost:46008 (CLOSE_WAIT)
kplex 3363 pi 64u IPv4 446037 0t0 TCP localhost:30330->localhost:46050 (CLOSE_WAIT)
kplex 3363 pi 65u IPv4 446116 0t0 TCP localhost:30330->localhost:46092 (CLOSE_WAIT)
kplex 3363 pi 66u IPv4 446157 0t0 TCP localhost:30330->localhost:46134 (CLOSE_WAIT)
kplex 3363 pi 67u IPv4 446188 0t0 TCP localhost:30330->localhost:46176 (CLOSE_WAIT)
kplex 3363 pi 68u IPv4 446209 0t0 TCP localhost:30330->localhost:46218 (CLOSE_WAIT)
kplex 3363 pi 69u IPv4 446257 0t0 TCP localhost:30330->localhost:46260 (CLOSE_WAIT)
kplex 3363 pi 70u IPv4 446267 0t0 TCP localhost:30330->localhost:46302 (CLOSE_WAIT)
kplex 3363 pi 71u IPv4 446283 0t0 TCP localhost:30330->localhost:46344 (CLOSE_WAIT)
kplex 3363 pi 72u IPv4 446305 0t0 TCP localhost:30330->localhost:46386 (CLOSE_WAIT)
kplex 3363 pi 73u IPv4 446367 0t0 TCP localhost:30330->localhost:46428 (CLOSE_WAIT)
kplex 3363 pi 74u IPv4 445135 0t0 TCP localhost:30330->localhost:46470 (CLOSE_WAIT)
kplex 3363 pi 75u IPv4 445172 0t0 TCP localhost:30330->localhost:46512 (CLOSE_WAIT)
kplex 3363 pi 76u IPv4 445210 0t0 TCP localhost:30330->localhost:46554 (CLOSE_WAIT)
kplex 3363 pi 77u IPv4 445220 0t0 TCP localhost:30330->localhost:46596 (CLOSE_WAIT)
kplex 3363 pi 78u IPv4 445268 0t0 TCP localhost:30330->localhost:46638 (CLOSE_WAIT)
kplex 3363 pi 79u IPv4 445277 0t0 TCP localhost:30330->localhost:46680 (CLOSE_WAIT)
kplex 3363 pi 80u IPv4 445307 0t0 TCP localhost:30330->localhost:46722 (CLOSE_WAIT)
kplex 3363 pi 81u IPv4 445366 0t0 TCP localhost:30330->localhost:46764 (CLOSE_WAIT)
kplex 3363 pi 82u IPv4 445394 0t0 TCP localhost:30330->localhost:46806 (CLOSE_WAIT)
kplex 3363 pi 83u IPv4 445395 0t0 TCP localhost:30330->localhost:46850 (CLOSE_WAIT)
kplex 3363 pi 84u IPv4 445406 0t0 TCP localhost:30330->localhost:46892 (CLOSE_WAIT)
kplex 3363 pi 85u IPv4 445420 0t0 TCP localhost:30330->localhost:46934 (CLOSE_WAIT)
kplex 3363 pi 86u IPv4 450607 0t0 TCP localhost:30330->localhost:46976 (CLOSE_WAIT)
kplex 3363 pi 87u IPv4 450641 0t0 TCP localhost:30330->localhost:47018 (CLOSE_WAIT)
kplex 3363 pi 88u IPv4 450684 0t0 TCP localhost:30330->localhost:47060 (CLOSE_WAIT)
kplex 3363 pi 89u IPv4 450723 0t0 TCP localhost:30330->localhost:47102 (CLOSE_WAIT)
kplex 3363 pi 90u IPv4 450783 0t0 TCP localhost:30330->localhost:47144 (CLOSE_WAIT)
kplex 3363 pi 91u IPv4 450828 0t0 TCP localhost:30330->localhost:47186 (CLOSE_WAIT)
kplex 3363 pi 92u IPv4 450874 0t0 TCP localhost:30330->localhost:47228 (CLOSE_WAIT)
kplex 3363 pi 93u IPv4 450922 0t0 TCP localhost:30330->localhost:47270 (CLOSE_WAIT)
kplex 3363 pi 94u IPv4 450931 0t0 TCP localhost:30330->localhost:47312 (CLOSE_WAIT)
kplex 3363 pi 95u IPv4 450945 0t0 TCP localhost:30330->localhost:47354 (CLOSE_WAIT)
kplex 3363 pi 96u IPv4 450967 0t0 TCP localhost:30330->localhost:47396 (CLOSE_WAIT)
kplex 3363 pi 97u IPv4 451015 0t0 TCP localhost:30330->localhost:47436 (CLOSE_WAIT)
kplex 3363 pi 98u IPv4 451076 0t0 TCP localhost:30330->localhost:47478 (CLOSE_WAIT)
kplex 3363 pi 99u IPv4 451085 0t0 TCP localhost:30330->localhost:47520 (CLOSE_WAIT)
kplex 3363 pi 100u IPv4 451140 0t0 TCP localhost:30330->localhost:47562 (CLOSE_WAIT)
kplex 3363 pi 101u IPv4 451157 0t0 TCP localhost:30330->localhost:47604 (CLOSE_WAIT)
kplex 3363 pi 102u IPv4 451176 0t0 TCP localhost:30330->localhost:47646 (CLOSE_WAIT)
kplex 3363 pi 103u IPv4 451208 0t0 TCP localhost:30330->localhost:47688 (CLOSE_WAIT)
kplex 3363 pi 104u IPv4 451236 0t0 TCP localhost:30330->localhost:47730 (CLOSE_WAIT)
kplex 3363 pi 105u IPv4 451295 0t0 TCP localhost:30330->localhost:47772 (CLOSE_WAIT)
kplex 3363 pi 106u IPv4 451334 0t0 TCP localhost:30330->localhost:47814 (CLOSE_WAIT)
kplex 3363 pi 107u IPv4 451384 0t0 TCP localhost:30330->localhost:47858 (CLOSE_WAIT)
kplex 3363 pi 108u IPv4 449263 0t0 TCP localhost:30330->localhost:47900 (CLOSE_WAIT)
kplex 3363 pi 109u IPv4 449274 0t0 TCP localhost:30330->localhost:47942 (CLOSE_WAIT)
kplex 3363 pi 110u IPv4 449283 0t0 TCP localhost:30330->localhost:47984 (CLOSE_WAIT)
kplex 3363 pi 111u IPv4 449324 0t0 TCP localhost:30330->localhost:48026 (CLOSE_WAIT)
kplex 3363 pi 112u IPv4 449398 0t0 TCP localhost:30330->localhost:48068 (CLOSE_WAIT)
kplex 3363 pi 113u IPv4 449470 0t0 TCP localhost:30330->localhost:48110 (CLOSE_WAIT)
kplex 3363 pi 114u IPv4 449512 0t0 TCP localhost:30330->localhost:48152 (CLOSE_WAIT)
kplex 3363 pi 115u IPv4 454733 0t0 TCP localhost:30330->localhost:48194 (CLOSE_WAIT)
kplex 3363 pi 116u IPv4 454803 0t0 TCP localhost:30330->localhost:48236 (CLOSE_WAIT)
kplex 3363 pi 117u IPv4 454836 0t0 TCP localhost:30330->localhost:48278 (CLOSE_WAIT)
kplex 3363 pi 118u IPv4 454879 0t0 TCP localhost:30330->localhost:48320 (CLOSE_WAIT)
kplex 3363 pi 119u IPv4 454952 0t0 TCP localhost:30330->localhost:48362 (CLOSE_WAIT)
kplex 3363 pi 120u IPv4 455027 0t0 TCP localhost:30330->localhost:48404 (CLOSE_WAIT)
kplex 3363 pi 121u IPv4 455046 0t0 TCP localhost:30330->localhost:48446 (CLOSE_WAIT)
kplex 3363 pi 122u IPv4 455092 0t0 TCP localhost:30330->localhost:48488 (CLOSE_WAIT)
kplex 3363 pi 123u IPv4 455129 0t0 TCP localhost:30330->localhost:48530 (CLOSE_WAIT)
kplex 3363 pi 124u IPv4 455206 0t0 TCP localhost:30330->localhost:48572 (CLOSE_WAIT)
kplex 3363 pi 125u IPv4 455248 0t0 TCP localhost:30330->localhost:48614 (CLOSE_WAIT)
kplex 3363 pi 126u IPv4 455301 0t0 TCP localhost:30330->localhost:48656 (CLOSE_WAIT)
kplex 3363 pi 127u IPv4 455347 0t0 TCP localhost:30330->localhost:48698 (CLOSE_WAIT)
kplex 3363 pi 128u IPv4 455385 0t0 TCP localhost:30330->localhost:48740 (CLOSE_WAIT)
kplex 3363 pi 129u IPv4 455464 0t0 TCP localhost:30330->localhost:48782 (CLOSE_WAIT)
kplex 3363 pi 130u IPv4 455502 0t0 TCP localhost:30330->localhost:48824 (CLOSE_WAIT)
kplex 3363 pi 131u IPv4 455525 0t0 TCP localhost:30330->localhost:48868 (CLOSE_WAIT)
kplex 3363 pi 132u IPv4 455604 0t0 TCP localhost:30330->localhost:48910 (CLOSE_WAIT)
kplex 3363 pi 133u IPv4 457734 0t0 TCP localhost:30330->localhost:48952 (CLOSE_WAIT)
kplex 3363 pi 134u IPv4 457757 0t0 TCP localhost:30330->localhost:48994 (CLOSE_WAIT)
kplex 3363 pi 135u IPv4 457793 0t0 TCP localhost:30330->localhost:49036 (CLOSE_WAIT)
kplex 3363 pi 136u IPv4 457886 0t0 TCP localhost:30330->localhost:49078 (CLOSE_WAIT)
kplex 3363 pi 137u IPv4 457922 0t0 TCP localhost:30330->localhost:49120 (CLOSE_WAIT)
kplex 3363 pi 138u IPv4 457947 0t0 TCP localhost:30330->localhost:49162 (CLOSE_WAIT)
kplex 3363 pi 139u IPv4 458002 0t0 TCP localhost:30330->localhost:49204 (CLOSE_WAIT)
kplex 3363 pi 140u IPv4 458073 0t0 TCP localhost:30330->localhost:49246 (CLOSE_WAIT)
kplex 3363 pi 141u IPv4 458113 0t0 TCP localhost:30330->localhost:49288 (CLOSE_WAIT)
kplex 3363 pi 142u IPv4 458161 0t0 TCP localhost:30330->localhost:49330 (CLOSE_WAIT)
kplex 3363 pi 143u IPv4 458197 0t0 TCP localhost:30330->localhost:49372 (CLOSE_WAIT)
kplex 3363 pi 144u IPv4 458251 0t0 TCP localhost:30330->localhost:49414 (CLOSE_WAIT)
kplex 3363 pi 145u IPv4 458299 0t0 TCP localhost:30330->localhost:49456 (CLOSE_WAIT)
kplex 3363 pi 146u IPv4 458325 0t0 TCP localhost:30330->localhost:49498 (CLOSE_WAIT)
kplex 3363 pi 147u IPv4 458399 0t0 TCP localhost:30330->localhost:49540 (CLOSE_WAIT)
kplex 3363 pi 148u IPv4 458415 0t0 TCP localhost:30330->localhost:49582 (CLOSE_WAIT)
kplex 3363 pi 149u IPv4 458430 0t0 TCP localhost:30330->localhost:49624 (CLOSE_WAIT)
kplex 3363 pi 150u IPv4 458449 0t0 TCP localhost:30330->localhost:49666 (CLOSE_WAIT)
kplex 3363 pi 151u IPv4 458498 0t0 TCP localhost:30330->localhost:49708 (CLOSE_WAIT)
kplex 3363 pi 152u IPv4 458529 0t0 TCP localhost:30330->localhost:49750 (CLOSE_WAIT)
kplex 3363 pi 153u IPv4 458575 0t0 TCP localhost:30330->localhost:49792 (CLOSE_WAIT)
kplex 3363 pi 154u IPv4 458620 0t0 TCP localhost:30330->localhost:49834 (CLOSE_WAIT)
kplex 3363 pi 155u IPv4 458662 0t0 TCP localhost:30330->localhost:49878 (CLOSE_WAIT)
kplex 3363 pi 156u IPv4 458673 0t0 TCP localhost:30330->localhost:49920 (CLOSE_WAIT)
kplex 3363 pi 157u IPv4 458692 0t0 TCP localhost:30330->localhost:49962 (CLOSE_WAIT)
kplex 3363 pi 158u IPv4 461835 0t0 TCP localhost:30330->localhost:50004 (CLOSE_WAIT)
kplex 3363 pi 159u IPv4 461868 0t0 TCP localhost:30330->localhost:50046 (CLOSE_WAIT)
kplex 3363 pi 160u IPv4 461969 0t0 TCP localhost:30330->localhost:50088 (CLOSE_WAIT)
kplex 3363 pi 161u IPv4 461988 0t0 TCP localhost:30330->localhost:50130 (CLOSE_WAIT)
kplex 3363 pi 162u IPv4 462001 0t0 TCP localhost:30330->localhost:50172 (CLOSE_WAIT)
kplex 3363 pi 163u IPv4 462022 0t0 TCP localhost:30330->localhost:50214 (CLOSE_WAIT)
kplex 3363 pi 164u IPv4 462042 0t0 TCP localhost:30330->localhost:50256 (CLOSE_WAIT)
kplex 3363 pi 165u IPv4 462076 0t0 TCP localhost:30330->localhost:50298 (CLOSE_WAIT)
kplex 3363 pi 166u IPv4 462098 0t0 TCP localhost:30330->localhost:50340 (CLOSE_WAIT)
kplex 3363 pi 167u IPv4 462172 0t0 TCP localhost:30330->localhost:50382 (CLOSE_WAIT)
kplex 3363 pi 168u IPv4 462201 0t0 TCP localhost:30330->localhost:50424 (CLOSE_WAIT)
kplex 3363 pi 169u IPv4 462256 0t0 TCP localhost:30330->localhost:50464 (CLOSE_WAIT)
kplex 3363 pi 170u IPv4 462276 0t0 TCP localhost:30330->localhost:50506 (CLOSE_WAIT)
kplex 3363 pi 171u IPv4 462344 0t0 TCP localhost:30330->localhost:50548 (CLOSE_WAIT)
kplex 3363 pi 172u IPv4 462390 0t0 TCP localhost:30330->localhost:50590 (CLOSE_WAIT)
kplex 3363 pi 173u IPv4 462409 0t0 TCP localhost:30330->localhost:50632 (CLOSE_WAIT)
kplex 3363 pi 174u IPv4 462454 0t0 TCP localhost:30330->localhost:50674 (CLOSE_WAIT)
kplex 3363 pi 175u IPv4 462513 0t0 TCP localhost:30330->localhost:50716 (CLOSE_WAIT)
kplex 3363 pi 176u IPv4 462551 0t0 TCP localhost:30330->localhost:50758 (CLOSE_WAIT)
kplex 3363 pi 177u IPv4 462584 0t0 TCP localhost:30330->localhost:50800 (CLOSE_WAIT)
kplex 3363 pi 178u IPv4 462613 0t0 TCP localhost:30330->localhost:50842 (CLOSE_WAIT)
kplex 3363 pi 179u IPv4 462686 0t0 TCP localhost:30330->localhost:50886 (CLOSE_WAIT)
kplex 3363 pi 180u IPv4 462786 0t0 TCP localhost:30330->localhost:50928 (CLOSE_WAIT)
kplex 3363 pi 181u IPv4 462837 0t0 TCP localhost:30330->localhost:50970 (CLOSE_WAIT)
kplex 3363 pi 182u IPv4 464970 0t0 TCP localhost:30330->localhost:51012 (CLOSE_WAIT)
kplex 3363 pi 183u IPv4 465012 0t0 TCP localhost:30330->localhost:51054 (CLOSE_WAIT)
kplex 3363 pi 184u IPv4 465042 0t0 TCP localhost:30330->localhost:51096 (CLOSE_WAIT)
kplex 3363 pi 185u IPv4 465087 0t0 TCP localhost:30330->localhost:51138 (CLOSE_WAIT)
kplex 3363 pi 186u IPv4 465101 0t0 TCP localhost:30330->localhost:51180 (CLOSE_WAIT)
kplex 3363 pi 187u IPv4 465122 0t0 TCP localhost:30330->localhost:51222 (CLOSE_WAIT)
kplex 3363 pi 188u IPv4 465127 0t0 TCP localhost:30330->localhost:51264 (CLOSE_WAIT)
kplex 3363 pi 189u IPv4 465142 0t0 TCP localhost:30330->localhost:51306 (CLOSE_WAIT)
kplex 3363 pi 190u IPv4 465154 0t0 TCP localhost:30330->localhost:51348 (CLOSE_WAIT)
kplex 3363 pi 191u IPv4 465182 0t0 TCP localhost:30330->localhost:51390 (CLOSE_WAIT)
kplex 3363 pi 192u IPv4 465193 0t0 TCP localhost:30330->localhost:51432 (CLOSE_WAIT)
kplex 3363 pi 193u IPv4 465207 0t0 TCP localhost:30330->localhost:51474 (CLOSE_WAIT)
kplex 3363 pi 194u IPv4 465214 0t0 TCP localhost:30330->localhost:51516 (CLOSE_WAIT)
kplex 3363 pi 195u IPv4 465240 0t0 TCP localhost:30330->localhost:51558 (CLOSE_WAIT)
kplex 3363 pi 196u IPv4 465254 0t0 TCP localhost:30330->localhost:51600 (CLOSE_WAIT)
kplex 3363 pi 197u IPv4 465269 0t0 TCP localhost:30330->localhost:51642 (CLOSE_WAIT)
kplex 3363 pi 198u IPv4 465289 0t0 TCP localhost:30330->localhost:51684 (CLOSE_WAIT)
kplex 3363 pi 199u IPv4 465343 0t0 TCP localhost:30330->localhost:51726 (CLOSE_WAIT)
kplex 3363 pi 200u IPv4 465405 0t0 TCP localhost:30330->localhost:51768 (CLOSE_WAIT)
kplex 3363 pi 201u IPv4 465427 0t0 TCP localhost:30330->localhost:51810 (CLOSE_WAIT)
kplex 3363 pi 202u IPv4 465434 0t0 TCP localhost:30330->localhost:51852 (CLOSE_WAIT)
kplex 3363 pi 203u IPv4 465484 0t0 TCP localhost:30330->localhost:51896 (CLOSE_WAIT)
kplex 3363 pi 204u IPv4 465560 0t0 TCP localhost:30330->localhost:51938 (CLOSE_WAIT)
kplex 3363 pi 205u IPv4 465649 0t0 TCP localhost:30330->localhost:51980 (CLOSE_WAIT)
kplex 3363 pi 206u IPv4 465710 0t0 TCP localhost:30330->localhost:52022 (CLOSE_WAIT)
kplex 3363 pi 207u IPv4 465753 0t0 TCP localhost:30330->localhost:52064 (CLOSE_WAIT)
kplex 3363 pi 208u IPv4 465801 0t0 TCP localhost:30330->localhost:52106 (CLOSE_WAIT)
kplex 3363 pi 209u IPv4 465817 0t0 TCP localhost:30330->localhost:52148 (CLOSE_WAIT)
kplex 3363 pi 210u IPv4 465844 0t0 TCP localhost:30330->localhost:52190 (CLOSE_WAIT)
kplex 3363 pi 211u IPv4 465853 0t0 TCP localhost:30330->localhost:52232 (CLOSE_WAIT)
kplex 3363 pi 212u IPv4 465858 0t0 TCP localhost:30330->localhost:52274 (CLOSE_WAIT)
kplex 3363 pi 213u IPv4 465882 0t0 TCP localhost:30330->localhost:52316 (CLOSE_WAIT)
kplex 3363 pi 214u IPv4 465909 0t0 TCP localhost:30330->localhost:52358 (CLOSE_WAIT)
kplex 3363 pi 215u IPv4 470031 0t0 TCP localhost:30330->localhost:52400 (CLOSE_WAIT)
kplex 3363 pi 216u IPv4 470081 0t0 TCP localhost:30330->localhost:52442 (CLOSE_WAIT)
kplex 3363 pi 217u IPv4 470103 0t0 TCP localhost:30330->localhost:52484 (CLOSE_WAIT)
kplex 3363 pi 218u IPv4 470693 0t0 TCP localhost:30330->localhost:52528 (ESTABLISHED)
signalk-s 3422 pi 3u IPv6 437383 0t0 TCP *:3000 (LISTEN)
pi@openplotter:~
Turn the node red send nmea to kplex again and back to this >
Code:
pi@openplotter:~ $ lsof -i
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
vncserver 1030 pi 10u IPv4 17556 0t0 TCP localhost:42565->localhost:54068 (ESTABLISHED)
kplex 3363 pi 4u IPv4 437300 0t0 UDP *:10110
kplex 3363 pi 5u IPv4 437303 0t0 TCP *:30330 (LISTEN)
kplex 3363 pi 6u IPv4 437305 0t0 UDP 10.10.10.1:50000
kplex 3363 pi 7u IPv4 437307 0t0 UDP 10.10.10.1:10111
kplex 3363 pi 9u IPv4 437310 0t0 TCP *:10113 (LISTEN)
kplex 3363 pi 222u IPv4 470871 0t0 TCP 10.10.10.1:30330->10.10.10.1:36160 (ESTABLISHED)
kplex 3363 pi 223u IPv4 470912 0t0 TCP localhost:30330->localhost:52706 (ESTABLISHED)
signalk-s 3422 pi 3u IPv6 437383 0t0 TCP *:3000 (LISTEN)
pi@openplotter:~ $
2018-08-06, 05:49 PM
Could you try once more with
when the problem is occurring?
What I am looking for is the other end of the single active connection, which is not included in the listing with all the CLOSE_WAIT connections that you sent:
My guess is that lsof is not listing all network connections without sudo.
What I am looking for is something like this:
You can see kplex server listening for new connections and then one established connection from application nc.
As far as I understand this kplex is not handling connections that are closed by a client properly but keeps the file descriptor open with the connection in CLOSE_WAIT state - when there is no data being processed. From what you are saying it closes the connections properly if there is data flowing.
What i don't understand is that in your listings signalk-server does not have any open connections: it is only listening for new http connections on port 3000.
Ooops, I meant
https://sailoog.gitbooks.io/openplotter-..._work.html
and the picture in
http://forum.openmarine.net/showthread.p...41#pid6541
Code:
sudo lsof -i | grep 30330
when the problem is occurring?
What I am looking for is the other end of the single active connection, which is not included in the listing with all the CLOSE_WAIT connections that you sent:
Code:
kplex 3363 pi 11u IPv4 434938 0t0 TCP 10.10.10.1:30330->10.10.10.1:55516 (ESTABLISHED)
My guess is that lsof is not listing all network connections without sudo.
What I am looking for is something like this:
Code:
lsof -i | grep 30330
kplex 8862 tjk 6u IPv6 0x1b4146e9b77ce467 0t0 TCP *:30330 (LISTEN)
kplex 8862 tjk 7u IPv6 0x1b4146e9af129447 0t0 TCP localhost:30330->localhost:56510 (ESTABLISHED)
nc 8863 tjk 5u IPv6 0x1b4146e9b69ff987 0t0 TCP localhost:56510->localhost:30330 (ESTABLISHED)
You can see kplex server listening for new connections and then one established connection from application nc.
As far as I understand this kplex is not handling connections that are closed by a client properly but keeps the file descriptor open with the connection in CLOSE_WAIT state - when there is no data being processed. From what you are saying it closes the connections properly if there is data flowing.
What i don't understand is that in your listings signalk-server does not have any open connections: it is only listening for new http connections on port 3000.
(2018-08-05, 08:16 PM)tkurki Wrote: [ -> ]I am confused - in http://forum.openmarine.net/showthread.p...41#pid6541 data routing is different from the pic in http://forum.openmarine.net/showthread.p...41#pid6541, which claims to be from the documentation??
Ooops, I meant
https://sailoog.gitbooks.io/openplotter-..._work.html
and the picture in
http://forum.openmarine.net/showthread.p...41#pid6541
2018-08-06, 06:15 PM
Just to clarify some background on this issue for OpenPlotter v1.x.x
This is the doc for v1.x.x and this is the data routing: https://docs.sailoog.com/openplotter-v1-...es-it-work
This is the SK server settings for v1.x.x:
Forget any data routing or SK settings for OP v0.x.x.
This is the doc for v1.x.x and this is the data routing: https://docs.sailoog.com/openplotter-v1-...es-it-work
This is the SK server settings for v1.x.x:
Code:
{
"enablePluginLogging": true,
"interfaces": {
"appstore": true,
"nmea-tcp": true,
"plugins": true,
"providers": true,
"rest": true,
"tcp": true,
"webapps": true,
"ws": true
},
"loggingDirectory": "/home/pi/Logs",
"mdns": false,
"pipedProviders": [
{
"enabled": false,
"id": "OPcan",
"pipeElements": [
{
"options": {
"logging": false,
"subOptions": {
"device": "",
"type": "ngt-1"
},
"type": "NMEA2000"
},
"type": "providers/simple"
}
]
},
{
"enabled": true,
"id": "OPpypilot",
"pipeElements": [
{
"options": {
"logging": false,
"subOptions": {
"host": "localhost",
"port": "20220",
"type": "tcp"
},
"type": "NMEA0183"
},
"type": "providers/simple"
}
]
},
{
"enabled": true,
"id": "OPkplex",
"pipeElements": [
{
"options": {
"logging": false,
"subOptions": {
"host": "localhost",
"port": "30330",
"type": "tcp"
},
"type": "NMEA0183"
},
"type": "providers/simple"
}
]
},
{
"enabled": true,
"id": "OPwifi",
"pipeElements": [
{
"options": {
"logging": false,
"subOptions": {
"port": "55561",
"type": "udp"
},
"type": "SignalK"
},
"type": "providers/simple"
}
]
},
{
"enabled": true,
"id": "OPserial",
"pipeElements": [
{
"options": {
"logging": false,
"subOptions": {
"port": "55559",
"type": "udp"
},
"type": "SignalK"
},
"type": "providers/simple"
}
]
},
{
"enabled": true,
"id": "OPnotifications",
"pipeElements": [
{
"options": {
"logging": false,
"subOptions": {
"port": "55558",
"type": "udp"
},
"type": "SignalK"
},
"type": "providers/simple"
}
]
},
{
"enabled": true,
"id": "OPsensors",
"pipeElements": [
{
"options": {
"logging": false,
"subOptions": {
"port": "55557",
"type": "udp"
},
"type": "SignalK"
},
"type": "providers/simple"
}
]
}
],
"port": 3000,
"security": {
"strategy": "@signalk/sk-simple-token-security"
},
"ssl": false,
Forget any data routing or SK settings for OP v0.x.x.
2018-08-06, 09:13 PM
(2018-08-06, 06:15 PM)Sailoog Wrote: [ -> ]Forget any data routing or SK settings for OP v0.x.x.
Sorry about that and thanks for the clarification. I thought I was linking the latest settings, as the link was to master in Github. Now I see that there are no 1.x tags in Github - have you moved the source someplace else?
A very small change would make the picture a bit clearer: moving the text TCP localhost 30330 next to the kplex box, like 20220 is next to PyPilot. This would make it more clear that kplex is the server, not SK.