This forum uses cookies
This forum makes use of cookies to store your login information if you are registered, and your last visit if you are not. Cookies are small text documents stored on your computer; the cookies set by this forum can only be used on this website and pose no security risk. Cookies on this forum also track the specific topics you have read and when you last read them. Please confirm whether you accept or reject these cookies being set.

A cookie will be stored in your browser regardless of choice to prevent you being asked this question again. You will be able to change your cookie settings at any time using the link in the footer.

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
*newbie* /dev/root full after 24hrs
#21
Just chiming in. I see the same issue with the latest version of OpenPlotter. Stopping SignalK seems to fix it.
Reply
#22
(2018-06-15, 08:29 PM)e-sailing Wrote: I had the same problem run.log "accept failed for connection to signalk: Too many open files"
I think it has to do with kplex.
If I attach a gps to kplex it needs 8 open files all the time.
If I now disconnect the gps and reboot. Every half minute it adds 1 open file untill the limit of the linux system is reached. Then SignalK fires errors until the disk is full.
To test it: Find the pid of kplex
"ps aux | grep kplex"
print out the counted open files of the pid (replace pid with the process number you got from last command) with
"ls /proc/pid/fd/ | wc -l"


What are the open files? ls -l /proc/pid/fd/


Sent from my iPhone using Tapatalk
Reply
#23
(2018-08-05, 02:41 PM)tkurki Wrote:
(2018-06-15, 08:29 PM)e-sailing Wrote: I had the same problem  run.log "accept failed for connection to signalk: Too many open files"
I think it has to do with kplex.
If I attach a gps to kplex it needs 8 open files all the time.
If I now disconnect the gps and reboot. Every half minute it adds 1 open file untill the limit of the linux system is reached. Then SignalK fires errors until the disk is full.
To test it: Find the pid of kplex
"ps aux | grep kplex"
print out the counted open files of the pid (replace pid with the process number you got from last command) with
"ls /proc/pid/fd/ | wc -l"


What are the open files? ls -l /proc/pid/fd/


Sent from my iPhone using Tapatalk
@ tkurki: It looks like this:

Code:
pi@openplotter:~ $ ps aux | grep kplex
pi        4057  0.1  0.0  26724   524 pts/0    Sl+  14:40   0:00 kplex
pi        4143  0.0  0.0   4372   564 pts/1    S+   14:40   0:00 grep --color=auto kplex
pi@openplotter:~ $ ls /proc/4057/fd
0  1  2  3  4  5  6
pi@openplotter:~ $ ls /proc/4057/fd
0  1  2  3  4  5  6  7
pi@openplotter:~ $ ls /proc/4057/fd
0  1  2  3  4  5  6  7
pi@openplotter:~ $ ls /proc/4057/fd
0  1  2  3  4  5  6  7  8
pi@openplotter:~ $ ls /proc/4057/fd
0  1  10  2  3  4  5  6  7  8  9
pi@openplotter:~ $
You see that the number increases with the time.
This is the same issue as http://forum.openmarine.net/showthread.php?tid=1355.
If kplex doesn't get any sentence it can't send anything. SignalK waits ca 30 seconds for information from kplex and then thinks the connection is down and doesn't close it. It just opens a new connection. This does happen until there are ca 1000 open connections.
I can't tell if it is a communication problem with kplex or signalk.
A solution could be to disable kplex.
But if there is a device, which is offline most of the time we can't disable kplex without losing functionality.
Another solution is to send a dummy sentence to keep kplex and signalk happy.
Reply
#24
I am confused - in http://forum.openmarine.net/showthread.p...41#pid6541 data routing is different from the pic in http://forum.openmarine.net/showthread.p...41#pid6541, which claims to be from the documentation??
Reply
#25
I'd like to understand this problem better.

You wrote "SignalK waits ca 30 seconds for information from kplex and then thinks the connection is down and doesn't close it. It just opens a new connection. ". I am not aware of any such logic in SK server. https://github.com/sailoog/openplotter/b...tings.json only has udp connections that have no connection - where is the connection between kplex and sk configured? tcp or udp? Which one is the server and which one the client? The diagrams I found we confusing, see my other post.

Can you run 

Code:
sudo lsof -iP | grep <kplexpid>

that should list all kplex open network connections.

If would be great if people could create issues at https://github.com/SignalK/signalk-server-node/issues if they encounter problems with SK server. So many forums, can't read them all...
Reply
#26
Had to sudo apt-get install lsof, then
Code:
pi@openplotter:~ $ lsof -i
COMMAND     PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
vncserver  1030   pi   10u  IPv4  17556      0t0  TCP localhost:42565->localhost:54068 (ESTABLISHED)
kplex      1424   pi    4u  IPv4  15356      0t0  UDP *:10110
kplex      1424   pi    5u  IPv4  15359      0t0  TCP *:30330 (LISTEN)
kplex      1424   pi    6u  IPv4  19457      0t0  UDP 10.10.10.1:50000
kplex      1424   pi    7u  IPv4  19459      0t0  UDP 10.10.10.1:10111
kplex      1424   pi    9u  IPv4  17334      0t0  TCP 10.10.10.1:30330->10.10.10.1:60504 (ESTABLISHED)
kplex      1424   pi   11u  IPv4  17376      0t0  TCP localhost:30330->localhost:56864 (ESTABLISHED)
signalk-s 13715   pi    3u  IPv6 159745      0t0  TCP *:3000 (LISTEN)

kplex conf >
Code:
[udp]
name=system
direction=in
port=10110

[tcp]
name=signalk
direction=out
mode=server
port=30330

###end of defaults

###OpenPlotter GUI settings

[udp]
name=tablet
direction=in
optional=yes
address=10.10.10.1
port=50000

[udp]
name=nodered
direction=in
optional=yes
address=10.10.10.1
port=10111

[serial]
name=gps
direction=in
optional=yes
filename=/dev/ttyOP_gps
baud=4800

Did some testing , I have a barometer NMEA created in node-red going in to port 10111. This looks like it goes into SigK through TCP30330. 

[Image: meO2EVK.png]


Just turned off the node-red nmea creation, this is where some strange things were happening before and kplex was maxing out a core. I'll do another lsof if and when it happens again. Kplex author is a friend so I'll email him, see if any of this means anything to him.

Thanks for your help!!!!! Smile
Reply
#27
And this after a little while > 



Code:
pi@openplotter:~ $ lsof -i
COMMAND    PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
vncserver 1030   pi   10u  IPv4  17556      0t0  TCP localhost:42565->localhost:54068 (ESTABLISHED)
kplex     3363   pi    4u  IPv4 437300      0t0  UDP *:10110
kplex     3363   pi    5u  IPv4 437303      0t0  TCP *:30330 (LISTEN)
kplex     3363   pi    6u  IPv4 437305      0t0  UDP 10.10.10.1:50000
kplex     3363   pi    7u  IPv4 437307      0t0  UDP 10.10.10.1:10111
kplex     3363   pi    9u  IPv4 437310      0t0  TCP *:10113 (LISTEN)
kplex     3363   pi   10u  IPv4 434794      0t0  TCP 10.10.10.1:30330->10.10.10.1:54944 (CLOSE_WAIT)
kplex     3363   pi   11u  IPv4 434938      0t0  TCP 10.10.10.1:30330->10.10.10.1:55516 (ESTABLISHED)
kplex     3363   pi   13u  IPv4 434811      0t0  TCP 10.10.10.1:30330->10.10.10.1:54946 (CLOSE_WAIT)
kplex     3363   pi   14u  IPv4 434812      0t0  TCP localhost:30330->localhost:43296 (CLOSE_WAIT)
kplex     3363   pi   15u  IPv4 438396      0t0  TCP localhost:30330->localhost:43990 (CLOSE_WAIT)
kplex     3363   pi   16u  IPv4 438597      0t0  TCP localhost:30330->localhost:44032 (CLOSE_WAIT)
kplex     3363   pi   17u  IPv4 438652      0t0  TCP localhost:30330->localhost:44074 (CLOSE_WAIT)
kplex     3363   pi   18u  IPv4 438702      0t0  TCP localhost:30330->localhost:44116 (CLOSE_WAIT)
kplex     3363   pi   19u  IPv4 438751      0t0  TCP localhost:30330->localhost:44158 (CLOSE_WAIT)
kplex     3363   pi   20u  IPv4 438781      0t0  TCP localhost:30330->localhost:44200 (CLOSE_WAIT)
kplex     3363   pi   21u  IPv4 438855      0t0  TCP localhost:30330->localhost:44242 (CLOSE_WAIT)
kplex     3363   pi   22u  IPv4 438879      0t0  TCP localhost:30330->localhost:44284 (CLOSE_WAIT)
kplex     3363   pi   23u  IPv4 438913      0t0  TCP localhost:30330->localhost:44326 (CLOSE_WAIT)
kplex     3363   pi   24u  IPv4 438945      0t0  TCP localhost:30330->localhost:44368 (CLOSE_WAIT)
kplex     3363   pi   25u  IPv4 439005      0t0  TCP localhost:30330->localhost:44410 (CLOSE_WAIT)
kplex     3363   pi   26u  IPv4 439038      0t0  TCP localhost:30330->localhost:44452 (CLOSE_WAIT)
kplex     3363   pi   27u  IPv4 439100      0t0  TCP localhost:30330->localhost:44494 (CLOSE_WAIT)
kplex     3363   pi   28u  IPv4 439148      0t0  TCP localhost:30330->localhost:44536 (CLOSE_WAIT)
kplex     3363   pi   29u  IPv4 439193      0t0  TCP localhost:30330->localhost:44578 (CLOSE_WAIT)
kplex     3363   pi   30u  IPv4 439210      0t0  TCP localhost:30330->localhost:44620 (CLOSE_WAIT)
kplex     3363   pi   31u  IPv4 439265      0t0  TCP localhost:30330->localhost:44662 (CLOSE_WAIT)
kplex     3363   pi   32u  IPv4 439279      0t0  TCP localhost:30330->localhost:44704 (CLOSE_WAIT)
kplex     3363   pi   33u  IPv4 441346      0t0  TCP localhost:30330->localhost:44746 (CLOSE_WAIT)
kplex     3363   pi   34u  IPv4 441354      0t0  TCP localhost:30330->localhost:44788 (CLOSE_WAIT)
kplex     3363   pi   35u  IPv4 441390      0t0  TCP localhost:30330->localhost:44832 (CLOSE_WAIT)
kplex     3363   pi   36u  IPv4 441440      0t0  TCP localhost:30330->localhost:44874 (CLOSE_WAIT)
kplex     3363   pi   37u  IPv4 441480      0t0  TCP localhost:30330->localhost:44916 (CLOSE_WAIT)
kplex     3363   pi   38u  IPv4 441496      0t0  TCP localhost:30330->localhost:44958 (CLOSE_WAIT)
kplex     3363   pi   39u  IPv4 441528      0t0  TCP localhost:30330->localhost:45000 (CLOSE_WAIT)
kplex     3363   pi   40u  IPv4 441848      0t0  TCP localhost:30330->localhost:45042 (CLOSE_WAIT)
kplex     3363   pi   41u  IPv4 441891      0t0  TCP localhost:30330->localhost:45084 (CLOSE_WAIT)
kplex     3363   pi   42u  IPv4 441946      0t0  TCP localhost:30330->localhost:45126 (CLOSE_WAIT)
kplex     3363   pi   43u  IPv4 442005      0t0  TCP localhost:30330->localhost:45168 (CLOSE_WAIT)
kplex     3363   pi   44u  IPv4 442047      0t0  TCP localhost:30330->localhost:45208 (CLOSE_WAIT)
kplex     3363   pi   45u  IPv4 442061      0t0  TCP localhost:30330->localhost:45250 (CLOSE_WAIT)
kplex     3363   pi   46u  IPv4 442091      0t0  TCP localhost:30330->localhost:45292 (CLOSE_WAIT)
kplex     3363   pi   47u  IPv4 442168      0t0  TCP localhost:30330->localhost:45334 (CLOSE_WAIT)
kplex     3363   pi   48u  IPv4 442188      0t0  TCP localhost:30330->localhost:45376 (CLOSE_WAIT)
kplex     3363   pi   49u  IPv4 442237      0t0  TCP localhost:30330->localhost:45418 (CLOSE_WAIT)
kplex     3363   pi   50u  IPv4 442296      0t0  TCP localhost:30330->localhost:45460 (CLOSE_WAIT)
kplex     3363   pi   51u  IPv4 442326      0t0  TCP localhost:30330->localhost:45502 (CLOSE_WAIT)
kplex     3363   pi   52u  IPv4 445441      0t0  TCP localhost:30330->localhost:45544 (CLOSE_WAIT)
kplex     3363   pi   53u  IPv4 445514      0t0  TCP localhost:30330->localhost:45586 (CLOSE_WAIT)
kplex     3363   pi   54u  IPv4 445585      0t0  TCP localhost:30330->localhost:45628 (CLOSE_WAIT)
kplex     3363   pi   55u  IPv4 445621      0t0  TCP localhost:30330->localhost:45670 (CLOSE_WAIT)
kplex     3363   pi   56u  IPv4 445684      0t0  TCP localhost:30330->localhost:45712 (CLOSE_WAIT)
kplex     3363   pi   57u  IPv4 445702      0t0  TCP localhost:30330->localhost:45754 (CLOSE_WAIT)
kplex     3363   pi   58u  IPv4 445748      0t0  TCP localhost:30330->localhost:45796 (CLOSE_WAIT)
kplex     3363   pi   59u  IPv4 445769      0t0  TCP localhost:30330->localhost:45840 (CLOSE_WAIT)
kplex     3363   pi   60u  IPv4 445820      0t0  TCP localhost:30330->localhost:45882 (CLOSE_WAIT)
kplex     3363   pi   61u  IPv4 445903      0t0  TCP localhost:30330->localhost:45924 (CLOSE_WAIT)
kplex     3363   pi   62u  IPv4 445972      0t0  TCP localhost:30330->localhost:45966 (CLOSE_WAIT)
kplex     3363   pi   63u  IPv4 445999      0t0  TCP localhost:30330->localhost:46008 (CLOSE_WAIT)
kplex     3363   pi   64u  IPv4 446037      0t0  TCP localhost:30330->localhost:46050 (CLOSE_WAIT)
kplex     3363   pi   65u  IPv4 446116      0t0  TCP localhost:30330->localhost:46092 (CLOSE_WAIT)
kplex     3363   pi   66u  IPv4 446157      0t0  TCP localhost:30330->localhost:46134 (CLOSE_WAIT)
kplex     3363   pi   67u  IPv4 446188      0t0  TCP localhost:30330->localhost:46176 (CLOSE_WAIT)
kplex     3363   pi   68u  IPv4 446209      0t0  TCP localhost:30330->localhost:46218 (CLOSE_WAIT)
kplex     3363   pi   69u  IPv4 446257      0t0  TCP localhost:30330->localhost:46260 (CLOSE_WAIT)
kplex     3363   pi   70u  IPv4 446267      0t0  TCP localhost:30330->localhost:46302 (CLOSE_WAIT)
kplex     3363   pi   71u  IPv4 446283      0t0  TCP localhost:30330->localhost:46344 (CLOSE_WAIT)
kplex     3363   pi   72u  IPv4 446305      0t0  TCP localhost:30330->localhost:46386 (CLOSE_WAIT)
kplex     3363   pi   73u  IPv4 446367      0t0  TCP localhost:30330->localhost:46428 (CLOSE_WAIT)
kplex     3363   pi   74u  IPv4 445135      0t0  TCP localhost:30330->localhost:46470 (CLOSE_WAIT)
kplex     3363   pi   75u  IPv4 445172      0t0  TCP localhost:30330->localhost:46512 (CLOSE_WAIT)
kplex     3363   pi   76u  IPv4 445210      0t0  TCP localhost:30330->localhost:46554 (CLOSE_WAIT)
kplex     3363   pi   77u  IPv4 445220      0t0  TCP localhost:30330->localhost:46596 (CLOSE_WAIT)
kplex     3363   pi   78u  IPv4 445268      0t0  TCP localhost:30330->localhost:46638 (CLOSE_WAIT)
kplex     3363   pi   79u  IPv4 445277      0t0  TCP localhost:30330->localhost:46680 (CLOSE_WAIT)
kplex     3363   pi   80u  IPv4 445307      0t0  TCP localhost:30330->localhost:46722 (CLOSE_WAIT)
kplex     3363   pi   81u  IPv4 445366      0t0  TCP localhost:30330->localhost:46764 (CLOSE_WAIT)
kplex     3363   pi   82u  IPv4 445394      0t0  TCP localhost:30330->localhost:46806 (CLOSE_WAIT)
kplex     3363   pi   83u  IPv4 445395      0t0  TCP localhost:30330->localhost:46850 (CLOSE_WAIT)
kplex     3363   pi   84u  IPv4 445406      0t0  TCP localhost:30330->localhost:46892 (CLOSE_WAIT)
kplex     3363   pi   85u  IPv4 445420      0t0  TCP localhost:30330->localhost:46934 (CLOSE_WAIT)
kplex     3363   pi   86u  IPv4 450607      0t0  TCP localhost:30330->localhost:46976 (CLOSE_WAIT)
kplex     3363   pi   87u  IPv4 450641      0t0  TCP localhost:30330->localhost:47018 (CLOSE_WAIT)
kplex     3363   pi   88u  IPv4 450684      0t0  TCP localhost:30330->localhost:47060 (CLOSE_WAIT)
kplex     3363   pi   89u  IPv4 450723      0t0  TCP localhost:30330->localhost:47102 (CLOSE_WAIT)
kplex     3363   pi   90u  IPv4 450783      0t0  TCP localhost:30330->localhost:47144 (CLOSE_WAIT)
kplex     3363   pi   91u  IPv4 450828      0t0  TCP localhost:30330->localhost:47186 (CLOSE_WAIT)
kplex     3363   pi   92u  IPv4 450874      0t0  TCP localhost:30330->localhost:47228 (CLOSE_WAIT)
kplex     3363   pi   93u  IPv4 450922      0t0  TCP localhost:30330->localhost:47270 (CLOSE_WAIT)
kplex     3363   pi   94u  IPv4 450931      0t0  TCP localhost:30330->localhost:47312 (CLOSE_WAIT)
kplex     3363   pi   95u  IPv4 450945      0t0  TCP localhost:30330->localhost:47354 (CLOSE_WAIT)
kplex     3363   pi   96u  IPv4 450967      0t0  TCP localhost:30330->localhost:47396 (CLOSE_WAIT)
kplex     3363   pi   97u  IPv4 451015      0t0  TCP localhost:30330->localhost:47436 (CLOSE_WAIT)
kplex     3363   pi   98u  IPv4 451076      0t0  TCP localhost:30330->localhost:47478 (CLOSE_WAIT)
kplex     3363   pi   99u  IPv4 451085      0t0  TCP localhost:30330->localhost:47520 (CLOSE_WAIT)
kplex     3363   pi  100u  IPv4 451140      0t0  TCP localhost:30330->localhost:47562 (CLOSE_WAIT)
kplex     3363   pi  101u  IPv4 451157      0t0  TCP localhost:30330->localhost:47604 (CLOSE_WAIT)
kplex     3363   pi  102u  IPv4 451176      0t0  TCP localhost:30330->localhost:47646 (CLOSE_WAIT)
kplex     3363   pi  103u  IPv4 451208      0t0  TCP localhost:30330->localhost:47688 (CLOSE_WAIT)
kplex     3363   pi  104u  IPv4 451236      0t0  TCP localhost:30330->localhost:47730 (CLOSE_WAIT)
kplex     3363   pi  105u  IPv4 451295      0t0  TCP localhost:30330->localhost:47772 (CLOSE_WAIT)
kplex     3363   pi  106u  IPv4 451334      0t0  TCP localhost:30330->localhost:47814 (CLOSE_WAIT)
kplex     3363   pi  107u  IPv4 451384      0t0  TCP localhost:30330->localhost:47858 (CLOSE_WAIT)
kplex     3363   pi  108u  IPv4 449263      0t0  TCP localhost:30330->localhost:47900 (CLOSE_WAIT)
kplex     3363   pi  109u  IPv4 449274      0t0  TCP localhost:30330->localhost:47942 (CLOSE_WAIT)
kplex     3363   pi  110u  IPv4 449283      0t0  TCP localhost:30330->localhost:47984 (CLOSE_WAIT)
kplex     3363   pi  111u  IPv4 449324      0t0  TCP localhost:30330->localhost:48026 (CLOSE_WAIT)
kplex     3363   pi  112u  IPv4 449398      0t0  TCP localhost:30330->localhost:48068 (CLOSE_WAIT)
kplex     3363   pi  113u  IPv4 449470      0t0  TCP localhost:30330->localhost:48110 (CLOSE_WAIT)
kplex     3363   pi  114u  IPv4 449512      0t0  TCP localhost:30330->localhost:48152 (CLOSE_WAIT)
kplex     3363   pi  115u  IPv4 454733      0t0  TCP localhost:30330->localhost:48194 (CLOSE_WAIT)
kplex     3363   pi  116u  IPv4 454803      0t0  TCP localhost:30330->localhost:48236 (CLOSE_WAIT)
kplex     3363   pi  117u  IPv4 454836      0t0  TCP localhost:30330->localhost:48278 (CLOSE_WAIT)
kplex     3363   pi  118u  IPv4 454879      0t0  TCP localhost:30330->localhost:48320 (CLOSE_WAIT)
kplex     3363   pi  119u  IPv4 454952      0t0  TCP localhost:30330->localhost:48362 (CLOSE_WAIT)
kplex     3363   pi  120u  IPv4 455027      0t0  TCP localhost:30330->localhost:48404 (CLOSE_WAIT)
kplex     3363   pi  121u  IPv4 455046      0t0  TCP localhost:30330->localhost:48446 (CLOSE_WAIT)
kplex     3363   pi  122u  IPv4 455092      0t0  TCP localhost:30330->localhost:48488 (CLOSE_WAIT)
kplex     3363   pi  123u  IPv4 455129      0t0  TCP localhost:30330->localhost:48530 (CLOSE_WAIT)
kplex     3363   pi  124u  IPv4 455206      0t0  TCP localhost:30330->localhost:48572 (CLOSE_WAIT)
kplex     3363   pi  125u  IPv4 455248      0t0  TCP localhost:30330->localhost:48614 (CLOSE_WAIT)
kplex     3363   pi  126u  IPv4 455301      0t0  TCP localhost:30330->localhost:48656 (CLOSE_WAIT)
kplex     3363   pi  127u  IPv4 455347      0t0  TCP localhost:30330->localhost:48698 (CLOSE_WAIT)
kplex     3363   pi  128u  IPv4 455385      0t0  TCP localhost:30330->localhost:48740 (CLOSE_WAIT)
kplex     3363   pi  129u  IPv4 455464      0t0  TCP localhost:30330->localhost:48782 (CLOSE_WAIT)
kplex     3363   pi  130u  IPv4 455502      0t0  TCP localhost:30330->localhost:48824 (CLOSE_WAIT)
kplex     3363   pi  131u  IPv4 455525      0t0  TCP localhost:30330->localhost:48868 (CLOSE_WAIT)
kplex     3363   pi  132u  IPv4 455604      0t0  TCP localhost:30330->localhost:48910 (CLOSE_WAIT)
kplex     3363   pi  133u  IPv4 457734      0t0  TCP localhost:30330->localhost:48952 (CLOSE_WAIT)
kplex     3363   pi  134u  IPv4 457757      0t0  TCP localhost:30330->localhost:48994 (CLOSE_WAIT)
kplex     3363   pi  135u  IPv4 457793      0t0  TCP localhost:30330->localhost:49036 (CLOSE_WAIT)
kplex     3363   pi  136u  IPv4 457886      0t0  TCP localhost:30330->localhost:49078 (CLOSE_WAIT)
kplex     3363   pi  137u  IPv4 457922      0t0  TCP localhost:30330->localhost:49120 (CLOSE_WAIT)
kplex     3363   pi  138u  IPv4 457947      0t0  TCP localhost:30330->localhost:49162 (CLOSE_WAIT)
kplex     3363   pi  139u  IPv4 458002      0t0  TCP localhost:30330->localhost:49204 (CLOSE_WAIT)
kplex     3363   pi  140u  IPv4 458073      0t0  TCP localhost:30330->localhost:49246 (CLOSE_WAIT)
kplex     3363   pi  141u  IPv4 458113      0t0  TCP localhost:30330->localhost:49288 (CLOSE_WAIT)
kplex     3363   pi  142u  IPv4 458161      0t0  TCP localhost:30330->localhost:49330 (CLOSE_WAIT)
kplex     3363   pi  143u  IPv4 458197      0t0  TCP localhost:30330->localhost:49372 (CLOSE_WAIT)
kplex     3363   pi  144u  IPv4 458251      0t0  TCP localhost:30330->localhost:49414 (CLOSE_WAIT)
kplex     3363   pi  145u  IPv4 458299      0t0  TCP localhost:30330->localhost:49456 (CLOSE_WAIT)
kplex     3363   pi  146u  IPv4 458325      0t0  TCP localhost:30330->localhost:49498 (CLOSE_WAIT)
kplex     3363   pi  147u  IPv4 458399      0t0  TCP localhost:30330->localhost:49540 (CLOSE_WAIT)
kplex     3363   pi  148u  IPv4 458415      0t0  TCP localhost:30330->localhost:49582 (CLOSE_WAIT)
kplex     3363   pi  149u  IPv4 458430      0t0  TCP localhost:30330->localhost:49624 (CLOSE_WAIT)
kplex     3363   pi  150u  IPv4 458449      0t0  TCP localhost:30330->localhost:49666 (CLOSE_WAIT)
kplex     3363   pi  151u  IPv4 458498      0t0  TCP localhost:30330->localhost:49708 (CLOSE_WAIT)
kplex     3363   pi  152u  IPv4 458529      0t0  TCP localhost:30330->localhost:49750 (CLOSE_WAIT)
kplex     3363   pi  153u  IPv4 458575      0t0  TCP localhost:30330->localhost:49792 (CLOSE_WAIT)
kplex     3363   pi  154u  IPv4 458620      0t0  TCP localhost:30330->localhost:49834 (CLOSE_WAIT)
kplex     3363   pi  155u  IPv4 458662      0t0  TCP localhost:30330->localhost:49878 (CLOSE_WAIT)
kplex     3363   pi  156u  IPv4 458673      0t0  TCP localhost:30330->localhost:49920 (CLOSE_WAIT)
kplex     3363   pi  157u  IPv4 458692      0t0  TCP localhost:30330->localhost:49962 (CLOSE_WAIT)
kplex     3363   pi  158u  IPv4 461835      0t0  TCP localhost:30330->localhost:50004 (CLOSE_WAIT)
kplex     3363   pi  159u  IPv4 461868      0t0  TCP localhost:30330->localhost:50046 (CLOSE_WAIT)
kplex     3363   pi  160u  IPv4 461969      0t0  TCP localhost:30330->localhost:50088 (CLOSE_WAIT)
kplex     3363   pi  161u  IPv4 461988      0t0  TCP localhost:30330->localhost:50130 (CLOSE_WAIT)
kplex     3363   pi  162u  IPv4 462001      0t0  TCP localhost:30330->localhost:50172 (CLOSE_WAIT)
kplex     3363   pi  163u  IPv4 462022      0t0  TCP localhost:30330->localhost:50214 (CLOSE_WAIT)
kplex     3363   pi  164u  IPv4 462042      0t0  TCP localhost:30330->localhost:50256 (CLOSE_WAIT)
kplex     3363   pi  165u  IPv4 462076      0t0  TCP localhost:30330->localhost:50298 (CLOSE_WAIT)
kplex     3363   pi  166u  IPv4 462098      0t0  TCP localhost:30330->localhost:50340 (CLOSE_WAIT)
kplex     3363   pi  167u  IPv4 462172      0t0  TCP localhost:30330->localhost:50382 (CLOSE_WAIT)
kplex     3363   pi  168u  IPv4 462201      0t0  TCP localhost:30330->localhost:50424 (CLOSE_WAIT)
kplex     3363   pi  169u  IPv4 462256      0t0  TCP localhost:30330->localhost:50464 (CLOSE_WAIT)
kplex     3363   pi  170u  IPv4 462276      0t0  TCP localhost:30330->localhost:50506 (CLOSE_WAIT)
kplex     3363   pi  171u  IPv4 462344      0t0  TCP localhost:30330->localhost:50548 (CLOSE_WAIT)
kplex     3363   pi  172u  IPv4 462390      0t0  TCP localhost:30330->localhost:50590 (CLOSE_WAIT)
kplex     3363   pi  173u  IPv4 462409      0t0  TCP localhost:30330->localhost:50632 (CLOSE_WAIT)
kplex     3363   pi  174u  IPv4 462454      0t0  TCP localhost:30330->localhost:50674 (CLOSE_WAIT)
kplex     3363   pi  175u  IPv4 462513      0t0  TCP localhost:30330->localhost:50716 (CLOSE_WAIT)
kplex     3363   pi  176u  IPv4 462551      0t0  TCP localhost:30330->localhost:50758 (CLOSE_WAIT)
kplex     3363   pi  177u  IPv4 462584      0t0  TCP localhost:30330->localhost:50800 (CLOSE_WAIT)
kplex     3363   pi  178u  IPv4 462613      0t0  TCP localhost:30330->localhost:50842 (CLOSE_WAIT)
kplex     3363   pi  179u  IPv4 462686      0t0  TCP localhost:30330->localhost:50886 (CLOSE_WAIT)
kplex     3363   pi  180u  IPv4 462786      0t0  TCP localhost:30330->localhost:50928 (CLOSE_WAIT)
kplex     3363   pi  181u  IPv4 462837      0t0  TCP localhost:30330->localhost:50970 (CLOSE_WAIT)
kplex     3363   pi  182u  IPv4 464970      0t0  TCP localhost:30330->localhost:51012 (CLOSE_WAIT)
kplex     3363   pi  183u  IPv4 465012      0t0  TCP localhost:30330->localhost:51054 (CLOSE_WAIT)
kplex     3363   pi  184u  IPv4 465042      0t0  TCP localhost:30330->localhost:51096 (CLOSE_WAIT)
kplex     3363   pi  185u  IPv4 465087      0t0  TCP localhost:30330->localhost:51138 (CLOSE_WAIT)
kplex     3363   pi  186u  IPv4 465101      0t0  TCP localhost:30330->localhost:51180 (CLOSE_WAIT)
kplex     3363   pi  187u  IPv4 465122      0t0  TCP localhost:30330->localhost:51222 (CLOSE_WAIT)
kplex     3363   pi  188u  IPv4 465127      0t0  TCP localhost:30330->localhost:51264 (CLOSE_WAIT)
kplex     3363   pi  189u  IPv4 465142      0t0  TCP localhost:30330->localhost:51306 (CLOSE_WAIT)
kplex     3363   pi  190u  IPv4 465154      0t0  TCP localhost:30330->localhost:51348 (CLOSE_WAIT)
kplex     3363   pi  191u  IPv4 465182      0t0  TCP localhost:30330->localhost:51390 (CLOSE_WAIT)
kplex     3363   pi  192u  IPv4 465193      0t0  TCP localhost:30330->localhost:51432 (CLOSE_WAIT)
kplex     3363   pi  193u  IPv4 465207      0t0  TCP localhost:30330->localhost:51474 (CLOSE_WAIT)
kplex     3363   pi  194u  IPv4 465214      0t0  TCP localhost:30330->localhost:51516 (CLOSE_WAIT)
kplex     3363   pi  195u  IPv4 465240      0t0  TCP localhost:30330->localhost:51558 (CLOSE_WAIT)
kplex     3363   pi  196u  IPv4 465254      0t0  TCP localhost:30330->localhost:51600 (CLOSE_WAIT)
kplex     3363   pi  197u  IPv4 465269      0t0  TCP localhost:30330->localhost:51642 (CLOSE_WAIT)
kplex     3363   pi  198u  IPv4 465289      0t0  TCP localhost:30330->localhost:51684 (CLOSE_WAIT)
kplex     3363   pi  199u  IPv4 465343      0t0  TCP localhost:30330->localhost:51726 (CLOSE_WAIT)
kplex     3363   pi  200u  IPv4 465405      0t0  TCP localhost:30330->localhost:51768 (CLOSE_WAIT)
kplex     3363   pi  201u  IPv4 465427      0t0  TCP localhost:30330->localhost:51810 (CLOSE_WAIT)
kplex     3363   pi  202u  IPv4 465434      0t0  TCP localhost:30330->localhost:51852 (CLOSE_WAIT)
kplex     3363   pi  203u  IPv4 465484      0t0  TCP localhost:30330->localhost:51896 (CLOSE_WAIT)
kplex     3363   pi  204u  IPv4 465560      0t0  TCP localhost:30330->localhost:51938 (CLOSE_WAIT)
kplex     3363   pi  205u  IPv4 465649      0t0  TCP localhost:30330->localhost:51980 (CLOSE_WAIT)
kplex     3363   pi  206u  IPv4 465710      0t0  TCP localhost:30330->localhost:52022 (CLOSE_WAIT)
kplex     3363   pi  207u  IPv4 465753      0t0  TCP localhost:30330->localhost:52064 (CLOSE_WAIT)
kplex     3363   pi  208u  IPv4 465801      0t0  TCP localhost:30330->localhost:52106 (CLOSE_WAIT)
kplex     3363   pi  209u  IPv4 465817      0t0  TCP localhost:30330->localhost:52148 (CLOSE_WAIT)
kplex     3363   pi  210u  IPv4 465844      0t0  TCP localhost:30330->localhost:52190 (CLOSE_WAIT)
kplex     3363   pi  211u  IPv4 465853      0t0  TCP localhost:30330->localhost:52232 (CLOSE_WAIT)
kplex     3363   pi  212u  IPv4 465858      0t0  TCP localhost:30330->localhost:52274 (CLOSE_WAIT)
kplex     3363   pi  213u  IPv4 465882      0t0  TCP localhost:30330->localhost:52316 (CLOSE_WAIT)
kplex     3363   pi  214u  IPv4 465909      0t0  TCP localhost:30330->localhost:52358 (CLOSE_WAIT)
kplex     3363   pi  215u  IPv4 470031      0t0  TCP localhost:30330->localhost:52400 (CLOSE_WAIT)
kplex     3363   pi  216u  IPv4 470081      0t0  TCP localhost:30330->localhost:52442 (CLOSE_WAIT)
kplex     3363   pi  217u  IPv4 470103      0t0  TCP localhost:30330->localhost:52484 (CLOSE_WAIT)
kplex     3363   pi  218u  IPv4 470693      0t0  TCP localhost:30330->localhost:52528 (ESTABLISHED)
signalk-s 3422   pi    3u  IPv6 437383      0t0  TCP *:3000 (LISTEN)
pi@openplotter:~

Turn the node red send nmea to kplex again and back to  this >
Code:
pi@openplotter:~ $ lsof -i
COMMAND    PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
vncserver 1030   pi   10u  IPv4  17556      0t0  TCP localhost:42565->localhost:54068 (ESTABLISHED)
kplex     3363   pi    4u  IPv4 437300      0t0  UDP *:10110
kplex     3363   pi    5u  IPv4 437303      0t0  TCP *:30330 (LISTEN)
kplex     3363   pi    6u  IPv4 437305      0t0  UDP 10.10.10.1:50000
kplex     3363   pi    7u  IPv4 437307      0t0  UDP 10.10.10.1:10111
kplex     3363   pi    9u  IPv4 437310      0t0  TCP *:10113 (LISTEN)
kplex     3363   pi  222u  IPv4 470871      0t0  TCP 10.10.10.1:30330->10.10.10.1:36160 (ESTABLISHED)
kplex     3363   pi  223u  IPv4 470912      0t0  TCP localhost:30330->localhost:52706 (ESTABLISHED)
signalk-s 3422   pi    3u  IPv6 437383      0t0  TCP *:3000 (LISTEN)
pi@openplotter:~ $
Reply
#28
Could you try once more with 

Code:
sudo lsof -i | grep 30330


when the problem is occurring?

What I am looking for is the other end of the single active connection, which is not included in the listing with all the CLOSE_WAIT connections that you sent:

Code:
kplex     3363   pi   11u  IPv4 434938      0t0  TCP 10.10.10.1:30330->10.10.10.1:55516 (ESTABLISHED)

My guess is that lsof is not listing all network connections without sudo.

What I am looking for is something like this:

Code:
lsof -i | grep 30330
kplex     8862  tjk    6u  IPv6 0x1b4146e9b77ce467      0t0  TCP *:30330 (LISTEN)
kplex     8862  tjk    7u  IPv6 0x1b4146e9af129447      0t0  TCP localhost:30330->localhost:56510 (ESTABLISHED)
nc        8863  tjk    5u  IPv6 0x1b4146e9b69ff987      0t0  TCP localhost:56510->localhost:30330 (ESTABLISHED)

You can see kplex server listening for new connections and then one established connection from application nc.

As far as I understand this kplex is not handling connections that are closed by a client properly but keeps the file descriptor open with the connection in CLOSE_WAIT state - when there is no data being processed. From what you are saying it closes the connections properly if there is data flowing.


What i don't understand is that in your listings signalk-server does not have any open connections: it is only listening for new http connections on port 3000.

(2018-08-05, 08:16 PM)tkurki Wrote: I am confused - in http://forum.openmarine.net/showthread.p...41#pid6541 data routing is different from the pic in http://forum.openmarine.net/showthread.p...41#pid6541, which claims to be from the documentation??

Ooops, I meant

https://sailoog.gitbooks.io/openplotter-..._work.html

and the picture in 

http://forum.openmarine.net/showthread.p...41#pid6541
Reply
#29
Just to clarify some background on this issue for OpenPlotter v1.x.x

This is the doc for v1.x.x and this is the data routing: https://docs.sailoog.com/openplotter-v1-...es-it-work


This is the SK server settings for  v1.x.x:


Code:
{
   "enablePluginLogging": true,
   "interfaces": {
       "appstore": true,
       "nmea-tcp": true,
       "plugins": true,
       "providers": true,
       "rest": true,
       "tcp": true,
       "webapps": true,
       "ws": true
   },
   "loggingDirectory": "/home/pi/Logs",
   "mdns": false,
   "pipedProviders": [
       {
           "enabled": false,
           "id": "OPcan",
           "pipeElements": [
               {
                   "options": {
                       "logging": false,
                       "subOptions": {
                           "device": "",
                           "type": "ngt-1"
                       },
                       "type": "NMEA2000"
                   },
                   "type": "providers/simple"
               }
           ]
       },
       {
           "enabled": true,
           "id": "OPpypilot",
           "pipeElements": [
               {
                   "options": {
                       "logging": false,
                       "subOptions": {
                           "host": "localhost",
                           "port": "20220",
                           "type": "tcp"
                       },
                       "type": "NMEA0183"
                   },
                   "type": "providers/simple"
               }
           ]
       },
       {
           "enabled": true,
           "id": "OPkplex",
           "pipeElements": [
               {
                   "options": {
                       "logging": false,
                       "subOptions": {
                           "host": "localhost",
                           "port": "30330",
                           "type": "tcp"
                       },
                       "type": "NMEA0183"
                   },
                   "type": "providers/simple"
               }
           ]
       },
       {
           "enabled": true,
           "id": "OPwifi",
           "pipeElements": [
               {
                   "options": {
                       "logging": false,
                       "subOptions": {
                           "port": "55561",
                           "type": "udp"
                       },
                       "type": "SignalK"
                   },
                   "type": "providers/simple"
               }
           ]
       },
       {
           "enabled": true,
           "id": "OPserial",
           "pipeElements": [
               {
                   "options": {
                       "logging": false,
                       "subOptions": {
                           "port": "55559",
                           "type": "udp"
                       },
                       "type": "SignalK"
                   },
                   "type": "providers/simple"
               }
           ]
       },
       {
           "enabled": true,
           "id": "OPnotifications",
           "pipeElements": [
               {
                   "options": {
                       "logging": false,
                       "subOptions": {
                           "port": "55558",
                           "type": "udp"
                       },
                       "type": "SignalK"
                   },
                   "type": "providers/simple"
               }
           ]
       },
       {
           "enabled": true,
           "id": "OPsensors",
           "pipeElements": [
               {
                   "options": {
                       "logging": false,
                       "subOptions": {
                           "port": "55557",
                           "type": "udp"
                       },
                       "type": "SignalK"
                   },
                   "type": "providers/simple"
               }
           ]
       }
   ],
   "port": 3000,
   "security": {
       "strategy": "@signalk/sk-simple-token-security"
   },
   "ssl": false,

Forget any data routing or SK settings for OP v0.x.x.
Reply
#30
(2018-08-06, 06:15 PM)Sailoog Wrote: Forget any data routing or SK settings for OP v0.x.x.

Sorry about that and thanks for the clarification. I thought I was linking the latest settings, as the link was to master in Github. Now I see that there are no 1.x tags in Github - have you moved the source someplace else?

A very small change would make the picture a bit clearer: moving the text TCP localhost 30330 next to the kplex box, like 20220 is next to PyPilot. This would make it more clear that kplex is the server, not SK.
Reply


Forum Jump:


Users browsing this thread: 3 Guest(s)