OpenMarine

Full Version: Checksums on incoming NMEA strings
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
There has been a little discussion on NMEA checksums over the past few months, but I don't think this one has been discussed in detail:

I have an old AutoHelm ST-60 with an NMEA output. It puts out DBT and HDM strings. I have trouble integrating it with other stuff. Depth randomly shows up and I don't think HDM ever does. I can actually see the strings coming in to OpenPlotter, but they don't seem to get used anywhere.

I think this may be due to the lack of checksums, and in the case of DBT strings, the presence of ONLY feet measurement. My thought was that I could create a couple of named pipes in the filesystem, pass the strings into one, manipulate them to add checksum and the other depth measurements, then pass the result to another named pipe that goes into KPLEX/OpenPlotter.

I've written the python code to do the string manipulation and it works great on a stored file of NMEA data that I received from the ST-60. It will happily take the strings, add the appropriate depth measurements to the DBT strings, then output both the DBT and HDM strings with checksums. Where I get stuck is how to get this data from KPLEX, and how to pass the data back to KPEX. I can't seem to get the named pipes to work as expected. The only other way I can think to do it is to perhaps create a small ramdisk and read from a file there, and write to another file, occasionally flushing the data stored there so as not to overrun the disk.

Have I taken myself down a rabbit hole here? Is there some other magic that I'm missing in Node-RED or KPLEX configuration that will do what I need to do? Am I wrong about the need for checksums and the other depth metrics in the strings I'm getting?

Your thoughts would be most appreciated.
This will not be any help but from what I can tell there is nothing that checks the validity of incoming data. My Raymarine chart plotter seems to send corrupted NMEA strings at totally random times - sometimes immediately and sometimes not during 24 hours. Kplex seems to be happy to interpret the corrupted string and eventually the corrupted data will crash SignalK.
I hope you can resolve your issue somehow because your solution might also be used to filter out bad data in my setup.
(2017-06-27, 08:49 PM)abarrow Wrote: [ -> ]Have I taken myself down a rabbit hole here? Is there some other magic that I'm missing in Node-RED or KPLEX configuration that will do what I need to do? Am I wrong about the need for checksums and the other depth metrics in the strings I'm getting?

Your thoughts would be most appreciated.

Node red might be an easier route - how does the nmea data get into the Pi from the ST60? 

You might be able to tweak this flow to do something usedful. 

This gets nmea data from openplotter port 10109 (opencpn output), then filters for just OCMDA messages then it changes the talker ID to WI - $WIMDA, afte rthat it strips the starting  '$' and also strips the checksum, then creates a new checksum and linefeed. Then it's output to UDP port 10108, Openplotter has a new UDP input port 10108 added in the nmea tab to get the data back into openplotter/kplex . (All this is because the konni logbook only seems to accept WIMDA sentences to autofill new log entries with pressure).
You might be able to fiddle with this changing the filter or  with a serial input node to do something useful. 

HTH





[Image: t9JkR9c.png]




Code:
[
   {
       "id": "8821724b.6009d",
       "type": "debug",
       "z": "f2a52aa4.57c488",
       "name": "",
       "active": false,
       "console": "false",
       "complete": "false",
       "x": 765.0000495910645,
       "y": 1570.0000495910645,
       "wires": []
   },
   {
       "id": "5fdabb0b.436d14",
       "type": "switch",
       "z": "f2a52aa4.57c488",
       "name": "Filter just pressure",
       "property": "payload",
       "propertyType": "msg",
       "rules": [
           {
               "t": "regex",
               "v": "^(?=.*OCMDA)",
               "vt": "str",
               "case": false
           }
       ],
       "checkall": "true",
       "outputs": 1,
       "x": 230,
       "y": 1580,
       "wires": [
           [
               "163e594f.d46a67",
               "37b1c3cb.b714ec"
           ]
       ]
   },
   {
       "id": "37b1c3cb.b714ec",
       "type": "function",
       "z": "f2a52aa4.57c488",
       "name": " '$OC\" to\"$WI\"",
       "func": "var nmeaFull=msg.payload;\n\nvar nmea=nmeaFull.substring(6 );     //\nnmea = '$WIMDA'  +  nmea;\n\n\nmsg.payload = nmea;\nreturn msg;",
       "outputs": 1,
       "noerr": 0,
       "x": 337.0000305175781,
       "y": 1516.9999647140503,
       "wires": [
           [
               "264b1a9d.6a8576"
           ]
       ]
   },
   {
       "id": "aae69ba7.ffae88",
       "type": "function",
       "z": "f2a52aa4.57c488",
       "name": "calc checksum",
       "func": "var nmea = msg.payload;\nvar checksum = 0; \n\nfor(var i = 0; i < nmea.length; i++) { \n  checksum = checksum ^ nmea.charCodeAt(i); \n}\nchecksum = checksum.toString(16);    //convert to hex\nnmea = '$' + nmea + '*' + checksum;  //make the full nmea sentence again\n\nmessage = 'Full message =' + nmea + '\\r\\n' + 'Checksum = ' + checksum ;\n\nmsg.payload = nmea;\nmsg.payload = msg.payload+\"\\r\\n\"\n\n\nreturn msg;",
       "outputs": 1,
       "noerr": 0,
       "x": 572.0000305175781,
       "y": 1512.0000495910645,
       "wires": [
           [
               "8821724b.6009d",
               "dbb55dc2.373dd"
           ]
       ]
   },
   {
       "id": "264b1a9d.6a8576",
       "type": "function",
       "z": "f2a52aa4.57c488",
       "name": "Strip front and back",
       "func": "var nmeaFull=msg.payload;\n\nvar nmeaStripped=nmeaFull.substring(1, nmeaFull.indexOf(\"*\"));     //cut of the leading \"$\" and the bad checksum\nmsg.payload = nmeaStripped;\nreturn msg;",
       "outputs": 1,
       "noerr": 0,
       "x": 467.0000991821289,
       "y": 1568.0000457763672,
       "wires": [
           [
               "aae69ba7.ffae88"
           ]
       ]
   },
   {
       "id": "dbb55dc2.373dd",
       "type": "udp out",
       "z": "f2a52aa4.57c488",
       "name": "",
       "addr": "127.0.0.1",
       "iface": "",
       "port": "10108",
       "ipv": "udp4",
       "outport": "",
       "base64": false,
       "multicast": "false",
       "x": 795.0000915527344,
       "y": 1517.000072479248,
       "wires": []
   },
   {
       "id": "3ad88f03.0c91a",
       "type": "tcp in",
       "z": "f2a52aa4.57c488",
       "name": "",
       "server": "client",
       "host": "10.10.10.1",
       "port": "10109",
       "datamode": "stream",
       "datatype": "utf8",
       "newline": "",
       "topic": "",
       "base64": false,
       "x": 100,
       "y": 1520,
       "wires": [
           [
               "5fdabb0b.436d14"
           ]
       ]
   },
   {
       "id": "cda80705.74ea58",
       "type": "comment",
       "z": "f2a52aa4.57c488",
       "name": "This flow creats the WIMNA sentance required by Opencpn logbook to create a pressure entry",
       "info": "",
       "x": 480,
       "y": 1460,
       "wires": []
   },
   {
       "id": "163e594f.d46a67",
       "type": "debug",
       "z": "f2a52aa4.57c488",
       "name": "",
       "active": false,
       "console": "false",
       "complete": "false",
       "x": 460.00001525878906,
       "y": 1608.000051498413,
       "wires": []
   }
]
I like this a lot! I'll have a look to see if I can modify it. I know the data is flowing into Openplotter via RS485, as I can see it in a diagnostic. If I can then manipulate it and put it out via a TCP port, Bob's your uncle!

Sincere thanks. I'll get to work.
Well, that was fun! Thanks for the help. It looks like I have something that works. 

I'll know when I get a chance to take it down to the boat, but using simulated strings in Node-Red is works great!

Here's the code:
Code:
[{"id":"def155ce.cb8dc8","type":"tab","label":"Flow 2"},{"id":"4fc0cc12.46de24","type":"switch","z":"def155ce.cb8dc8","name":"Filter just DBT and HDM","property":"payload","propertyType":"msg","rules":[{"t":"regex","v":"^(?=.*IIDBT)","vt":"str","case":false},{"t":"regex","v":"^(?=.*IIHDM)","vt":"str","case":false}],"checkall":"true","outputs":2,"x":443.99993896484375,"y":367.99993896484375,"wires":[["d06bc454.854278"],["142750e6.16540f"]]},{"id":"de39de6.89c9d2","type":"function","z":"def155ce.cb8dc8","name":"calc checksum","func":"var nmea = msg.payload;\nvar checksum = 0; \n\nfor(var i = 0; i < nmea.length; i++) { \n  checksum = checksum ^ nmea.charCodeAt(i); \n}\nchecksum = checksum.toString(16);    //convert to hex\nnmea = '$' + nmea + '*' + checksum;  //make the full nmea sentence again\n\nmessage = 'Full message =' + nmea + '\\r\\n' + 'Checksum = ' + checksum ;\n\nmsg.payload = nmea;\nmsg.payload = msg.payload+\"\\r\\n\";\n\nvar msg1 = {\n     payload: msg.payload\n };\n return msg1;","outputs":1,"noerr":0,"x":1018,"y":423,"wires":[["6fe378e5.112f58","bc1148e9.02e848"]]},{"id":"142750e6.16540f","type":"function","z":"def155ce.cb8dc8","name":"Strip front and back","func":"var nmeaFull=msg.payload;\n\n    \nvar nmeaStripped=nmeaFull.substring(1, 99)\n\nmsg.payload = nmeaStripped;\nreturn msg;","outputs":1,"noerr":0,"x":802.0000610351562,"y":423,"wires":[["de39de6.89c9d2"]]},{"id":"6fe378e5.112f58","type":"udp out","z":"def155ce.cb8dc8","name":"","addr":"127.0.0.1","iface":"","port":"10108","ipv":"udp4","outport":"","base64":false,"multicast":"false","x":1246,"y":204.00003051757812,"wires":[]},{"id":"63c11a8c.9927a4","type":"comment","z":"def155ce.cb8dc8","name":"This takes NMEA Output by SeaTalk and adds checksum","info":"","x":506.9999542236328,"y":180.9999542236328,"wires":[]},{"id":"f6520ef8.395e2","type":"serial in","z":"def155ce.cb8dc8","name":"SeaTalk NMEA Input","serial":"c5d7ff46.19d73","x":104,"y":470,"wires":[["4fc0cc12.46de24"]]},{"id":"c2d153f6.75ee2","type":"inject","z":"def155ce.cb8dc8","name":"Inject SeaTalk DBT String","topic":"","payload":"$IIDBT,0008.4,f,,,,","payloadType":"str","repeat":"1","crontab":"","once":false,"x":178,"y":269,"wires":[["4fc0cc12.46de24"]]},{"id":"427edb50.17af24","type":"inject","z":"def155ce.cb8dc8","name":"Inject SeaTalk HDM String","topic":"","payload":"$IIHDM,268.,M","payloadType":"str","repeat":"1","crontab":"","once":false,"x":176,"y":344,"wires":[["4fc0cc12.46de24"]]},{"id":"bc1148e9.02e848","type":"debug","z":"def155ce.cb8dc8","name":"","active":true,"console":"false","complete":"false","x":1254,"y":423,"wires":[]},{"id":"d06bc454.854278","type":"function","z":"def155ce.cb8dc8","name":"Create full DBT string","func":"var nmeaFull=msg.payload;\n\nvar nmeaSplit=nmeaFull.split(\",\");\n   \nvar nmeaFinal=nmeaSplit[0]+\",\"+nmeaSplit[1]+\",f,\";\nnmeaFinal=nmeaFinal+(nmeaSplit[1]/3.28084).toFixed(2)+\",M,\";\nnmeaFinal=nmeaFinal+(nmeaSplit[1]/20).toFixed(2)+\",F\";\n\nmsg.payload = nmeaFinal;\nreturn msg;","outputs":1,"noerr":0,"x":671,"y":253,"wires":[["142750e6.16540f"]]},{"id":"c5d7ff46.19d73","type":"serial-port","z":"","serialport":"/dev/ttyOP_485","serialbaud":"4800","databits":"8","parity":"none","stopbits":"1","newline":"\\n","bin":"false","out":"char","addchar":false}]
(2017-06-28, 05:46 PM)abarrow Wrote: [ -> ]Well, that was fun! Thanks for the help. It looks like I have something that works. 

I'll know when I get a chance to take it down to the boat, but using simulated strings in Node-Red is works great!

Here's the code:
Code:
[{"id":"def155ce.cb8dc8","type":"tab","label":"Flow 2"},{"id":"4fc0cc12.46de24","type":"switch","z":"def155ce.cb8dc8","name":"Filter just DBT and HDM","property":"payload","propertyType":"msg","rules":[{"t":"regex","v":"^(?=.*IIDBT)","vt":"str","case":false},{"t":"regex","v":"^(?=.*IIHDM)","vt":"str","case":false}],"checkall":"true","outputs":2,"x":443.99993896484375,"y":367.99993896484375,"wires":[["d06bc454.854278"],["142750e6.16540f"]]},{"id":"de39de6.89c9d2","type":"function","z":"def155ce.cb8dc8","name":"calc checksum","func":"var nmea = msg.payload;\nvar checksum = 0; \n\nfor(var i = 0; i < nmea.length; i++) { \n  checksum = checksum ^ nmea.charCodeAt(i); \n}\nchecksum = checksum.toString(16);    //convert to hex\nnmea = '$' + nmea + '*' + checksum;  //make the full nmea sentence again\n\nmessage = 'Full message =' + nmea + '\\r\\n' + 'Checksum = ' + checksum ;\n\nmsg.payload = nmea;\nmsg.payload = msg.payload+\"\\r\\n\";\n\nvar msg1 = {\n     payload: msg.payload\n };\n return msg1;","outputs":1,"noerr":0,"x":1018,"y":423,"wires":[["6fe378e5.112f58","bc1148e9.02e848"]]},{"id":"142750e6.16540f","type":"function","z":"def155ce.cb8dc8","name":"Strip front and back","func":"var nmeaFull=msg.payload;\n\n    \nvar nmeaStripped=nmeaFull.substring(1, 99)\n\nmsg.payload = nmeaStripped;\nreturn msg;","outputs":1,"noerr":0,"x":802.0000610351562,"y":423,"wires":[["de39de6.89c9d2"]]},{"id":"6fe378e5.112f58","type":"udp out","z":"def155ce.cb8dc8","name":"","addr":"127.0.0.1","iface":"","port":"10108","ipv":"udp4","outport":"","base64":false,"multicast":"false","x":1246,"y":204.00003051757812,"wires":[]},{"id":"63c11a8c.9927a4","type":"comment","z":"def155ce.cb8dc8","name":"This takes NMEA Output by SeaTalk and adds checksum","info":"","x":506.9999542236328,"y":180.9999542236328,"wires":[]},{"id":"f6520ef8.395e2","type":"serial in","z":"def155ce.cb8dc8","name":"SeaTalk NMEA Input","serial":"c5d7ff46.19d73","x":104,"y":470,"wires":[["4fc0cc12.46de24"]]},{"id":"c2d153f6.75ee2","type":"inject","z":"def155ce.cb8dc8","name":"Inject SeaTalk DBT String","topic":"","payload":"$IIDBT,0008.4,f,,,,","payloadType":"str","repeat":"1","crontab":"","once":false,"x":178,"y":269,"wires":[["4fc0cc12.46de24"]]},{"id":"427edb50.17af24","type":"inject","z":"def155ce.cb8dc8","name":"Inject SeaTalk HDM String","topic":"","payload":"$IIHDM,268.,M","payloadType":"str","repeat":"1","crontab":"","once":false,"x":176,"y":344,"wires":[["4fc0cc12.46de24"]]},{"id":"bc1148e9.02e848","type":"debug","z":"def155ce.cb8dc8","name":"","active":true,"console":"false","complete":"false","x":1254,"y":423,"wires":[]},{"id":"d06bc454.854278","type":"function","z":"def155ce.cb8dc8","name":"Create full DBT string","func":"var nmeaFull=msg.payload;\n\nvar nmeaSplit=nmeaFull.split(\",\");\n   \nvar nmeaFinal=nmeaSplit[0]+\",\"+nmeaSplit[1]+\",f,\";\nnmeaFinal=nmeaFinal+(nmeaSplit[1]/3.28084).toFixed(2)+\",M,\";\nnmeaFinal=nmeaFinal+(nmeaSplit[1]/20).toFixed(2)+\",F\";\n\nmsg.payload = nmeaFinal;\nreturn msg;","outputs":1,"noerr":0,"x":671,"y":253,"wires":[["142750e6.16540f"]]},{"id":"c5d7ff46.19d73","type":"serial-port","z":"","serialport":"/dev/ttyOP_485","serialbaud":"4800","databits":"8","parity":"none","stopbits":"1","newline":"\\n","bin":"false","out":"char","addchar":false}]

WoooHooo ... I will give that a try on my communications errors !

Thanks for posting !
Just got back from the boat, and yep, it works great! I learned a couple of things:

1. I had a serial port configured in OpenPlotter for the incoming NMEA strings from SeaTalk. When Node-RED started trying to access it, nothing worked. Duhhh, need only one process accessing the device, thank you very much! I deleted the incoming port in the OpenPlotter NMEA0183 page, and the data started flowing through Node-RED.
2. Another obvious thing - I needed an incoming stream into OpenPlotter from Node-Red. I used UDP port 10108.
3. Somehow my script wasn't stripping the final return after the HDM strings, and OpenCPN was seeing it as a string with no CRC (since it was on the next line). I did a quick function to strip the return on those strings, and HDM started coming into the system.

Thanks so much to those who helped on this. It's always a great feeling to see something like this working. I hope this helps others.