This forum uses cookies
This forum makes use of cookies to store your login information if you are registered, and your last visit if you are not. Cookies are small text documents stored on your computer; the cookies set by this forum can only be used on this website and pose no security risk. Cookies on this forum also track the specific topics you have read and when you last read them. Please confirm whether you accept or reject these cookies being set.

A cookie will be stored in your browser regardless of choice to prevent you being asked this question again. You will be able to change your cookie settings at any time using the link in the footer.

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Logging data
#1
Well that was easy!!!

After years of jumping through hoops with node-red saving data to a sqlite databases then creating queries going into node-red charts, last night I had a look at other offerings. Simple! Install influxdb and chronograf then set up the signalk store to influxdb app and that's it! 
I did try influxdb and some other viewing options ages ago but gave up as it was such a resource hog, seems much better behaved these days  Cool

Chronograf has a handy plot delayed data so you can overlay a day ago or whatever to compare. 
(Below is just an hour delay, haven't got a days data yet )   

[Image: FuHaAKo.png]
Reply
#2
thanks for posting

I had held off trying the Influxdb option- I thought the Raspberry has enough to do without being a database server as well....

But by the sound of it, it does not use too much processor/memory ?
Are you using the SD card or an external USB drive for the database & how much space does it take up ?
I'm guessing it would be possible to write a query to delete old data- say over 30 days old- if disk space is an issue ?


Andrew
Reply
#3
(2018-08-26, 08:56 AM)barney2074 Wrote: thanks for posting

I had held off trying the Influxdb option- I thought the Raspberry has enough to do without being a database server as well....

But by the sound of it, it does not use too much processor/memory ?
Are you using the SD card or an external USB drive for the database & how much space does it take up ?
I'm guessing it would be possible to write a query to delete old data- say over 30 days old- if disk space is an issue ?


Andrew

Watching htop it seems to have a blip of activity every little while but it all seems to work OK. Disk space no doubt would add up if recording lots of data every second , you can set a retention time so if will only keep data for  a day or month or whatever you want.
Reply
#4
(2018-08-26, 08:56 AM)barney2074 Wrote: thanks for posting

I had held off trying the Influxdb option- I thought the Raspberry has enough to do without being a database server as well....

But by the sound of it, it does not use too much processor/memory ?
Are you using the SD card or an external USB drive for the database & how much space does it take up ?
I'm guessing it would be possible to write a query to delete old data- say over 30 days old- if disk space is an issue ?


Andrew

Revisiting this again - influxdb was having a 'blip' of high CPU usage every few seconds, from just watching htop. Google came up with this after some searching which seems to solve that, it's floating around 2% CPU now in htop so much better >
https://github.com/influxdata/influxdb/issues/9475
Just change  a line in the config >

Code:
[monitor]

store-enabled = false

Also installed telegraf for monitoring the Pi but not really dug deep in there yet though looks very useful as an easy way to view what the pi is up to.
Reply
#5
thanks Paddy

For now, I'm staying with the log files to produce charts such as the below
The main issue for me is the ease in getting data off the boat
i.e it is easy to transfer a few files at the end of a day's racing

https://snapshot.raintank.io/dashboard/s...kvqO478UGK
Reply
#6
(2018-10-01, 08:47 AM)barney2074 Wrote: thanks Paddy

For now, I'm staying with the log files to produce charts such as the below
The main issue for me is the ease in getting data off the boat
i.e it is easy to transfer a few files at the end of a day's racing

https://snapshot.raintank.io/dashboard/s...kvqO478UGK

This is what I'm looking for. How do you generate these reports?
Reply
#7
(2021-09-23, 04:13 PM)baltisailer Wrote:
(2018-10-01, 08:47 AM)barney2074 Wrote: thanks Paddy

For now, I'm staying with the log files to produce charts such as the below
The main issue for me is the ease in getting data off the boat
i.e it is easy to transfer a few files at the end of a day's racing

https://snapshot.raintank.io/dashboard/s...kvqO478UGK

This is what I'm looking for. How do you generate these reports?
Hi

I used 'Grafana' to produce the reports. It obviously depends on your skill level, but it is relatively easy to use.
The simplest method is to bring data from the InfluxDB directly into Grafana

Subsequent to this discussion thread, I did start using InfluxDB on a raspberry PI running on the boat.

I used a separate PI for this database- can't remember the details- but the higher spec one - 3B??, and removed this PI at the end of a day's sailing, and produced Grafana reports at home.
The Openplotter PI stayed in the boat, since it was hooked up to the instruments etc

Hope this helps

Andrew
Reply
#8
you may also like to see this https://forum.openmarine.net/showthread.php?tid=3478
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)