This forum uses cookies
This forum makes use of cookies to store your login information if you are registered, and your last visit if you are not. Cookies are small text documents stored on your computer; the cookies set by this forum can only be used on this website and pose no security risk. Cookies on this forum also track the specific topics you have read and when you last read them. Please confirm whether you accept or reject these cookies being set.

A cookie will be stored in your browser regardless of choice to prevent you being asked this question again. You will be able to change your cookie settings at any time using the link in the footer.

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Anyone datalogging?
#17
(2017-11-17, 01:23 PM)shark24 Wrote: I'm aware of the concept of time series databases. Our boats aren't exactly like the Queen Mary 2 so we won't produce terabytes of data. From my point of view most of the databases you can run on the pi like mysql, influxdb, etc. are slowing down the system. That's why I prefer rather logging data to a file/sqlite than having a database server in the background bothering my cpu. Certainly the data queries will take longer but my pi isn't kept busy constantly. I don't know why you would need two years of sensor data on your pi and why you need a good performance for the data analysis. And even if you want time series data of a long range for a graph it'll be one slow select query and that's it. The select query takes 80 instead of 10 milliseconds - pardon my French but I don't give a rat's ***. I won't sacrifice over all system performance for two or three slow queries it takes to get the  some data from bigger ranged time series.   
If you want to use openplotter and the pi for real time data analysis of big time series maybe you got the wrong software and hardware.
Very much agree with all of that. I played around with RRDtools for a while, which worked OK but was a bit clunky to work with. Sqlite does all that is needed taking little CPU time. Handy having it as just one file as well, you can copy across to a faster machine easily to back up or do some more intensive plotting/sql. I've had one running pretty much constantly since about august recording once a minute, file is around 22Meg.
Reply


Messages In This Thread
Anyone datalogging? - by PaddyB - 2017-11-14, 10:03 PM
RE: Anyone datalogging? - by shark24 - 2017-11-15, 10:19 AM
RE: Anyone datalogging? - by PaddyB - 2017-11-15, 10:51 AM
RE: Anyone datalogging? - by shark24 - 2017-11-15, 11:45 AM
RE: Anyone datalogging? - by PaddyB - 2017-11-15, 02:27 PM
RE: Anyone datalogging? - by tocan - 2017-11-15, 03:13 PM
RE: Anyone datalogging? - by PaddyB - 2017-11-15, 03:45 PM
RE: Anyone datalogging? - by tocan - 2017-11-15, 05:20 PM
RE: Anyone datalogging? - by tocan - 2017-11-15, 03:51 PM
RE: Anyone datalogging? - by shark24 - 2017-11-15, 04:49 PM
RE: Anyone datalogging? - by tocan - 2017-11-15, 09:06 PM
RE: Anyone datalogging? - by shark24 - 2017-11-16, 11:10 AM
RE: Anyone datalogging? - by PaddyB - 2017-11-16, 10:46 AM
RE: Anyone datalogging? - by tocan - 2017-11-16, 09:01 PM
RE: Anyone datalogging? - by shark24 - 2017-11-17, 01:23 PM
RE: Anyone datalogging? - by PaddyB - 2017-11-17, 02:09 PM
RE: Anyone datalogging? - by tocan - 2017-11-17, 09:09 AM
RE: Anyone datalogging? - by tocan - 2017-11-17, 02:35 PM
RE: Anyone datalogging? - by PaddyB - 2017-11-17, 03:09 PM
RE: Anyone datalogging? - by shark24 - 2017-11-19, 09:08 AM
RE: Anyone datalogging? - by tocan - 2017-11-19, 04:44 PM
RE: Anyone datalogging? - by shark24 - 2017-11-19, 06:14 PM
RE: Anyone datalogging? - by tocan - 2017-11-19, 07:16 PM
RE: Anyone datalogging? - by shark24 - 2017-11-19, 08:21 PM
RE: Anyone datalogging? - by tocan - 2017-11-19, 11:23 PM
RE: Anyone datalogging? - by PaddyB - 2017-11-22, 11:54 PM

Forum Jump:


Users browsing this thread: 1 Guest(s)