(2017-11-17, 01:23 PM)shark24 Wrote: I'm aware of the concept of time series databases. Our boats aren't exactly like the Queen Mary 2 so we won't produce terabytes of data. From my point of view most of the databases you can run on the pi like mysql, influxdb, etc. are slowing down the system. That's why I prefer rather logging data to a file/sqlite than having a database server in the background bothering my cpu. Certainly the data queries will take longer but my pi isn't kept busy constantly. I don't know why you would need two years of sensor data on your pi and why you need a good performance for the data analysis. And even if you want time series data of a long range for a graph it'll be one slow select query and that's it. The select query takes 80 instead of 10 milliseconds - pardon my French but I don't give a rat's ***. I won't sacrifice over all system performance for two or three slow queries it takes to get the some data from bigger ranged time series.Very much agree with all of that. I played around with RRDtools for a while, which worked OK but was a bit clunky to work with. Sqlite does all that is needed taking little CPU time. Handy having it as just one file as well, you can copy across to a faster machine easily to back up or do some more intensive plotting/sql. I've had one running pretty much constantly since about august recording once a minute, file is around 22Meg.
If you want to use openplotter and the pi for real time data analysis of big time series maybe you got the wrong software and hardware.
Anyone datalogging?
|
|
« Next Oldest | Next Newest »
|
Messages In This Thread |
Anyone datalogging? - by PaddyB - 2017-11-14, 10:03 PM
RE: Anyone datalogging? - by shark24 - 2017-11-15, 10:19 AM
RE: Anyone datalogging? - by PaddyB - 2017-11-15, 10:51 AM
RE: Anyone datalogging? - by shark24 - 2017-11-15, 11:45 AM
RE: Anyone datalogging? - by PaddyB - 2017-11-15, 02:27 PM
RE: Anyone datalogging? - by tocan - 2017-11-15, 03:13 PM
RE: Anyone datalogging? - by PaddyB - 2017-11-15, 03:45 PM
RE: Anyone datalogging? - by tocan - 2017-11-15, 05:20 PM
RE: Anyone datalogging? - by tocan - 2017-11-15, 03:51 PM
RE: Anyone datalogging? - by shark24 - 2017-11-15, 04:49 PM
RE: Anyone datalogging? - by tocan - 2017-11-15, 09:06 PM
RE: Anyone datalogging? - by shark24 - 2017-11-16, 11:10 AM
RE: Anyone datalogging? - by PaddyB - 2017-11-16, 10:46 AM
RE: Anyone datalogging? - by tocan - 2017-11-16, 09:01 PM
RE: Anyone datalogging? - by shark24 - 2017-11-17, 01:23 PM
RE: Anyone datalogging? - by PaddyB - 2017-11-17, 02:09 PM
RE: Anyone datalogging? - by tocan - 2017-11-17, 09:09 AM
RE: Anyone datalogging? - by tocan - 2017-11-17, 02:35 PM
RE: Anyone datalogging? - by PaddyB - 2017-11-17, 03:09 PM
RE: Anyone datalogging? - by shark24 - 2017-11-19, 09:08 AM
RE: Anyone datalogging? - by tocan - 2017-11-19, 04:44 PM
RE: Anyone datalogging? - by shark24 - 2017-11-19, 06:14 PM
RE: Anyone datalogging? - by tocan - 2017-11-19, 07:16 PM
RE: Anyone datalogging? - by shark24 - 2017-11-19, 08:21 PM
RE: Anyone datalogging? - by tocan - 2017-11-19, 11:23 PM
RE: Anyone datalogging? - by PaddyB - 2017-11-22, 11:54 PM
|
Users browsing this thread: 1 Guest(s)