2020-08-06, 11:55 AM
(This post was last modified: 2020-08-06, 12:45 PM by jamos.tan@gmail.com.)
Came across this explanation about why SD is not a good idea for continuous writing:
---
Here's a good read: http://www.embeddedarm.com/about/resource.php?item=459
I had a loooooooong discussion about this with a colleague of mine after we started seeing boards die.
Basically you're eventually doomed unless you mount the whole disk as read only since the wear leveling algorithms in the flash have no knowledge of what a partition is and will eventually end up with suppesed-to-be-read-only data mixed in with the writable partition erase blocks. If you're writing to flash, it will eventually fail by unfortunate design.
It tooks his previous company 6 months of fighting to come to terms with this in their last product. They had to write data, so eventually used usb flash that the customer could easily replace when things eventually died. They tried every flash card they could get their hands on, read only partitions, etc and eventually had to give up.
Use the SD card you say! Any micro SD card you can put in the slot is absolutely not meant for continuous writing. The SD card spec has a very specific use case in mind (video and images), and logging or using it as a sparse write file system goes completely against the intended SD card design specs. Industrial grade write-tolerant flash will cost you hundreds of dollars more than something on Amazon.
With our current product, I told my boss that I was worried about corruption and that we would eventually go to read only once we debugged the boards. Within two weeks of only log messages, all of our boards started dyeing. The next day, all disks were mounted as read only and issues are debugged with the in-memory log files. We haven't seen any failures in 6 months now.
The easy solution is trying to force the answer of "why are you writing anything to persistent storage?" to be "there's no good reason since it eventually bricks our product". If you want something that will last forever, you will not write to standard flash media. If you can't, then maybe use a usb flash drive (MUCH better life than a micro sd card) and count the days until it corrupts or someone pulls the power at an inopportune time. You could always use a battery backup to get rid of the power off issue. :-\
This is all doom and gloom, but it's a consequence of inconsistent power, buffers, and the destruction nature of quantum tunneling.
---
I still didn't understand why SSD could be better, so i did more investigating and came across this: https://serverfault.com/questions/923971...power-loss
In there they say that (S)SD can be safe, if you buy enterprise products which have built-in capacitors with enough power to finish the writing if there should be a power loss. However, this would make the solution a bit more expensive.
At the moment, I feel that SD is more designed for photo's, video's (in our case, marine water charts) and if we want a real stable solution we should configure them as read-only. But, with an OS, or if there are databases present, (cheap) (S)SD is not an option, and just waiting for disaster which will come at some point.
I am beginning to think that SATA will be the next thing then? I'll need to do some more investigating into power usage, because this needs a motor to spin and I will need to see if linux / debian system will cause problems in itself regardless of solid state drives.
And this article explains the SD power failure completely: http://www.storagesearch.com/ssd-power-going-down.html
Edit: this article explains that all disk types can get corrupted when there is power loss: https://www.datanumen.com/blogs/ssd-will...-failures/
So, it's either expensive SSD's, or use a UPS with shutdown command, or use read only mode for the boot sector, like described here: https://raspberrypi.stackexchange.com/qu...e-raspbian
This instruction for read only seems better: https://medium.com/swlh/make-your-raspbe...558694de79
Edit: Ok, so apparently if you use read-only mode, all desktop functionality will stop. Not sure if webservers, signalk, openplotter apps will be running, but if you use desktop mode this solution will not be feasible.
---
Here's a good read: http://www.embeddedarm.com/about/resource.php?item=459
I had a loooooooong discussion about this with a colleague of mine after we started seeing boards die.
Basically you're eventually doomed unless you mount the whole disk as read only since the wear leveling algorithms in the flash have no knowledge of what a partition is and will eventually end up with suppesed-to-be-read-only data mixed in with the writable partition erase blocks. If you're writing to flash, it will eventually fail by unfortunate design.
It tooks his previous company 6 months of fighting to come to terms with this in their last product. They had to write data, so eventually used usb flash that the customer could easily replace when things eventually died. They tried every flash card they could get their hands on, read only partitions, etc and eventually had to give up.
Use the SD card you say! Any micro SD card you can put in the slot is absolutely not meant for continuous writing. The SD card spec has a very specific use case in mind (video and images), and logging or using it as a sparse write file system goes completely against the intended SD card design specs. Industrial grade write-tolerant flash will cost you hundreds of dollars more than something on Amazon.
With our current product, I told my boss that I was worried about corruption and that we would eventually go to read only once we debugged the boards. Within two weeks of only log messages, all of our boards started dyeing. The next day, all disks were mounted as read only and issues are debugged with the in-memory log files. We haven't seen any failures in 6 months now.
The easy solution is trying to force the answer of "why are you writing anything to persistent storage?" to be "there's no good reason since it eventually bricks our product". If you want something that will last forever, you will not write to standard flash media. If you can't, then maybe use a usb flash drive (MUCH better life than a micro sd card) and count the days until it corrupts or someone pulls the power at an inopportune time. You could always use a battery backup to get rid of the power off issue. :-\
This is all doom and gloom, but it's a consequence of inconsistent power, buffers, and the destruction nature of quantum tunneling.
---
I still didn't understand why SSD could be better, so i did more investigating and came across this: https://serverfault.com/questions/923971...power-loss
In there they say that (S)SD can be safe, if you buy enterprise products which have built-in capacitors with enough power to finish the writing if there should be a power loss. However, this would make the solution a bit more expensive.
At the moment, I feel that SD is more designed for photo's, video's (in our case, marine water charts) and if we want a real stable solution we should configure them as read-only. But, with an OS, or if there are databases present, (cheap) (S)SD is not an option, and just waiting for disaster which will come at some point.
I am beginning to think that SATA will be the next thing then? I'll need to do some more investigating into power usage, because this needs a motor to spin and I will need to see if linux / debian system will cause problems in itself regardless of solid state drives.
And this article explains the SD power failure completely: http://www.storagesearch.com/ssd-power-going-down.html
Edit: this article explains that all disk types can get corrupted when there is power loss: https://www.datanumen.com/blogs/ssd-will...-failures/
So, it's either expensive SSD's, or use a UPS with shutdown command, or use read only mode for the boot sector, like described here: https://raspberrypi.stackexchange.com/qu...e-raspbian
This instruction for read only seems better: https://medium.com/swlh/make-your-raspbe...558694de79
Edit: Ok, so apparently if you use read-only mode, all desktop functionality will stop. Not sure if webservers, signalk, openplotter apps will be running, but if you use desktop mode this solution will not be feasible.