[Voyage-linux] Best Practices for writing logs & data to CF

[Nils Toedtmann]

Dear Voyage community


First: Voyage Linux is great! It makes building embedded systems
significantly easier - thanks to everyone who contributed!


I have a number of questions about writing data to CF on ALIX (before
you rant about even wanting to do this - see my use case and motivation
in the PS below).

I apologize in advance if i haven't found existing answers in the Voyage
documentation. I did do some research, but admit that being only a
sysadmin, i didn't dive into the depths of Voyage's development docs.


Here are my questions:

 * Most documentation says that CF wears down quickly when many write
operations happen. But those docs are 5-10 years old. Do modern
'industrial grade' CF cards still suffer from this problem? I wonder
because nowadays it is pretty common to use flash based SSDs in
write-heavy systems.

 * PC Engine recommend [1] to have partitions aligned to flash's 2K or
4K page size. But most partitioning software assumes 63 sectors (512
bytes each) per track. How would i partition a CF card such that the
partitions are aligned properly, e.g. using Voyage's fdisk? Maybe just
use multiples of '8 tracks = 63*4K'?

 * I have the rootfs on a ro partition, and write my data to a separate
rw partition. Is it safe to assume that even if a powercycle corrupts my
data partition, that the rootfs partition is still intact?

 * Besides your recommendations to adjust the vm.dirty_* syscontrols
when writing to CF [2] - What are your recommended file systems and
mount options for rw data partitions to reduce the risk of data loss on
a powercycle? E.g. i see people recommending to use ext3 or ext4 with
'journal=data' and/or 'sync' (at the expense of reduced write speed). Is
that sound?

 * Does Voyage Linux do a file system check when it mounts additional
file systems if they have fs_passno=2 (6th field) in /etc/fstab? Does
the fsck call include the "-y" option to force automatic fixing mode?

 * Would it work if i symlinked /var/log/ into a rw data partition and
removed it from 'SYNC_DIRS' in /etc/init.d/voyage-sync ?


Any advise would be greatly appreciated!


Cheers, /nils.


[1] http://www.pcengines.ch/cfperf.htm
[2] http://linux.voyage.hk/content/getting-started-v09x


PS: Disclaimer & use case:

I know that CF is not advised for permanent storage when powercycles can
happen at any time. We use ALIX + Voyage linux to collect and cache, and
send sensor data to central servers. The data we write to CF is not
critical. However, for a number of reasons it would be very helpful if
the data survived powercycles:

 * Cached sensor data that hasn't been sent out yet (some of our ALIXes
have unstable uplinks, e.g. 3G) would not get lost

 * Stored states would allow the data collection to resume significantly
quicker

 * Logs up to just before a system fault (e.g. OOM) would be very
helpful for debugging