[Voyage-linux] Best Practices for writing logs & data to CF

[John Coltrane]

Hello Nils,

On 07/11/2013 10:06 PM, Nils Toedtmann wrote:
> Hi John
>
> On 11.07.2013 18:57, John Coltrane wrote:
>>>    * Besides your recommendations to adjust the vm.dirty_* syscontrols
>>> when writing to CF [2] - What are your recommended file systems and
>>> mount options for rw data partitions to reduce the risk of data loss on
>>> a powercycle? E.g. i see people recommending to use ext3 or ext4 with
>>> 'journal=data' and/or 'sync' (at the expense of reduced write speed). Is
>>> that sound?
>> I have seen recommendations to use EXT4 without journaling, to get the
>> advantages of EXT4 without the disadvantages of EXT3 and older. So far I
>> have no bad experience. You can format a fresh partition by doing a
>>
>> mke2fs -t ext4 -O ^has_journal /dev/sdaX
>>
>> or remove the journal of an existing FS with
>>
>> tune2fs -O ^has_journal /dev/sdaX
>
> Turning of journalling probably increases write speed and reduces write
> load on the flash (so it increases the CF's life expectancy). But it
> surely makes it *more* likely to end up with an inconsistent file system.
>
> But my aim is to *reduce* the risk of corrupt read-write file systems,
> and i am willing to pay with write performance or even with CF life time.
I think it is a good idea to use filesystems (with deactivated 
journaling) as often as possible as readonly. But thats a rule you can 
apply to all FS on Flash devices. If you can, you can also try FSs 
developed to be used on Flash devices (NilFS or something similar), but 
I have no experience dealing with this kind of stuff.
Whatever FS you use, try to shut down the OS not by pulling he plug, but 
by using the/a shutdown procedure, so that the FS has a chance to come 
to a clean state. A readonly FS is always in a clean state (my opinion).

You can also partition your FS in a way to separate regions with write 
activity  (for instance /var or /home) to one or more region of the 
physical device, or to separate physical devices (if it is possible), to 
reduce the risk of getting corrupt FSs with valuable, must-have data. Or 
you use a kind of failover strategy, kind of mirroring, whatever. Or you 
do a RAID5 over several Flash cards, that should be no problem. It's all 
kind of planning, risk management.

Hope that helps

-John

>
> /nils.
>
>