[Voyage-linux] aufs: Sync to flash with our reboot ?

[Robert Rawlins - Think Blue]

Hi Mike,

Thanks for the suggestion and an update on your experience. I have a few files which need to be kept in sync with the disk, a single directory of multimedia files and also a MySQL database. I'll look into the addition of an SSD module but it'd something I'd rather not venture into.

What does the SSD module give me that a separate writable partition does not? What about perhaps using an USB memory stick mounted for writable storage? Would that suffice as an SSD module?

Thanks mate,

Rob

-----Original Message-----
From: Mike O'Connor [mailto:mike at pineview.net] 
Sent: 08 June 2009 00:44
To: Robert Rawlins - Think Blue
Cc: voyage-linux at voyage.hk
Subject: Re: [Voyage-linux] aufs: Sync to flash with our reboot ?

Hi Robert

Yes I've been using the script which was put forward supplied by Guido.
My issue with it not working was simply the fact that I was using a
custom kernel which did not have the aufs patches.

I'm not writing out the changes except when I think I need to.

I'm of the option that any one who need permanate storage should use a
SSD module and run a normal linux OS.

For the parts of the file system which are changing a lot like /var/log
I would just mount a TMPFS over the top and leave the rest of on the disk.

Mike


Robert Rawlins - Think Blue wrote:
> Hey Mike,
>
> Did you ever make much progress on your solution for this? We're in a very similar situation to this where we have data we need to persist on the disk so that it's available even after a power loss. I've been toying with creating second partitions on the disk, which appears to work O.K. however is still prone to corruption on occasion.
>
> How are you implementing this? Are you using a cron entry to call the resync script on a regular basis to ensure that any data changes are persisted to the disk? Have you found it to be reliable?
>
> I'm really looking to find a solution to this as data loss is proving a real pain in the neck for me :-(
>
> Cheers mate,
>
> Robert
>
> -----Original Message-----
> From: voyage-linux-bounces+robert.rawlins=thinkbluemedia.co.uk at list.voyage.hk [mailto:voyage-linux-bounces+robert.rawlins=thinkbluemedia.co.uk at list.voyage.hk] On Behalf Of Mike O'Connor
> Sent: 08 May 2009 07:03
> To: Guido De Rosa
> Cc: voyage-linux at voyage.hk
> Subject: Re: [Voyage-linux] aufs: Sync to flash with our reboot ?
>
> Hi Guido
>
> I'm trying out this script, I seem to have a problem, the /var/log
> directory is not being overlaided with aufs.
>
> voyage:/tmp/voyage-0.6.1/etc/rc1.d# cat /proc/mounts
> rootfs / rootfs rw 0 0
> none /sys sysfs rw,nosuid,nodev,noexec 0 0
> none /proc proc rw,nosuid,nodev,noexec 0 0
> udev /dev tmpfs rw,size=10240k,mode=755 0 0
> /dev/disk/by-label/ROOT_FS / ext2 rw,noatime,errors=continue 0 0
> tmpfs /lib/init/rw tmpfs rw,nosuid,mode=755 0 0
> varrun /var/run tmpfs rw,nosuid,mode=755 0 0
> varlock /var/lock tmpfs rw,nosuid,nodev,noexec 0 0
> usbfs /proc/bus/usb usbfs rw,nosuid,nodev,noexec 0 0
> tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0
> devpts /dev/pts devpts rw,nosuid,noexec,gid=5,mode=620 0 0
> tmpfs /tmp tmpfs rw,nosuid,nodev 0 0
>
> Which means the remount,ro during the boot process fails.
>
> I see during the boot process it doing the voyage-sync start, no errors
> are printed to the screen.
>
> I think the issue is that syslogd has been restarted before the mounting
> of aufs.
>
> One thing to note is that I have done a apt-get upgrade before starting
> the proccess of getting the new voyage-sync
>
> Mike
>
>
>
> Guido De Rosa wrote:
>   
>> 2009/3/30 Kim-man 'Punky' TSE <punkytse at punknix.com>:
>>
>>   
>>     
>>> I would try I could have some update on it later this week. �After I tested
>>> Guido's update and some of my additions.
>>>     
>>>       
>> Hi! Please always consider the last update:
>>
>> http://bitbucket.org/gderosa/net-appliances/src/tip/voyage/etc/init.d/voyage-sync
>>
>> Specifically, since rev9, some extra check has been added:
>>
>> 			# verify that $SYNC_DIR is actually mounted as a
>> 			# $UNIONFS; otherwise, avoid dangerous over-writings
>> 			# of $SYNCFS_ROOT/$SYNC_DIR
>> 			if (mount | grep $SYNC_DIR | grep "type $UNIONFS" > /dev/null) ; then
>> 				echo -n "  Sync'ing $SYNC_DIR to $SYNCFS_ROOT$SYNC_DIR ... "
>> 				rsync $RSYNC_OPTS $SYNC_DIR $SYNCFS_ROOT/`dirname $SYNC_DIR`
>> 				log_action_end_msg $?
>> 			fi
>>
>> Think about the following scenario:you've been using AUFS/UNIONFS for
>> a while and recently you have upgraded the kernel, but forgot to add
>> aufs support; you also have added to VOYAGE_SYNC_DIRS some directories
>> with important data; in this scenario, after the first reboot,
>> mounting of $SYNC_DIR fails, and you get stuck with old $SYNC_DIR
>> contained in your root partition. After another reboot, new data in
>> $SYNCFS_ROOT are overwritten by older ones which is definitely not
>> what you want. That's why an extra check is necessary.
>>
>> So, just for the first time, when you just added a new directory to
>> VOYAGE_SYNC_DIR, you will have to manually copy it to  $SYNCFS_ROOT/ .
>>
>> (Maybe an helper function could be added to
>> /usr/local/sbin/voyage-sync to do this syncing-without-checking
>> automatically, but definitely not in /etc/init.d )
>>
>>   
>>     
>
>
> _______________________________________________
> Voyage-linux mailing list
> Voyage-linux at list.voyage.hk
> http://list.voyage.hk/mailman/listinfo/voyage-linux
>
>