R: [Voyage-linux] ext3

[Rob Jenkins]

Hi Jeff,

Don't know if this would work for you, but I just installed MySQL client
on my boxes and have them send their updates directly to a MySQL server
in my data center. I have a little over 100 boxes with updates happening
frequently (check in time, OS and environment stats, users stats, etc.).

I did also write a small PHP application to tunnel a serialized data
string to the server and have all the MySQL functions happen there to
scale the network. I have not implemented it yet but it cut down on
process time and bandwidth. You could easily do the same in Ruby.

Just tossing it out there,

Rob

On Thu, 2009-03-26 at 05:57 -0700, Jeff R. Allen wrote:
> > I think Jeff's problem is to have both data and filesystem integrity even if the system experience a powerloss.
> >
> > ....Jeff, may you give us some more specifics?
> 
> Sure, I'll give you some more details in hopes that someone does my
> job for me. :)
> 
> The constraints I am working under are:
>   * Devices will be installed where maintenance is very
> expensive/difficult. They will be solar-powered, and with intermittent
> and low bandwidth uplink. A three year lifetime without repair is
> absolutely required, a five year lifespan would be better.
>   * Devices will be running applications that were not originally
> designed for the embedded/remote/low-power/low-bandwidth setting. I
> seek to minimize the changes to the apps.
>   * The applications demand things like Java servlets, SQL databases,
> Ruby on Rails, and all the rest of the heavy-weight software stacks
> found in Software As a Service settings.
> 
> My current impression (not from measurement, but from look at best
> practice in the low-power device industry) is that I MUST NOT have a
> SQL server use a CF disk as backing store for routine transactions. I
> think it is certain the write transaction rate the CF disk will see
> would be incompatible with constraint #1.
> 
> My current idea is that MySQL's data directory will be in ramfs, and
> that I'll have to build in hooks to the app to schedule a sync to
> non-volatile storage when required. But that means my databases can
> never be larger than RAM, which is a bummer. What I'd really like is a
> database config where the underlying tables are readable from
> non-volatile storage, but the application of transaction logs onto
> those tables was controllable, so that I can batch the updates (and
> remount RW only during the update application).
> 
> It has long occurred to me that I should be recruiting colleagues not
> from the web-development community, but from JPL (where NASA makes
> space probes). Their environment is much more like my target
> environment. :)
> 
>   -jeff
> 
> _______________________________________________
> Voyage-linux mailing list
> Voyage-linux at list.voyage.hk
> http://list.voyage.hk/mailman/listinfo/voyage-linux