[gfarm-discuss:17220] Re: Gfarm version 2.0.0 released

["Wilfred Li" <wilfred@xxxxxxxx>]

Hi, Osamu,

Could you please also provide a description of the security improvements
in v2?

Thanks

Wilfred

-----Original Message-----
From: Osamu Tatebe [mailto:tatebe@xxxxxxxxxxxxxxxx] 
Sent: Wednesday, November 28, 2007 8:39 AM
To: gfarm-discuss@xxxxxxxxxx
Cc: Matt
Subject: [gfarm-discuss:17215] Re: [gfarm-announce:12757]Gfarm version
2.0.0 released

Hi Matt,

(I mailed this to gfarm-discuss ML not gfarm-announce ML.)

Gfarm v1 directly uses a postgresql (or a openldap) server as
a metadata server.  On the other hand, Gfarm v2 uses a gfmd
as a metadata server.  Gfmd in Gfarm v2 basically caches all
metadata in memory, and accesses a backend database (postgresql
or openldap) server in background.  When responding a request
there is no disk wait.  Also, we minimuze the number of
interactions with the metadata server in Gfarm v2.  That is why
performance for small files is improved.

Latency of a file creation and other file system operations is
almost the same as that of NFS exported in async mode.
Regarding I/O bandwidth, Gfarm v2 can achieve scalable
performance when you increase the number of clients (and the
number of file system nodes).

Regarding a failover of the metadata server, you can set up
a stand-by metadata server when a backend database is shared.
Moreover, the backend database (postgresql or openldap)
itself can be configured with some redundancy by many ways.

Thanks,
Osamu

On Wed, 28 Nov 2007 12:32:20 +0000
Matt <mattmoran76@xxxxxxxxx> wrote:

> Osamu,
> 
> Congratulations on releasing Gfarm v2.
> 
> Can you advise on how you have improved the handling of small files,
as I
> noticed you are still using either a ldap or PostgreSQL backend for
the
> mds.  When I was testing gfarm, I found when reaching millions of
files that
> this was my bottleneck.
> 
> It's possible that I didn't have the mds configured properly to grab
all
> available memory (4gb), is there specific settings for this? will a
64bit
> version make more memory available to the mds?
> 
> Also, can you advise if performance has been improved, as I found
results
> using iozone disappointing compared to a standard NFS server and
lustre
> cluster.
> 
> Thirdly, have you any thoughts or suggestions about clustering/failing
over
> the mds server?
> 
> Thanks,
> 
> Matt
> 
> On 28/11/2007, Osamu Tatebe <tatebe@xxxxxxxxxxxxxxxx> wrote:
> >
> > Hi all,
> >
> > We are very pleased to announce the first public release of Gfarm
file
> > system version 2.0.0, aka Gfarm v2, which is a major new release of
> > the Gfarm file system. It solves several security issues in Gfarm
v1,
> > and improves performance for small files dramatically.
> >
> >                 http://sourceforge.net/projects/gfarm/
> >
> > To mount the Gfarm file system, gfarm2fs is also required to be
> > installed. Since it is a major new release, there is no
compatibility
> > in structure of a spool directory on each file system node. Easiest
> > way to migrate files from a Gfarm v1 file system is
> >
> > 1. Install Gfarm v2 to a different target directory, and configure a
> >    Gfarm v2 file system using a different spool directory and
> >    different ports from Gfarm v1.
> >
> > 2. Mount Gfarm v1 file system and Gfarm v2 file system by gfarmfs
and
> >    gfarm2fs, respectively, on different mount points.
> >
> > 3. Copy files possibly in parallel using multiple clients.
> >
> > Enjoy!
> > Osamu