Gfarm - Release note

[2007.3.31] Version 1.4.1 is released

Updated Features

• Backend database
  • support a new data schema to reduce data amount in the PostgreSQL server. Note that backward compatibility with the old database schema is supported.
• gfexport(1) - improve the performance of gfexport.
• gfdump(1) - add -v option to see the progress of gfdump.
• gfarm_agent(1) - report initialization error when it fails to read the configuration file. (Bugzilla-ja #75)
• libgfarm
  • print verbose error messages in UDP processing.
  • improve UDP re-transmission scheme to avoid aggressive re-transmission that may make UDP packets dropped. The first timeout is changed from 10 milliseconds to 8 seconds.
  • increase default TCP connection timeout from 5 seconds to 30 seconds.
  • make a reconnection mechanism to gfsd robust against more network error cases.
• Bug fix
  • gfrm(1) - fix a bug that gfrm cannot delete a file that does not have any file section copy information.
  • gfwhoami(1) - fix a bug that gfarm_terminate() is not executed.
  • gfarm_agent(1)
    • fix a bug that gfarm_agent might consume 100% of CPU when pthread_create() fails, at least, in Fecora Core 5 (kernel 2.6.19-1.2288.fc5smp).
  • libgfarm
    • fix a bug that gfs_rename() doesn't return an error when a directory rename operation fails and undoes. (Bugzilla-ja #76)
    • fix a bug that gfs_pio_seek() does not work in some situations. (Bugzilla-ja #78)
    • fix a bug that a client may cause segmentation fault after re-connecting to a gfarm_agent metadata cache server.
    • fix a bug that gfarm_pgsql_path_info_get() does not retry just after a PostgreSQL server is restarted.
    • fix a bug that gfarm_pgsql_terminate() does not close the connection.
  • configure - use autoconf-2.59 instead of 2.60 to suppress harmless WARNING messages.
  • Syscall hooking library
    • remove redefinition of "offsetof," which is exposed in glibc-2.3.1 in Fedora Core 6.
    • acccept //gfarm also as a mount directory.

[2006.11.13] Version 1.4 is released

Updated Features

• gfrm(1)
  • support parallel removal of files to speed up operations if gfrm is compiled in OpenMP C compiler.
  • add -j <#thread>, -D , -H , -N <#replica>, -n, -q, -v options. -j option is only available if gfrm is compiled in OpenMP C compiler.
• gfwhere(1) - add -r and -R options for recursive listing.
• config-gfarm - support CentOS.
• gfsplck(1)
  • add a functionality to release text-busy status. A file which was being opened by crashed application programs holds text-busy status. (Bugzilla-ja #60)
• gfimport_text(1) - add -N <#host> option.
• gfarm_agent(1)
  • surpress too much and unimportant log output to improve performance.
  • add -L option to specify a syslog level.
• gfarm.conf(5)
  • add "level log_level" directive.
  • add "record_atime disable" directive.
• gfmd(8) and gfsd(8) - add -L option.
• libgfarm
  • improve a scheduling algorithm:
    • when a gfsd stops during reading a file, retry another file replica automatically.
    • when a connection error to gfsd is observed, remove the host entry from a scheduling cache.
    • when opening a file with the O_TRUNK flag, schedule a filesystem node regardless of the original location.
  • do not ignore SIGPIPE for applications that do not check return value in write(2). (Bugzilla-ja #68)
  • add more check against integer overflow.
• Documentation
  • improve all documents: add more detailed and more exact explanation.
• Bug fix
  • gfls(1) - fix a bug that a longer pathname than the screen width causes an error.
  • gfreg(1) - fix a bug in recursive mode that a directory name cannot be changed.
  • config-gfarm(1)
    • resolve a problem that postmaster fails to start up in some cases.
  • gfmd(8) - add description about -d option in usage display. (Bugzilla-ja #62)
  • gfsd(8) - fix a bug that errno is not preserved by SIGCHLD handler.
  • libgfarm
    • fix a bug such that gfs_client_terminate() does not remove connection entries from a connection cache. This may cause core dump in some calling sequences.
    • fix a bug that a file cannot be deleted if it has no fragment information.
    • fix a serious bug in scheduling which may cause libgfarm to hang up. (Bugzilla-ja #66)
  • fix a package dependency problem in gfarm.spec.

[2006.8.7] Version 1.3.1 is released

New Features

• gfchmod(1) - chmod command for Gfarm.

• gfusage(1) - display total disk space usage of each user.

• Documentation
  • KNOWN_PROBLEMS - describe known problems of Gfarm 1.3.1.

Update Features

• libgfarm
  • improve the functionality of connection cache for gfsd to tolerate restart of gfsd, and not to exhaust the file descriptors. Now, the number of cached connections is managed by the "gfsd_connection_cache" directive in gfarm.conf (default is 16).
  • add definition of gfarm_error_t and GFARM_ERR_NO_ERROR, and gfarm_error_string(3) function.
  • report an error in closing a connection.
  • devise a countermeasure against integer overflow attack by using malloc() and realloc() in a safe way.

• gfarm_agent(1)
  • add "-m" option for master mode in case that there is a single gfarm_agent metadata cache server running. This improves metadata access performance by caching path information also.

• config-gfarm(1)
  • add "-a " option for specifying an authentication type.
  • support the PostgreSQL 7.4 as a metadata server.
  • show usage with "--help" option.
  • set keepalive option by default to cope with poor networking environment.

• config-gfsd(1)
  • change spool-directory argument to be optional.
  • add "-l" option which specifies a listen address.
  • show usage with "--help" option.

• config-agent(1)
  • show usage with "--help" option.

• gfsd(8)
  • add "-l" option which specifies a listen address,
  • support read-only mode experimentally, which is enabled by creating ".readonly" file in the top spool directory by administrators.

• Documentation
  • describe about a file/directory owner problem in README.hook.

• Bug fix
  • fix a big scheduling bug such that local disk does not have priority to be scheduled, introduced by the Gfarm version 1.3.
  • fix a portability problem to x86_64.
  • fix a bug regarding reconnection to gfarm_agent.
  • gfsd(8) - fix disk space calculation on FFS on *BSDs.
  • fix a bug of mkdir(1) via system call hooks on Solaris 9.

Unsupported Features

• NetBSD 1.6.2 is not supported anymore.

[2006.6.12] Version 1.3 is released

Compatibility Note

• libgfarm uses new protocols for gfsd. If client library is updated to the version 1.3, gfsd needs to be updated to the version 1.3 and restarted.

New Features

• Solaris 9 and 10 syscall hooking support.

• AIX 5.2 partial support. AIX 5.2 is now supported except libgfs_hook.so syscall hooking library. It is needed to specify --without-gfshook in configure.

• gfarm_agent(1) - a cluster-wide metadata cache server. It has new functionality to share metadata cache among several PC nodes.

• config-agent(1) - new configuration script to set up gfarm_agent metadata cache server. For details, refer to INSTALL.RPM document.

• gfdump(1) - new metadata dump and restore tool;

  % gfdump -d -f dumpfile to dump metadata

  % gfdump -r -f dumpfile to restore metadata

  Note that host information is dumped by gfhost not by gfdump.

• gfs_statfsnode(3) and gfs_statfsnode_cached(3) - new functions to acquire free disk space.

• Environment variable
  • GFARM_WRITE_TARGET_DOMAIN environment variable specifies the domain name that has priority for new files to be created.
  • GFARM_WRITE_LOCAL_PRIORITY environment variable changes policy where a new file is created on a file system node. By default, local storage is always selected if enough disk space is available. On the other hand, if 0 or 'disable' is set to the environment variable, local storage is not always selected for better load balancing and disk capacity control.

• gfarm.conf - Gfarm configuration file
  • minimum_free_disk_space directive specifies the minimum free disk space for file system node scheduling. Any file system node with less free space than specified has low priority in selection. The default size is 128 MBytes.
  • write_local_priority directive changes policy where a new file is created on a file system node. See the description of GFARM_WRITE_LOCAL_PRIORITY environment variable above.
  • agent_serverhost and agent_serverport directives specify a gfarm_agent metadata cache server.
  • several directives for cache management.

Updated Features

• gfreg(1) - the -H and -D options can be specified in all modes

• configure - the --without-gfshook option is added to disable to build syscall hooking library

• gfsd(8) - multiple spool directories support by running multiple gfsds with a different virtual IP address on the same server host. Sanity check is added to detect input/output error.

• libgfarm - add access control for gfs_chdir, gfs_opendir, gfs_unlink, gfs_unlink_section, gfs_unlink_section_replica, and gfs_rename.

• gfs_fstat(3) - return current file size in section mode.

• Performance Improvement
  • config-gfarm(1) - use autovacuum functionality for PostgreSQL backend if supported.
  • do not re-calculate checksum at close time in any case.
  • implement new scheduling algorithm which uses RTT (Round Trip Time) and disk free space to make it possible to select near file replicas and near file system nodes and to avoid disk full.
  • reduce memory consumption when using a PostgreSQL backend.

• Robustness Improvement
  • try to re-connect when the connection to a gfarm_agent server is broken due to rebooting or some other reason.
  • set 5 second timeout in connect() to avoid slow response in unstable lossy networks.

• Bug fix
  • gfs_pio_create(3) and gfs_mkdir(3) can create a file or a directory with invalid mode.
  • gfls(1) - fix a segmentation fault problem.
  • ctime is not updated.
  • libgfarm (metadb_pgsql.c) - fix memory leak.
  • gfs_chmod(3) - fix a 'chmod +x' bug in OpenLDAP backend.
  • gfs_unlink(3) - fix a bug such that path information in metadata remains when unlinking a binary file that does not have the corresponding physical file.
  • config-gfarm(1) - make it robust for PostgreSQL installation.
  • fix a bug such that '.' and '..' can be registered incorrectly in metadata.
  • plug memory leak.
  • fix a segmentation fault when re-caching the directory cache managed by red-black tree.

[2006.1.20] Version 1.2.9 is release

New Features

• PostgreSQL support as a backend metadata database server.

Updated Features

• config-gfarm(1) - PostgreSQL support. Use password authentication for PostgreSQL and OpenLDAP backend by default.

• gfrep(1) - add the -m option for replica migration, and the -v option for verbose messages.

• gfps(1) - add the -v option to display GSI authentication error.

• gfs_unlink(3) - do not return error when at least one file replica is successfully unlinked.

• libgfarm - Gfarm library
  • Do not keep a GSI proxy credential since it may be expired.
  • Make netparam statement in gfarm.conf effective in on-demand replication.
  • Fault tolerent support in file replication.

• Performance Improvement
  • Directory cache is managed by a red-black tree instead of hash to improve performance in large number of files in a directory.
  • Host information cache and path information cache are added to reduce metadata access overhead.

• Bug fix
  • gfreg -f does not work. (Bugzilla #16)
  • gfsplck(1) - files with no fragment information cannot be fixed.
  • Invalid metadata is created when the corresponding physical file creation fails. (Bugzilla-ja #47)
  • gfrep -N creates more number of replicas than specified. (Bugzilla-ja #44)
  • gfrep(1) - unavailable host can be selected as a source host in replication.
  • gfs_chmod(3) - fix a bug when changing execution bits.

• Documentation
  • Gfarm-FAQ - add section 2.7: "Operation not permitted" error happens when accessing or creating a file having execution bits.
  • README.hook - update solaris section.

[2005.9.28] Version 1.2 PL4 is released

Updated Features

• config-gfarm(1) and config-gfsd(1) - generate startup scripts for SuSE and other Linux distributions.

• Environment variable
  • GFARM_SYSLOG_OUTPUT environment variable can specify a syslog facility. This is also applied for debug message generated by libgfs_hook_debug.so.

[2005.9.12] Version 1.2 PL3 is released

Updated Features

• config-gfarm(1) and config-gfsd(1) are now supported in all platforms.

• Bug fix
  • cannot create any file or directory in an empty Gfarm file system.

[2005.9.8] Version 1.2 PL2 is released

Updated Features

• gfrep(1) - the -S option is added to specify the source domainname.

• Bug fix
  • When opening a file in write mode during text file busy, it will be truncated. This bug was introduced in the version 1.2 RC2.

  • Syscall hooking library - fix a filename translation for '/gfarm/.filename' and '~filename' in Gfarm file system.

[2005.9.6] Version 1.2 PL1 is released

Updated Features

• config-gfarm - specify cachesize automatically when using Berkeley DB.

• Documentation
  • README.hook - simplify and reorganize the structure. add issues of possible segmentation fault by libgssapi collision.

  • gfarm_url_section_replicate_{,from_}to(3) and gfarm_url_fragments_replicate(3) - added

• Bug fix
  • gfrun(1) may fail when a job is submitted by an unregistered client via gfarm_agent.

  • gfexec(1) - fix uninitialized variable reference.

[2005.8.31] Version 1.2 RC2 is released

Updated Features

• UTF-8 support

• FreeBSD and NetBSD supports

• File locking support against replication during updating the file, which prevents from creating inconsistent file replicas.

• A newly created file can be accessed by other processes before it is closed.

• When copying, moving and removing executable files, only the target architecture section of the executable files is modified. This means you can install and remove executable binaries in different architecture independently or separately.

• gfarm.conf(5) - add client_architecture directive to specify a client architecture.

• Environment variable
  • GFARM_CONFIG_FILE environment variable can specify a Gfarm configuration file instead of ~/.gfarmrc. (Bugzilla-ja #38)

  • GFARM_ARCHITECTURE environment variable can specify a client architecture.

• configure - honors the --sysconfdir option to specify a directory of gfarm.conf.

• gfsd(8), gfmd(8) - accepts level N limited proxy to enable access to Gfarm file system from a process invoked by Globus jobmanager.

• gfrm(1) - supports recursive deletion of (file replicas of) a particular file section.

• gfkey(1) - the -p option is added to specify the term of validity in second.

• gfs_unlink_section(3) - new function to unlink a particular section of a file.

• gfs_pio_sync(3), gfs_pio_datasync(3) - new functions

• gfs_seekdir(3), gfs_telldir(3) - new functions

• Syscall hooking library
  • New syscall hook entries; fsync, fdatasync, pread, pread64, pwrite, and pwrite64.

  • lseek(2) hook supports a seek operation for a directory. (Bugzilla-ja #36, #37)

  • stat(2) hook fills 'st_blocks' in struct stat, which makes "du" command work.

• Robustness Improvement
  • gfarm_agent(1) - the functionality is enhanced to accept all metadata requests. When using gfarm_agent, client programs do not connect to an LDAP server, which reduces the number of connections to the LDAP server and CPU load of the LDAP server.

  • libgfarm(3) - tries to re-connect when the connection to an LDAP server is broken due to rebooting or some other reason.

• Documentation
  • samba-hook, samba-gfarmfs - describes how to export a Gfarm file system to Windows clients via samba.

  • nfs-gfarmfs - describes how to export a Gfarm file system in NFS.

  • export-gfarm - summarizes the status to export a Gfarm file system in various protocols including FTP, HTTP, CIFS, and NFS.

• Bug fix
  • fix a segfault problem or inappropriate error message problem when Gfarm I/O libraries are called before calling gfarm_initialize. (Bugzilla-ja #32)

  • fix a race condition when multiple processes fix an invalid metadata.

  • gfarm_initialize(3) disables the syscall hook library discussin in gfarm-discuss-ja mailing list.

  • '=' cannot be included in a filename.

  • gfreg creates a file whose name is '-' when 'cat file | gfreg - gfarm:dir' (bugzilla-ja #35)

  • a remote file opened by a process can be closed by the child process.

  • fix memory leak.

  • gfreg(1) - fix a bug related to the -r option.

  • fix a segfault problem when using an old tcsh and gfarm_agent.

  • unlink(2) hook cannot remove a file with no fragment information.

[2005.5.17] Version 1.1.1 is released

Updated Features

• gfarm_agent(1) - Gfarm filesystem metadata cache server. This dramatically improves performance and response time to access Gfarm file system. For details, see a manual page of gfarm_agent.

• gfreg(1) - the -r option is added to support recursive registration when directories or files are specified. The -p and -i options are added to specify a registration mode explicitly.

• gfrm(1) - supports recursive deletion of file replica. gfrm -h host -r dir.

• Configuration utilities for Debian packages;

  config-gfarm	set up metadata server and Gfarm configuration file.
  config-gfsd	set up file system node.

• Performance Improvement
  • Improve metadata cache management significantly.

  • Improve gfrun(1) performance.

• Syscall hooking library
  • New syscall hook entries; mknod and link.

  • Bug fixes; chown, lchown, fchown, setxattr, lsetxattr, and fsetxattr, and execve.

• Documentation
  • Gfarm-FAQ
    • Add how to execute gfarm_agent automatically only when it is not there?
    • Add how to delete invalid metadata when a file system node is crashed.
    • Add a comment for gfchmod and gfmv.

• Bug fix
  • A file can be created even under a file not a directory.

  • Fix 'segmentation fault' when reading, writing, and seeking a file in global file view.

  • A file '' is wrongly considered as the current working directory in Gfarm file system.

  • Fix 'segmentation fault' when renaming a directory.

  • gfrm deletes '.' or '..' when it is specified.

  • gfs_pio_truncate does not maintain file pointer.

  • Syscall hook library does not allow to access a large file.

  • gfreg cannot register a file via a socket.

  • gfsck cannot remove invalid metadata that does not have fragment information.

[2005.3.9] Version 1.1.0 is released

Updated Features

• Configuration utilities for RPM binary packages;

  config-gfarm	set up metadata server and Gfarm configuration file.
  config-gfsd	set up file system node.

  For details, see INSTALL.RPM.

• gfrep(1) - supports recursive replication when directories are specified, and supports the -N option that specifies the number of file replicas.

• gfs_rename(3) - supports to rename not only a file but also a directory.

• Syscall hooking library
  • New syscall hook entries; truncate, ftruncate, symlink, chown, lchown, fchown, setxattr, lsetxattr, and fsetxattr.

  • redirect support, i.e. cat a > /gfarm/~/a
    'sh ./configure' works well in Gfarm file system!

  • tcsh and zsh support. (old tcsh still has a problem when a command error happens.)

• Documentation

  INSTALL.RPM	Installation document using RPM binary packages

• Bug fix
  • Service certificate cannot be used when host cert or user proxy cert is not available.

  • vim cannot open a file when it is specified by a relative path.

  • 'gfrun ls' lists the contents in the physical spool directory.

  • open(file, O_WRONLY|O_TRUNC) fails when the file exists.

  • gfs_rename and syscall hook entry for rename. (Bugzilla #6)

  • Fix an oops when seeking beyond the buffer in SEEK_CUR.

  • Fix 'segmentation fault' when executing tcsh.

  • Fix 'no such object' error when accessing a new file or a new directory created by another application.

  • Fix 'invalid argument' error when reading and writing the same file.

  • gfsplck recognizes an incorrect section name when the filename includes ':'.

[2004.11.1] Version 1.0.4 is released

Updated Features

• gfs_pio_open(3), gfs_pio_create(3) - supports the read-write mode and the append mode.

• gfs_pio_create(3) - supports the exclusive flag.

• New client command; gfdf --- reports Gfarm file system disk space usage.

• New I/O functions; gfs_fchmod, gfs_rename, gfs_pio_truncate. Gfs_rename only supports to rename a file not a directory in this version.

• Global and local file view enhancement for executable files.

• gfs_chmod(3) - allows to change execution bits if it has only one file fragment.

• x86_64 and ia64 supports

• Shared library support

• gfrun - added the -b option to specify global file view as a default file view for programs linked with Gfarm syscall-hooking library.

• gfls - added the -a and -i options.

• gfwhere - added the -s option to display the file segment size.

• gfexec - shell script can be executed without "#!/bin/sh".

• gfrcmd - added the -N option to specify a target name of GSI authentication.

• gfsd - added new protocols to create a hard link, to rename a file, to truncate a file, and to get file system statistics.

• libgfarm - improved metadata cache management.

• Syscall hooking library
  • New syscall hook entries; fchmod, rename, dup, getxattr, lgetxattr, and fgetxattr.

  • '/gfarm' is now translated to 'gfarm:/' not 'gfarm:'. '/gfarm/~' and '/gfarm/.' specifies the home directory and the current working directory in Gfarm file system, respectively.

  • More support for bash. It is possible to change the current working directory by 'cd' such as 'cd /gfarm/~'. For more details, refer to README.hook.en document.

  • Shared libraries in Gfarm file system can be opened by specifying pathnames such as /gfarm/lib:/gfarm/~/lib in LD_LIBRARY_PATH.

• Bug fix
  • *SECURITY FIX* - In GSI authentication, the remote party does not authorized. This fix makes it possible to link Gfarm with the GSSAPI library in Globus Toolkit version 3.2.1 or later.

  • gfrm cannot remove a file that has invalid metadata. (Bugzilla #12)

  • gfimport_text creates an incomplete Gfarm file. [gfarm-discuss:00092]

  • gfsplck does not delete an invalid file when it is not a regular file. (Bugzilla-ja #29)

  • Include path for openssl is missing. (Bugzilla-ja #33)

  • Inactive local file system node may be selected. (Bugzilla-ja #31)

  • 'rm -r' cannot remove a directory (Bugzilla #13)

  • gfsplck registers a lock file by mistake. (Bugzilla-ja #18)

  • Fixed a segfault problem during parallel gsi_auth authentications.

  • Fixed a problem of on-demand replication failure.

  • Fixed a hang problem during scheduling file system nodes or gfhost.

  • Fixed a segfault problem when using syscall hooking library.

• Documentation

  README.hook.en	describes how to access Gfarm file system by existing binary programs without any modification as if it is mounted at /gfarm.

[2004.7.5] Version 1.0.3.1 is released

Updated Features

• Added new I/O functions; gfs_chmod.

• gfs_unlink, gfs_rmdir - it now checks the permission of the parent directory.

• gfrm - '-r' and '-R' options are added to remove the contents of directories recursively.

• Syscall hooking libraries
  • Added system call hook entries for directory operations; chmod, mkdir, rmdir, and utime.

  • '/gfarm' is now considered as a mount point for Gfarm fils system. This make '/gfarm:' and '/gfarm@' obsolete.

• Bug fix

  gfs_hook.o	Open(2) and stat(2) syscall hooks select the same file view. (Bugzilla-ja #26) Stat(2) returns 0 for files that are created but not closed (by the same process). Although this fix is not enough, Fortran program compiled by g77 can generate a Gfarm file. (Bugzilla-ja #28)

[2004.5.25] Version 1.0.3 is released

Updated Features

• Added new I/O functions; gfs_mkdir, gfs_rmdir, gfs_pio_gets, gfs_pio_readline, gfs_pio_readdelim.

• gfrmdir - it terminates very quickly. (Bugzilla-ja #20)

• gfstat - it also displays the information of assumed fragment number. (Bugzilla-ja #16)

• gfexec - The way to pass node number and other information to a gfarm process is changed again to be able to be inherited by any child process. This has a side effect that all gfarm applications need to be re-compiled. '-s', '-I', '-N' options are added.

• Bug fix

  libgfarm.a	Fixed a segfault problem especially when using syscall hooking library. the error message "Success" even when opening a file fails. Fixed a segfault bug in gfs_pio_fileno(). (Bugzilla-ja #25)
  gfs_hook.o	Fixed a bug such that select() results in timeout when checking readability of a file descriptor for a remote Gfarm file. Fixed a bug such that a parallel process cannot be executed correctly, which was introduced in the version 1.0.2. Fixed a bug when hooking fstat().

[2004.4.15] Version 1.0.2 is released

Updated Features

• Added a new client command; gfwhoami --- prints effective user name.

• Supported OpenLDAP-2.1.X or later (LDAPv3).

• Supported special characters; ' ', ',', '+', '"', '\', '<', '>', and ';' in file names. (bugzilla-ja #13)

• Syscall hooking libraries
  • Added system call hook entries for directory operations; dgetents, dgetents64, open, open64, close, stat, fstat, chdir, fchdir, getpwd, and access.
  • Modified semantics of '/gfarm:'. It is considered to be the root directory in Gfarm file system instead of a current working directory. Applications do not assume '/gfarm:' indicates a current working directory in Gfarm file system.
  • Changed the way to pass node number and other information to a gfarm process, when 'gfexec' command is used. You do not need to modify application source codes any more to utilize Gfarm file system. Just link with gfs_hook.o.
    This has a side effect that all gfarm applications need to be re-compiled. Or, you may be able to use "-S" option of "gfrun" command, if you cannot recompile gfarm applications.

• Bug fix

  libgfarm.a	Fixed a bug such that gfs_pio_set_view_*() occasionally returns the error message "Success" even when opening a file fails. Fixed a bug of default file view. (Bugzilla #4)
  gfs_hook.o	Plugged memory leak.

• Documentation

  INSTALL	Added the description related to OpenLDAP-2.1.X or later.
  Gfarm-FAQ	Added section for performance tuning.
  README.hook	Updated the description.

[2003.12.8] Version 1.0.1 is released

Updated Features

• Performance Improvement

  gfrun	Parallel program execution overhead is extremely reduced.

• Fault Tolerance Improvement
  • Inconsistency between a metadata and the corresponding physical file in a local spool is automatically fixed during on-demand replication.

• Servers
  • Added the -P option to specify a 'pid' file.

• Bug fix

  gfreg	Fixed a broken gfrep command included in version 1.0.
  gfexec	Closed all connections before execution of a program.
  gfrep	Fixed a bug such that the replication often fails with permission denied, introduced in 1.0b5.
  libgfarm.a	Fixed a race condition during on-demand replication. Fixed a bug such that a file with one-letter file name such as 'gfarm:a' cannot be created. Added an error check of existence of the current working directory.

• Documentation

  INSTALL	Updated the description.
  Gfarm-FAQ	Added limitations and frequent errors in Gfarm file system.

[2003.11.25] Version 1.0 is released

Updated Features

• Client commands
  • New command

    gfsck	administration command fixing inconsistency between file system metadata and physical file locations when disk crash happens.
    gfexec	executes Gfarm command on a file system node.
    gfstat	displays Gfarm file or directory status.

  • New features

    gfreg	A binary registration can also be registered by specifying the target hostname. (Bugzilla-ja #8) Supported '-' for an input file.
    gfrun	Added an existence check for a command. (Bugzilla-ja #12)

• Bug fix

  gfreg	Fixed a bug to leave an invalid metadata entry.
  gfps	Fixed a segfault problem when specifying invalid job-ids.
  gfhost	Fixed missing variable initialization which makes gfhost and scheduling functions hang.
  libgfarm.a	Fixed a critical bug related to global file view. (Bugzilla-ja #11) (gfs_utimes) Fixed a bug such that an uninitialized value will be set to ctimespec.

[2003.10.2] Version 1.0 beta 5 is released

Updated Features

• Parallel I/O Library
  • [New feature] Added new functions, gfs_glob, gfs_chdir, gfs_realpath.
  • [Fault tolerance] When the parent directory in a spool directory is missing, it is tried to be created.

• Bug fix

  gfmkdir, gfrmdir	Added the -h option for usage. (Bug 3 in Gfarm bugzilla-jp)
  gfrun	The current working directory is inherited to each parallel process.
  gfls	Fixed a multi-column display format bug with the -F option. (Bug 5 in Gfarm bugzilla-ja)
  libgfarm.a	Fixed a bug such that process's umask unexpectedly modifies the access permission in a spool directory. Fixed a memory leak. Fixed a bug for resolving a Gfarm path name.
  debian package	Added missing commands for manipulating directories.

[2003.9.5] Version 1.0 beta 4 is released

Updated Features

• Client commands
  • New command

    gfregister	administration command to register standard Gfarm parallel commands to a Gfarm filesystem.

  • New feature

    gfmpirun_p4	The executed process can be suspended or executed in the background.

• Bug fix

  libgfarm.a	Fixed a bug not to create a directory or a file at the top level. (Bug 1 in Gfarm bugzilla-ja) Fixed a bug not to delegate the user credential in the gsi_auth authentication mode. (Bug 2 in Gfarm bugzilla)

[2003.8.21] Version 1.0 beta 3 is released

Updated Features

• Client commands
  • New command
    gfcd, gfpwd, gfmkdir, gfrmdir
    gfront - Gfarm file system browser

    

  • New feature

    gfrun	Enhanced the host selection to schedule hosts which can run a specified program. Added -u and -g options to manually decide to pass gfarm specific command-line options or not. Added the -v option to display GSSAPI major and minor error status. Added the -p option to profile Gfarm parallel I/O APIs. Added the -r option for on-demand replication. Added the -I option to specify an index name explicitly. This option is effective only with the file-affinity scheduling.
    gfhost	Added a feature such that gfhost selects hosts that the user successfully logs on.
    gfrep	Supported a special case for replicating a Gfarm file with only one file fragment. In this case, the file can be replicated by specifying only the -d option.
    gfreg	Added the -D <domain> option to limit filesystem nodes within the specified domainname.
    gfsched	Added the -p <program> option to restrict the selection of hosts that can run the specified program. Added the -N <#nodes> option. Added the -U option to retain old behavior (i.e. do not check whether the user can log in or not.)
    gfrcmd	Added a functionality to execute .bashrc at login time. Added the -v option to display GSSAPI major and minor error status.
    gfmpirun_p4	Added -G <Gfarm file> and -N <# nodes> options.

• Parallel I/O Library
  • [Fault tolerance] When a file to be opened is missing for some reason, try to open another file replica.
  • [Fault tolerance] Only filesystem nodes the user successfully logged on are selected by every command.
  • [Performance] Employed better process scheduling.
  • [Performance] Improved the overhead of GSI authentication and the performance of data transfer with GSI.
  • [New feature] Host alias name can be specified as a hostname.

• Syscall hooking libraries
  • Added an entry to hook utime().

• Configuration file
  • '*' can also be specified in a host specification part that means all hosts. 'ALL' will be deprecated.

• Documentation

  README.hook	Added a section for C++ programs.
  INSTALL	Updated the description.

• Bug fix

  gfs_hook.o	Fixed a bug when hooking lseek64 on linux. Fixed a bug such that a Gfarm file is occasionally deleted.
  libgfarm.a	Added a missing error condition. Fixed memory leak at LDAP access that is not described in RFC1823. Fix an off-by-one error that may cause core dump.
  gfsd	Fixed a bug such that gfsd that executes a user process cannot terminate and consume 100% of CPU when the invoked gfrcmd is already terminated before that.

[2003.5.19] Version 1.0 beta 2 is released

Updated Features

• Client commands

  gfreg	A new feature is added to register multiple local files.
  gfrun	The executed process can be suspended or executed in the background.
  gfrcmd	A delegated credential is exported to be delegated by a invoked child process by gfrcmd.

• Servers (gfsd and gfmd)
  Information about authentication and disconnection is now logged.

• Parallel I/O Library
  Only available filesystem nodes are selected by every command. Node selection algorithm by load average is improved.

  Flags, GFARM_FILE_REPLICATE and GFARM_FILE_NOT_REPLICATE, for gfs_pio_{open,create} and gfs_pio_set_view_{local,index,global} are implemented.

• Documentation

  README.hook	An example of static compilation with the Globus library is added.
  INSTALL, GUIDE	Examples of the gfreg command are updated.
  Gfarm-FAQ	Deleted the item 2.1, and added a new item 2.1.

• Bug fix

  gfls	Changed an inappropriate error message from "not a directory" to "no fragment information".
  gfrep	A bug of buffer overrun is fixed.
  gfsd	Resource and memory leaks are plugged.
  gfmd	Resource and memory leaks are plugged. A minor race condition is fixed. Fix a bug which made gfmd core dump at authentication failure.

[2003.4.25] Version 1.0 beta 1 is released

Features

• Supported client commands

  file management	gfls, gfreg, gfrep, gfrm, gfwhere
  file utilities	gfexport, gfimport_text, gfimport_fixed
  scheduling & host management	gfsched, gfhost
  process management	gfrun, gfrcmd, gfmpirun_p4, gfps
  security	gfkey, digest

  Unsupported commands; gfmkdir, gfrmdir, gfchmod, gfchown, gfchgrp, gfcp, gfcd, gfchdir, gfpwd, gfdf, and gfsck.

• Gfarm filesystem
  Gfarm filesystem supports a Gfarm file that is a group of ranked files. Each file of a Gfarm file can be replicated and stored in any filesystem node.

• Gfarm filesystem daemon (gfsd)
  Gfsd has a capability of remote file operations, user authentication, file replication, remote program invocation, and node status monitoring.

• Gfarm job-management daemon (gfmd)
  Gfmd has a capability of job management for gfps.

• Gfarm filesystem metadata daemon (slapd)
  This implementation utilizes the slapd provided by the OpenLDAP software.

• Syscall hooking libraries
  Legacy applications can utilize a Gfarm filesystem just linking with a syscall-hooking library, gfs_hook.o or gfs_hook_mpi.o.

• Sample Gfarm parallel commands
  gfgrep - parallel grep
  gfwc - parallel word count
  gfcp - parallel file copy
  gfcombine - parallel file copy with reduced number of fragments by combining file fragments