Name

gfarm2.conf — Gfarm configuration file

DESCRIPTION

gfarm2.conf is a text file that contains a Gfarm configuration. Gfarm server processes gfmd and gfsd refer to /usr/local/gfarm/etc/gfmd.conf and /usr/local/gfarm/etc/gfarm2.conf, respectively, by default. Since this configuration file is only read at startup, it is necessary to restart servers when the contents of the configuration file are updated.

Application programs, such as gfls and gfhost, refer to both /usr/local/gfarm/etc/gfarm2.conf, and a file specified by an environment variable, GFARM_CONFIG_FILE. If both configuration files exist, the file specified by the environment variable, GFARM_CONFIG_FILE, is read first. Both files have the same grammar. If the environment variable GFARM_CONFIG_FILE doesn't exist, ~/.gfarm2rc in a user's home directory is used instead.

Each line of gfarm2.conf consists of one statement. When the line ends with the character ``\'', the line continues for the next line. A word beginning with ``#'' causes that word and all remaining characters on that line to be ignored.

Host_specification

Host_specification has the following forms.

III.JJJ.KKK.LLL

Specifies an IP address with four octets from 0 to 255, separated by ".".

III.JJJ.KKK.LLL/MM

Specifies a network address with an IP address and a netmask from 0 to 31 separated by "/".

domain.name

Specifies a host name.

.domain.name

Specifies all hosts which belong to the domain.name.

*

Specifies all hosts.

Statement

The following statements are supported.

spool directory

The spool statement specifies a spool directory for a Gfarm filesystem on this filesystem node.

For example,


spool /var/spool/gfarm

spool_server_listen_address IP-address

The spool_server_listen_address statement specifies the IP address at which the gfsd accepts TCP and UDP requests. The default address is all IP addresses of the host. This option is useful when one wants to invoke multiple gfsd to provide multiple spool directories on the host.

For example,


spool_server_listen_address 192.168.121.1

spool_server_listen_backlog number

This statement specifies the second parameter of the listen(2) system call that gfsd invokes. The default value is SOMAXCONN (typically 128, but depends on the OS) on OSes which define this symbol in <sys/socket.h>, or 5 if it's undefined.

On an environment which has many gfarm clients, it's recommened to increase this value after increasing system limit.

This parameter is only available in gfarm2.conf, and ignored in gfmd.conf.

For example,


spool_server_listen_backlog 128

spool_server_cred_type cred_type

This statement specifies the type of credential used by gfsd for GSI authentication. This is ignored when you are using sharedsecret authentication.

If this statement isn't used on the server side, the server uses a host certificate, if the server is invoked with root privileges. Or, if the server is invoked as a non-privileged user, the server uses the user's certificate.

If this statement isn't used on the client side, the client assumes that the server that the client is going to connect is using a host certificate of the server host. Thus, if the server is not invoked with root privileges, but invoked with user privileges where the user is the same as the user who invoked the client, the client side needs to specify the following one line.

Example:


spool_server_cred_type self

The possible types of cred_type are ``self'', ``host'', ``user'' and ``mechanism-specific''. And those are used with the spool_server_cred_service and spool_server_cred_name statements as follows:

self

This keyword specifies that the certificate that the user currently has is used. You must not use either the spool_server_cred_service or spool_server_cred_name statement, if you are using this type.

host

This keyword specifies that a host certificate or a service certificate is used. To choose a service certificate, the name of the service may be specified by the spool_server_cred_service statement. If ``host'' is specified as the service name, a host certificate in the file ``/etc/grid-security/hostcert.pem'' will be used. If any server name other than ``host'' is specified, a service certificate in the file ``/etc/grid-security/SERVICE/SERVICEcert.pem'' will be used. If the service name is omitted, ``host'' will be used as the service name by default. Only the Common Name field of a certificate will be used to check the server's identity for both a host certificate and a service certificate. And the Common Name field must be in the ``CN=SERVERNAME/HOSTNAME'' format. Also, the hostname must match the canonical name configured by the gfhost command exactly. Alias hostnames are not allowed.

This feature corresponds to the GSS_C_NT_HOSTBASED_SERVICE feature in GSSAPI (RFC2743/RFC2744).

Example:


spool_server_cred_type host
spool_server_cred_service host

user

This keyword specifies that a user certificate is used. The account name of the user may be specified by the spool_server_cred_name statement. If the account name is omitted, the user who invoked the command will be used by default. You must not specify a service name using the spool_server_cred_service statement, if you are using a user certificate.

To map from the account name to a Distinguished Name of a certificate, file ``/etc/grid-security/grid-mapfile'' is used. Thus, if there isn't such a file, or if the user isn't specified in this file, this feature cannot be used.

This feature corresponds to the GSS_C_NT_USER_NAME feature in GSSAPI (RFC2743/RFC2744).

Example:


spool_server_cred_type user
spool_server_cred_name guest

mechanism-specific

This keyword specifies that spool_server_cred_name is treated as a raw X.509 Distinguished Name serving as a server's certificate. You must not specify a service name using a spool_server_cred_service statement, if you are using this type.

This feature corresponds to a case where GSS_C_NO_OID is specified as a Name Type in GSSAPI (RFC2743/RFC2744).

Example:


spool_server_cred_type mechanism-specific
spool_server_cred_name "/O=Grid/O=Globus/OU=example.com/CN=John Smith"

spool_server_cred_service cred_service

This statement specifies the service name of a service certificate used by gfsd for GSI authentication, when ``host'' is specified in spool_server_cred_type statement. This is ignored when you are using sharedsecret authentication. Please read the description of the spool_server_cred_type statement for details.

spool_server_cred_name cred_name

This statement specifies the setting of a certificate used by gfsd for GSI authentication. What this setting means depends on the type specified in the spool_server_cred_type statement. This is ignored when you are using sharedsecret authentication. Please read the description of the spool_server_cred_type statement for details.

spool_check_level level

This statement enables the consistency check of files stored in the spool directory at start-up of gfsd. The level name corresponds how many -c command-line options are specified to gfsd, i.e. the level "display" means the -c option, "delete" means the -cc option, and "lost_found" means the -ccc option, which is the default. For detail about the -c option, refer to the manual page of gfsd(8). The level "disable" disables the consistency check.

For example,


spool_check_level disable

spool_base_load load-average

This statement specifies the CPU load adding to the real CPU load. This option is only available for a gfsd node (or a file system node). The default base load is 0.0. This option can be used to pretend to be busy or idle to change the scheduling priority.

For example,


spool_base_load 1.0

metadb_server_host hostname

The metadb_server_host statement specifies the host name on which gfmd is running.

This statement cannot be omitted.

For example,


metadb_server_host ldap.example.com

metadb_server_port port

The metadb_server_port statement specifies the tcp port number the gfmd is listening on. The default port number is 601.

For example,


metadb_server_port 601

metadb_server_cred_type cred_type

This statement specifies the type of credential used by gfmd for GSI authentication. This is ignored when you are using sharedsecret authentication. Please read the description of the spool_server_cred_type statement on the configuration of this statement.

metadb_server_cred_service cred_service

This statement specifies the service name of a service certificate used by gfmd for GSI authentication, when ``host'' is specified in metadb_server_cred_type statement. This is ignored when you are using sharedsecret authentication. Please read the description of the spool_server_cred_type statement on the configuration of this statement.

metadb_server_cred_name cred_name

This statement specifies the setting of a certificate used by gfmd for GSI authentication. What this setting means depends on the type specified in the metadb_server_cred_type statement. This is ignored when you are using sharedsecret authentication. Please read the description of the spool_server_cred_type statement on the configuration of this statement.

metadb_server_listen_backlog number

This statement specifies the second parameter of the listen(2) system call that gfmd invokes. The default value is SOMAXCONN (typically 128, but depends on the OS) on OSes which define this symbol in <sys/socket.h>, or 5 if it's undefined.

This parameter is only available in gfmd.conf, and ignored in gfarm2.conf.

For example,


metadb_server_listen_backlog 128

metadb_server_max_descriptors number

This directive specifies the number of file descriptor limit for the gfmd metadata server process. If not specified, the default limit is min(OS's hard limit, 131072). You may have to increase this value on a larger configuration, or decrease on a smaller configuration. See Gfarm FAQ to see how to calculate this value.

This parameter is only available in gfmd.conf, and ignored in gfarm2.conf.

For example,


metadb_server_max_descriptors 262144

metadb_server_stack_size bytes

This directive specifies the size of each thread in the gfmd metadata server process. If not specified, the default size of the OS is used. This parameter is used to cut down the size of virtual memory space used by gfmd.

For example, the default stack size on CentOS 5/i386 is 10MB, thus, you can decrease the size of the virtual memory space of gfmd to 1/40, by specifying 256KB as this parameter.

This parameter is only available in gfmd.conf, and ignored in gfarm2.conf.

For example,


metadb_server_stack_size 262144

metadb_server_thread_pool_size size

This directive specifies the maximum number of threads in a thread pool in the gfmd. It is effective to specify around the number of CPU cores of the metadata server node. Default is 16.

This parameter is only available in gfmd.conf, and ignored in gfarm2.conf.

For example,


metadb_server_thread_pool_size 16

metadb_server_job_queue_length length

This directive specifies the length of job queue in the gfmd. It is effective to specify around the maximum number of clients that access the Gfarm file system at the same time. Default is 16000.

This parameter is only available in gfmd.conf, and ignored in gfarm2.conf.

For example,


metadb_server_job_queue_length 160

metadb_server_heartbeat_interval seconds

This directive specifies the interval of heartbeat in seconds for gfmd to check availability of each gfsd. Default is 180 seconds.

Until gfarm-2.3.0, this parameter was only available in gfmd.conf, and ignored in gfarm2.conf. But since gfarm-2.4.0, gfsd also uses this parameter to detect whether gfmd is down or not, this parameter has to be specified in both gfarm2.conf and gfmd.conf.

For example,


metadb_server_heartbeat_interval 180

metadb_server_dbq_size size

This directive specifies the queue length of metadata updates for a backend database in gfmd. Longer queue length may avoid slow down due to waiting backend database updates in case of frequent metadata operations. Default is 65536.

This parameter is only available in gfmd.conf, and ignored in gfarm2.conf.

For example,


metadb_server_dbq_size 65536

ldap_server_host hostname

The ldap_server_host statement specifies the host name on which an LDAP server is running. This statement is required when the LDAP server is used for a backend database of gfmd. This statement is used in gfmd.conf not gfarm2.conf.

For example,


ldap_server_host ldap.example.com

ldap_server_port port

The ldap_server_port statement specifies the tcp port number of the LDAP server.

This statement cannot be omitted if ldap_server_host is specified.

For example,


ldap_server_port 602

ldap_base_dn LDAP_base_distinguished_name

The ldap_base_dn statement specifies the base-distinguished name of the LDAP database.

This statement cannot be omitted if ldap_server_host is specified.

For example,


ldap_base_dn "dc=example, dc=com"

ldap_bind_dn LDAP_bind_distinguished_name

The ldap_bind_dn statement specifies the distinguished name for the bind operation which is used for authentication to the LDAP database.

For example,


ldap_bind_dn "cn=gfarmuser, dc=example, dc=com"

ldap_bind_password password

The ldap_bind_password statement specifies the password for the bind operation which is used for authentication to the LDAP database.

For example,


ldap_bind_password "secret-ldap-password"

postgresql_server_host hostname

The postgresql_server_host statement specifies the host name on which a PostgreSQL server is running. This statement is required when the PostgreSQL server is used for a backend database of gfmd. This statement is used in gfmd.conf not gfarm2.conf.

For example,


postgresql_server_host postgresql.example.com

postgresql_server_port port

The postgresql_server_port statement specifies the tcp port number of the PostgreSQL server.

This statement cannot be omitted if postgresql_server_host is specified.

For example,


postgresql_server_port 602

postgresql_dbname dbname

The postgresql_dbname statement specifies the database name of the PostgreSQL database.

This statement cannot be omitted if postgresql_server_host is specified.

For example,


postgresql_dbname gfarm

postgresql_user user

The postgresql_user statement specifies the username used to connect the PostgreSQL database.

For example,


postgresql_user gfarm

postgresql_password password

The postgresql_password statement specifies the password used to connect the PostgreSQL database.

For example,


postgresql_password gfarm

postgresql_conninfo connection_info

The postgresql_conninfo statement specifies the connection option used to connect the PostgreSQL database.

For example,


postgresql_conninfo "sslmode=require connect_timeout=30"

auth validity method Host_specification

This statement specifies the authentication method when communicating with the host(s) specified by the third argument.

The first argument should be either the enable or disable keyword. The second argument, auth method, should be the gsi, gsi_auth, or sharedsecret keyword. The third argument specifies the host(s) by using Host specification.

The auth statement may be specified any number of times. For each authentication method, it becomes a candidate when the first entry whose host_specification matches the target host has the enable keyword. When there is no corresponding entry, or when the first corresponding entry has the disable keyword, the authentication method does not become a candidate.

This process takes place on both client and server sides. Candidates for authentication method on both sides will be tried.

The order of statements with different authentication methods is not relevant. When there are several candidates for the authentication method for the host, the order of the authentication trial is sharedsecret, gsi_auth, and then gsi.

The GSI methods are available if and only if the --with-globus option is specified at configuration. When the methods are not available, an auth statement with gsi or gsi_auth will be ignored.

This statement cannot be omitted.

For example,


auth disable sharedsecret 192.168.0.100
auth disable sharedsecret 192.168.0.101
auth enable sharedsecret 192.168.0.0/24
auth enable gsi_auth 10.0.0.0/8
auth enable gsi *

In this example, all hosts which belong to the network address 192.168.0.0/24, except for two hosts, 192.168.0.100 and 192.168.0.101, will be tested for authenticated by both sharedsecret and gsi; all hosts which belong to the network address 10.0.0.0/8 will be tested for authentication by both gsi_auth and gsi; and all other hosts will be authenticated by gsi. Note that two hosts, 192.168.0.100 and 192.168.0.101, will be tested for authentication by gsi only.

sockopt option[=value] [LISTENER | Host_specification]

The sockopt parameter specifies the socket option option via the setsockopt(2) system call.

When LISTENER (all capital letters) is specified by the second argument, the socket option is applied to any socket on the server side (accepting side).

When the host_specification is specified by the second argument, the socket option is applied to sockets that connect to the specified host(s). If the second argument is "*", the socket option is applied to any hosts on the client side (connecting side).

If the second argument is omitted, the socket option is applied to every socket.

The following socket options can be specified.

debug. The SO_DEBUG socket option is specified. A value is not necessary.

keepalive. The SO_KEEPALIVE socket option is specified. A value is not necessary.

sndbuf. The SO_SNDBUF socket option is specified with a value.

rcvbuf. The SO_RCVBUF socket option is specified with a value.

tcp_nodelay. The TCP_NODELAY socket option is specified. A value is not necessary.

For example,


sockopt tcp_nodelay 192.168.0.0/24
sockopt sndbuf=1048576 10.0.0.0/8
sockopt sndbuf=1048576 LISTENER
sockopt rcvbuf=1048576 10.0.0.0/8
sockopt rcvbuf=1048576 LISTENER

known_network Host_specification

The known_network statement specifies a network address for file system nodes. It is used to group file system nodes at file systen node scheduling. File system nodes that are not specified in this directive are assumed to be in an IPv4 class C network.

Multiple networks can be specified as follows.


known_network 192.168.0.0/25
known_network 192.168.0.128/25

network_receive_timeout seconds

The network_receive_timeout directive specifies the time, in seconds, to wait for data reception via network. The default value is 60 seconds. This directive is effective in both gfarm2.conf and gfmd.conf.

When a Gfarm application program receives no data from a gfsd filesystem server during the timeout period, it aborts the communication with the server, and then it tries to communicate with another server to continue the operation.

When a gfmd metadata server receives no data from a gfsd filesystem server during the timeout period, it aborts the communication with the gfsd server and gfmd considers the gfsd server is down.

admin_user user

This directive specifies an administrator user name, which is specified in gfmd.conf.

admin_user_gsi_dn user_gsi_dn

This directive specifies a subject DN of an administrator, which is specified in gfmd.conf.

local_user_map user-map-file [hostname:[port]]

This directive specifies a file name user-map-file for mapping local user names to global user names. This map file is used only for sharedsecret authentication. When this file is not specified, a global user name is assumed to be same as the local user name.

user-map-file is needed when you have to use the sharedsecret authentication method in the case where you have different unix account names on different filesystem nodes. In such a case, the user-map-file on each filesystem node should have an entry from each local user name to a unique global user name.

Example:


local_user_map /etc/gfarm/gfarm-usermap

Each line of the user-map-file consists of two fields separated by spaces; the first field is a global user name, and the second field is a local user name.

Example of the user mapping file:


foobar foo
quux baz

According to the first line of this mapping file, a global user name, "foobar", is mapped to a local user name, "foo", on this node.

When hostname/port of the meta data server is specified, user mapping files different in each server are applied. When the port is omitted, default port (601) is used. local_user_map can be specified two or more times. When the same definitions of local user name exist, the first definition is selected.

Example of multiple user mapping file:


local_user_map /etc/gfarm/gfarm-usermap1-1
local_user_map /etc/gfarm/gfarm-usermap1-2
local_user_map /etc/gfarm/gfarm-usermap2-1 server2
local_user_map /etc/gfarm/gfarm-usermap2-2 server2

In this example, when connecting to server2, it applies in order of gfarm-usermap2-1 and gfarm-usermap2-2. When connecting to excluding server2, it applies in order of gfarm-usermap1-1 and gfarm-usermap1-2.

local_group_map group-map-file [hostname:[port]]

This directive specifies a file name group-map-file for mapping global group names to local group names. This map file is used by legacy clients that use local group id such as gfarm2fs and gfarm dsi for Globus GridFTP to display mapped local groups. When this file is not specified, a local group name is assumed to be same as the global group name.

Example:


local_group_map /etc/gfarm/gfarm-groupmap

Each line of the group-map-file consists of two fields separated by spaces; the first field is a global group name, and the second field is a local group name.

When hostname/port of the meta data server is specified, group mapping files different in each server are applied. The specification method is similar to local_user_map.

schedule_cache_timeout seconds

This directive specifies the time (in seconds) until the cache used for filesystem node scheduling expires. The cache holds information on each filesystem node, e.g. load average, disk free space, and whether authentication succeeds or not. The default time is 600 seconds, i.e. ten minutes.

For example,


schedule_cache_timeout 60

schedule_concurrency number

This directive specifies the number how many filesystem nodes will be searched concurrently at filesystem scheduling. If this parameter is increased, the scheduling will finish quicker, but the load to filesystem nodes will increase. The default number is 10 nodes.

For example,


schedule_concurrency 10

schedule_concurrency_per_net number

This directive specifies the number how many filesystem nodes will be used in same network at measuring network latency for each network of filesystem nodes. It may be better to increase this parameter, if many filesystem nodes are stopping, or many filesystem nodes are under higher load. If this parameter is increased, it's recommened to increase schedule_concurrency parameter too. The default number is 3.

For example,


schedule_concurrency_per_net 3

schedule_idle_load_thresh load-average

This directive specifies the threshold of CPU load average to be considered idle. The file system nodes whose CPU load average is equal to or below the specified CPU load average are to be scheduled at first. The default load average is 0.1.

For example,


schedule_idle_load_thresh 0.1

schedule_busy_load_thresh load-average

This directive specifies the threshold of CPU load average to be considered busy. The file system nodes whose CPU load average is above the specified CPU load average are to be scheduled lastly. The default load average is 0.5.

For example,


schedule_busy_load_thresh 0.5

schedule_virtual_load load-average

This directive specifies the virtual CPU load average. The virtual CPU load is added when the host is scheduled to avoid scheduling the same host multiple times. The default load average is 0.3.

For example,


schedule_virtual_load 0.3

schedule_candidates_ratio ratio

This parameter specifies the threshold ratio of number of filesystem nodes from number of scheduling nodes, to decide to discontinue to search more filesystem nodes which load averages are less or equal to the schedule_busy_load_thresh parameter. If this parameter is increased, the accuracy of scheduling will be increased, but scheduling process will become slower. The default ratio is 4.0.

For example,


schedule_candidates_ratio 4.0

schedule_rtt_thresh_diff microseconds

This parameter specifies the threshold time of network latency which divides the domains of scheduling priority. If a network latency difference from nearer network is larger than this value, the network's scheduling priority is lowered. There is a similar parameter schedule_rtt_thresh_ratio, and the scheduling priority is lowered if the network latency exceeds one of the parameters. The default is 1000 microseconds. i.e. 1 millisecond.

For example,


schedule_rtt_thresh_diff 1000

schedule_rtt_thresh_ratio ratio

This parameter specifies the threshold ratio of network latency which divides the domains of scheduling priority. If a network latency is larger than this ratio from nearer network, the network's scheduling priority is lowered. There is a similar parameter schedule_rtt_thresh_diff, and the scheduling priority is lowered if the network latency exceeds one of the parameters. The default ratio is 4.0.

For example,


schedule_rtt_thresh_ratio 4.0

schedule_rtt_thresh ratio

This directive is just an alias of the schedule_rtt_thresh_ratio directive since gfarm-2.5.8. Please use that directive instead. This directive remains for compatibility with older versions. The default ratio is 4.0.

For example,


schedule_rtt_thresh 4.0

write_local_priority validity

Gfarm nearly always chooses localhost, when it selects a filesystem node to write a file. By specifing the disable keyword in this directive, Gfarm handles localhost the same as other filesystem nodes. The default is enable.

For example,


write_local_priority disable

write_target_domain domain

Gfarm tries to choose a filesystem node in the domain specified by this directive, when it selects a node to write a file. If there is no suitable node for writing in this domain, or if this directive is not specified, the filesystem node will be chosen from among all nodes. Note that if you specify this directive more than once, only the first one will be used and the others are ignored.

For example,


write_target_domain example.org

minimum_free_disk_space bytes

This directive specifies free disk space (in bytes) which is required on filesystem nodes. The Gfarm scheduler excludes filesystem nodes which have less free space than this parameter, when it schedules nodes for jobs which may write files. The free space value may have a suffix like ``k'' (kilo bytes), ``M'' (mega bytes), ``G'' (giga bytes) and ``T'' (tera bytes). The default size is 128M bytes.

This directive has to be specified in both gfarm2.conf and gfmd.conf. After restarting the gfmd and the Gfarm client, it is effective.

For example,


minimum_free_disk_space 1G

simultaneous_replication_receivers number

This directive specifies maximum number of simultaneous gfmd-initiated replications to same host. The default is 20.

For example,


simultaneous_replication_receivers 40

gfsd_connection_cache number

This directive specifies maximum number of cached gfsd connections. The default is 16.

For example,


gfsd_connection_cache 32

xmlattr_size_limit bytes

This directive specifies maximum size of XML extended attributes. The default is 786432, and numbers greater than 983040 are not allowed.

This parameter is only available in gfmd.conf, and ignored in gfarm2.conf.

For example,


xmlattr_size_limit 786432

xattr_size_limit bytes

This directive specifies maximum size of extended attributes. The default is 65536, and numbers greater than 983040 are not allowed.

This parameter is only available in gfmd.conf, and ignored in gfarm2.conf.

For example,


xattr_size_limit 65536

attr_cache_limit number

This directive specifies maximum number of cached attributes in gfarm library. The default is 40000.

For example,


attr_cache_limit 100000

attr_cache_timeout milliseconds

This directive specifies maximum time until cached attributes expire in milliseconds. The default is 1000, i.e. 1 second.

For example,


attr_cache_timeout 3600000

log_level priority_level

This directive specifies a level of log priority. The log output, which priority is inferior to this level, will not be sent to syslog or standard error. The priority levels are "emerg", "alert", "crit", "err", "warning", "notice", "info" and "debug" in highest first order. The default level is "info". It's not recommended to specify a level higher or equal to "crit".

For example,


log_level debug

log_message_verbose_level level

This directive specifies how verbose the log message is. The default value is 0, which outputs the log message id. The level 1 additionally outputs the file name and the line of source code. The level 2 additionally outputs the function name.

For example,


log_message_verbose_level 1

log_auth_verbose validity

When "enable" is specified, verbose messages about authentication are logged.

Example:


log_auth_verbose enable

file_trace validity

When "enable" is specified, an audit log feature for file creation, modification and removal is enabled.

This directive is available in both gfarm2.conf and gfmd.conf.

Here is an example:


file_trace enable

fatal_action action_name

This parameter specifies the behavior at a fatal error. The default action is backtrace_and_abort.

For example,


fatal_action backtrace_and_abort

The following actions are supported:

backtrace_and_abort

Print a backtrace, and then call abort(3) .

backtrace_and_exit

Print a backtrace, and then call exit(3) .

abort

Call abort(3) .

exit

Call exit(3) .

debug_command command_line

When a Gfarm application program crashes (received SIGSEGV, SIGBUG, etc.), a debug command specified in this directive is executed. For example,


debug_command env DISPLAY=:0.0 xterm -e gdb %e %p

"%e" and "%p" are expanded to an actual program name and a process ID, respectively. If you'd like to use "%" character itself in the command line, type "%%".

This directive is available in both gfarm2.conf and gfmd.conf. If this directive is omitted, no debug command is executed.

no_file_system_node_timeout seconds

If there is no file system node available, Gfarm client library periodically tries to find a file system node. This directive specifies the timeout to try in seconds. The default is 30 seconds.

For example,


no_file_system_node_timeout 30

gfmd_reconnection_timeout seconds

If the connection to the metadata server is disconnected, Gfarm client library periodically tries to reconnect. This directive specifies the timeout to try in seconds. The default is 30 seconds.

For example,


gfmd_reconnection_timeout 30

atime type

When "disable" is specified, atime (access time) is not updated. When "relative" is specified, atime is only updated if the previous atime is earlier than the current mtime (modify time) or ctime (change time), or if the previous atime is more than 1 day old. When "strict" is specified, atime is always updated. The default setting is "relative".

This parameter is only available in gfmd.conf.

For example,


atime disable

client_file_bufsize bytes

This directive specifies the file buffer size in bytes used in the Gfarm client library (libgfarm). The default size is 1048568 bytes (= 1MiB - 8B). The file buffer size should be less than the default value not for deterioration of remote access performance for version 2.5.0 or earlier.

For example,


client_file_bufsize 65536

client_parallel_copy num-of-parallel

This directive specifies the number of parallel for gfprep and gfpcopy to work in parallel. When this number is not specified, the default number of parallel is 4.

For example,


client_palallel_copy 10

profile validity

When "enable" is specified, Gfarm outputs the profile information.

Example:


profile enable

metadb_server_list gfmd-hostname[:port] ...

This parameter specifies connection destinations of redundant metadata servers delimited by space. If port is not specified, port will be 601 which is the default value.

This parameter is only available in gfarm2.conf, and ignored in gfmd.conf.

Example:


metadb_server_list mds1 mds2:701

metadb_replication validity

When "enable" is specified, the redundant metadata feature is enabled. The default value is disable.

This parameter is only available in gfmd.conf.

Example:


metadb_replication enable

synchronous_replication_timeout seconds

This parameter specifies timeout for executing synchronous replication in seconds. If the timeout period passed, the connection to the slave gfmd where replicates to will disconnect. The replication process in the master gfmd for the disconnected slave gfmd stops until the slave gfmd connects. The default is 10.

This parameter is only available in gfmd.conf.

Example:


synchronous_replication_timeout 30

synchronous_journaling validity

When "enable" is specified, fdatasync is called after update information is saved to the journal file. The timing of calling fdatasync is after writing the records from transaction-begin to transaction-end in one transaction. The default is "enable".

This parameter is only available in gfmd.conf.

Example:


synchronous_journaling disable

metadb_server_force_slave validity

When "enable" is specified, even if the gfmd is set to default master, it run as slave gfmd forcedly. The default is "disable".

NOTE: Please do not use this feature for now, because this implementation is not complete.

This parameter is only available in gfmd.conf.

Example:


metadb_server_force_slave enable

metadb_server_slave_max_size number

This parameter specifies the maximum number of slave gfmds. The default is "32".

This parameter is only available in gfmd.conf.

Example:


metadb_server_slave_max_size 40

metadb_journal_dir directory

This parameter specifies the directory for the journal file. The value of this parameter is specified at config-gfarm execution . When this parameter is not specified, the gfmd does not start.

This parameter is only available in gfmd.conf.

Example:


metadb_journal_dir /var/gfarm-metadata/journal

metadb_journal_max_size file_size

This parameter specifies the max size of the journal file in bytes. Beware that the larger the size is, the longer it takes time for the gfmd to complete starting. The default is 33554432 (32 MBytes).

This parameter is only available in gfmd.conf.

Example:


metadb_journal_max_size 8388608

metadb_journal_recvq_size size

This parameter specifies the maximum number of on-memory records of the journal data that the slave gfmd receives from the master gfmd. If it reaches this value, the slave gfmd gives up and exits, because the gfmd thinks that it's too slow to apply the journal data to the backend database. The default value is 100000.

This parameter is only available in gfmd.conf.

Example:


metadb_journal_recvq_size 100000

replica_check validity

When "enable" is specified, the replica_check system in gfmd can check and fix the number and placement of file replicas automatically. The replica_check works only when necessary. The default value is "enable".

This parameter is only available in gfmd.conf.

Example:


replica_check disable

replica_check_host_down_thresh seconds

This statement specifies an interval to start replica check after a gfsd is down. The default value is 10800 seconds (3 hours).

This parameter is only available in gfmd.conf.

Example:


replica_check_host_down_thresh 3600

replica_check_sleep_time nanoseconds

This statement specifies the sleep time in nanoseconds during the replica check when gfmd is busy. The default value is 100000 nanoseconds (0.1 millisecond). The maximum value is 1000000000 nanoseconds (1 second).

This parameter is only available in gfmd.conf.

Example:


replica_check_sleep_time 0

replica_check_minimum_interval seconds

This statement specifies the minimum interval in seconds, to start the next replica check. The default value is 10 seconds.

This parameter is only available in gfmd.conf.

Example:


replica_check_minimum_interval 5

GRAMMAR

This is a grammar of gfarm2.conf described by the BNF notation.

<statement> ::=

<spool_statement> |
<spool_server_listen_address_statement> |
<spool_server_listen_backlog_statement> |
<spool_server_cred_type_statement> |
<spool_server_cred_service_statement> |
<spool_server_cred_name_statement> |
<spool_check_level_statement> |
<spool_base_load_statement> |
<metadb_server_host_statement> |
<metadb_server_port_statement> |
<metadb_server_cred_type_statement> |
<metadb_server_cred_service_statement> |
<metadb_server_cred_name_statement> |
<metadb_server_listen_backlog_statement> |
<metadb_server_max_descriptors_statement> |
<metadb_server_stack_size_statement> |
<metadb_server_thread_pool_size_statement> |
<metadb_server_job_queue_length_statement> |
<metadb_server_heartbeat_interval_statement> |
<metadb_server_dbq_size_statement> |
<ldap_server_host_statement> |
<ldap_server_port_statement> |
<ldap_base_dn_statement> |
<ldap_bind_dn_statement> |
<ldap_bind_password_statement> |
<postgresql_server_host_statement> |
<postgresql_server_port_statement> |
<postgresql_dbname_statement> |
<postgresql_user_statement> |
<postgresql_password_statement> |
<postgresql_conninfo_statement> |
<auth_statement> |

<sockopt_statement> |



<known_network_statement> |
<network_receive_timeout_statement> |
<admin_user_statement> |
<admin_user_gsi_dn_statement> |
<local_user_map_statement> |
<local_group_map_statement> |
<schedule_cache_timeout_statement> |
<schedule_concurrency_statement> |
<schedule_concurrency_per_net_statement> |
<schedule_idle_load_thresh_statement> |
<schedule_busy_load_thresh_statement> |
<schedule_virtual_load_statement> |
<schedule_candidates_ratio_statement> |
<schedule_rtt_thresh_diff_statement> |
<schedule_rtt_thresh_ratio_statement> |
<schedule_rtt_thresh_statement> |
<write_local_priority_statement> |
<write_target_domain_statement> |
<minimum_free_disk_space_statement> |
<simultaneous_replication_receivers_statement> |
<gfsd_connection_cache_statement> |
<xmlattr_size_limit_statement> |
<xattr_size_limit_statement> |
<attr_cache_limit_statement> |
<attr_cache_timeout_statement> |
<log_level_statement> |
<log_message_verbose_level_statement> |
<log_auth_verbose_statement> |
<file_trace_statement> |
<fatal_action_statement> |
<debug_command_statement> |

<atime_statement> |
<client_file_bufsize_statement> |
<client_parallel_copy_statement> |
<profile_statement> |
<metadb_server_list_statement> |
<metadb_replication_statement> |
<synchronous_replication_timeout_statement> |
<synchronous_journaling_statement> |
<metadb_server_force_slave_statement> |
<metadb_server_slave_max_size_statement> |
<metadb_journal_dir_statement> |
<metadb_journal_max_size_statement> |
<metadb_journal_recvq_size_statement> |
<replica_check_statement> |
<replica_check_host_down_thresh_statement> |
<replica_check_sleep_time_statement> |
<replica_check_minimum_interval_statement>

<spool_statement> ::=

"spool" <pathname>

<spool_server_listen_address_statement> ::=

"spool_server_listen_address" <ipv4_address>

<spool_server_listen_backlog_statement> ::=

"spool_server_listen_backlog" <number>

<spool_server_cred_type_statement> ::=

"spool_server_cred_type" <cred_type>

<spool_server_cred_service_statement> ::=

"spool_server_cred_service" <cred_service>

<spool_server_cred_name_statement> ::=

"spool_server_cred_name" <cred_name>

<spool_check_level_statement> ::=

"spool_check_level" <spck_level>

<spool_base_load_statement> ::=

"spool_base_load" <load>

<metadb_server_host_statement> ::=

"metadb_server_host" <hostname>

<metadb_server_port_statement> ::=

"metadb_server_port" <portnumber>

<metadb_server_cred_type_statement> ::=

"metadb_server_cred_type" <cred_type>

<metadb_server_cred_service_statement> ::=

"metadb_server_cred_service" <cred_service>

<metadb_server_cred_name_statement> ::=

"metadb_server_cred_name" <cred_name>

<metadb_server_listen_backlog_statement> ::=

"metadb_server_listen_backlog" <number>

<metadb_server_max_descriptors_statement> ::=

"metadb_server_max_descriptors" <number>

<metadb_server_stack_size_statement> ::=

"metadb_server_stack_size" <number>

<metadb_server_thread_pool_size_statement> ::=

"metadb_server_thread_pool_size" <number>

<metadb_server_job_queue_length_statement> ::=

"metadb_server_job_queue_length" <number>

<metadb_server_heartbeat_interval_statement> ::=

"metadb_server_heartbeat_interval" <number>

<metadb_server_dbq_size_statement> ::=

"metadb_server_dbq_size" <number>

<ldap_server_host_statement> ::=

"ldap_server_host" <hostname>

<ldap_server_port_statement> ::=

"ldap_server_port" <portnumber>

<ldap_base_dn_statement> ::=

"ldap_base_dn" <string>

<ldap_bind_dn_statement> ::=

"ldap_bind_dn" <string>

<ldap_bind_password_statement> ::=

"ldap_bind_password" <string>

<postgresql_server_host_statement> ::=

"postgresql_server_host" <hostname>

<postgresql_server_port_statement> ::=

"postgresql_server_port" <portnumber>

<postgresql_dbname_statement> ::=

"postgresql_dbname" <string>

<postgresql_user_statement> ::=

"postgresql_user" <string>

<postgresql_password_statement> ::=

"postgresql_password" <string>

<postgresql_conninfo_statement> ::=

"postgresql_conninfo" <string>

<auth_statement> ::=

"auth" <validity> <auth_method> <hostspec>

<auth_command> ::=

"enable" | "disable"

<auth_method> ::=

"gsi" | "gsi_auth" | "sharedsecret"

<sockopt_statement> ::=

"sockopt" <socket_option>[=<number>] [""LISTENER" | <hostspec>]

<socket_option> ::=

"debug" | "keepalive" | "sndbuf" | "rcvbuf" |
"tcp_nodelay"

<known_network_statement> ::=

"known_network" <hostspec>

<network_receive_timeout_statement> ::=

"network_receive_timeout" <number>

<admin_user_statement> ::=

"admin_user" <string>

<admin_user_gsi_dn_statement> ::=

"admin_user_gsi_dn" <string>

<local_user_map_statement> ::=

"local_user_map" <local_user_and_group_map_spec>

<local_group_map_statement> ::=

"local_group_map" <local_user_and_group_map_spec>

<local_user_and_group_map_spec> ::=

<pathname> |
<pathname> <hostname>|
<pathname> <hostname> ":" <portnumber>

<schedule_cache_timeout_statement> ::=

"schedule_cache_timeout" <number>

<schedule_concurrency_statement> ::=

"schedule_concurrency" <number>

<schedule_concurrency_per_net_statement> ::=

"schedule_concurrency_per_net" <number>

<schedule_idle_load_thresh_statement> ::=

"schedule_idle_load_thresh" <load>

<schedule_busy_load_thresh_statement> ::=

"schedule_busy_load_thresh" <load>

<schedule_virtual_load_statement> ::=

"schedule_virtual_load" <load>

<schedule_candidates_ratio_statement> ::=

"schedule_candidates_ratio" <floating_point_number>

<schedule_rtt_thresh_diff_statement> ::=

"schedule_rtt_thresh_diff" <number>

<schedule_rtt_thresh_ratio_statement> ::=

"schedule_rtt_thresh_ratio" <floating_point_number>

<schedule_rtt_thresh_statement> ::=

"schedule_rtt_thresh" <floating_point_number>

<write_local_priority_statement> ::=

"write_local_priority" <validity>

<write_target_domain_statement> ::=

"write_target_domain" <string>

<minimum_free_disk_space_statement> ::=

"minimum_free_disk_space" <size>

<simultaneous_replication_receivers_statement> ::=

"simultaneous_replication_receivers" <number>

<gfsd_connection_cache_statement> ::=

"gfsd_connection_cache" <number>

<xmlattr_size_limit_statement> ::=

"xmlattr_size_limit" <size>

<xattr_size_limit_statement> ::=

"xattr_size_limit" <size>

<attr_cache_limit_statement> ::=

"attr_cache_limit" <number>

<attr_cache_timeout_statement> ::=

"attr_cache_timeout" <number>

<log_level_statement> ::=

"log_level" <log_priority>

<log_message_verbose_level_statement> ::=

"log_message_verbose_level" <number>

<log_auth_verbose_statement> ::=

"log_auth_verbose" <validity>

<file_trace_statement> ::=

"file_trace" <validity>

<fatal_action_statement> ::=

"fatal_action" <fatal_action_name>

<debug_command_statement> ::=

"debug_command" <string_list>

<no_file_system_node_timeout_statement> ::=

"no_file_system_node_timeout" <number>

<gfmd_reconnection_timeout_statement> ::=

"gfmd_reconnection_timeout" <number>

<atime_statement> ::=

"atime" <atime_type>

<client_file_bufsize_statement> ::=

"client_file_bufsize" <size>

<client_parallel_copy_statement> ::=

"client_parallel_copy" <number>

<profile_statement> ::=

"profile" <validity>

<metadb_server_list_statement> ::=

"metadb_server_list" <metadb_server_list_spec>

<metadb_server_list_spec> ::=

<hostname_port_spec> |
<hostname_port_spec> <metadb_server_list_spec>

<metadb_replication_statement> ::=

"metadb_replication" <validity>

<synchronous_replication_timeout_statement> ::=

"synchronous_replication_timeout" <number>

<synchronous_journaling_statement> ::=

"synchronous_journaling" <validity>

<metadb_server_force_slave_statement> ::=

"metadb_server_force_slave" <validity>

<metadb_server_slave_max_size_statement> ::=

"metadb_server_slave_max_size" <number>

<metadb_journal_dir_statement> ::=

"metadb_journal_dir" <pathname>

<metadb_journal_max_size_statement> ::=

"metadb_journal_max_size" <number>

<metadb_journal_recvq_size_statement> ::=

"metadb_journal_recvq_size" <number>

<replica_check_statement> ::=

"replica_check" <validity>

<replica_check_host_down_thresh_statement> ::=

"replica_check_host_down_thresh" <number>

<replica_check_sleep_time_statement> ::=

"replica_check_sleep_time" <number>

<replica_check_minimum_interval_statement> ::=

"replica_check_minimum_interval" <number>

<string_list> ::=

<string> |
<string> <string_list>

<hostspec> ::=

<ipv4_address> | <ipv4_address> "/" <address_mask> |
<hostname> | "." <domain_name> | "*"

<pathname> ::=

<pathname_character> <pathname_character>*

<pathname_character> ::=

<hostname_character> | "," | "/" | "_"

<hostname_port_spec> ::=

<hostname>|
<hostname> ":" <portnumber>

<hostname> ::=

<hostname_character> <hostname_character>*

<hostname_character> ::=

<alphabet> | <digit> | "-" | "."

<portnumber> ::=

<number>

<size> ::=

<number> [ "k" | "M" | "G" | "T" ]

<number> ::=

<digit> [<digit>*]

<digit> ::=

"0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9"

<string> ::=

""" <double_quoted_character>* """

<double_quoted_character> ::=

<any_character_except_backslash_and_double_quotation> |
"\\" | "\""

<validity> ::=

"enable" | "disable"

<spck_level> ::=

"disable" | "display" | "delete" | "lost_found"

<atime_type> ::=

"disable" | "relative" | "strict"

<log_priority> ::=

"emerg" | "alert" | "crit" | "err" | "warning" |
"notice" | "info" | "debug"

<fatal_action_name> ::=

"backtrace_and_abort" | "backtrace_and_exit" |
"abort" | "exit"

EXAMPLES

The following is an example of gfarm2.conf to use sharedsecret authentication.


metadb_server_host metadb.example.org
metadb_server_port 601
auth enable sharedsecret *
sockopt keepalive

The following is an example of gfmd.conf to use PostgreSQL to store the metadata, and to allow access from filesystem nodes and clients at IP address 192.168.0.0/24, via sharedsecret authentication.


metadb_server_host metadb.example.org
metadb_server_port 601
postgresql_server_host metadb.example.org
postgresql_server_port 5432
postgresql_dbname gfarm
postgresql_user gfarm
postgresql_password "secret-postgresql-password"
auth enable sharedsecret 192.168.0.0/24
admin_user john
sockopt keepalive

The following is an example of gfmd.conf to use LDAP to store the metadata, and to allow access from filesystem nodes and clients at any IP address, via GSI authentication.


metadb_server_host metadb.example.com
metadb_server_port 601
ldap_server_host metadb.example.com
ldap_server_port 602
ldap_base_dn "dc=example, dc=com"
ldap_bind_dn "cn=gfarmuser, dc=example, dc=com"
ldap_bind_password "secret-ldap-password"
auth enable gsi *
spool_server_cred_type host
spool_server_cred_service gfsd
admin_user john
admin_user_gsi_dn "/C=JP/ST=Tokyo/L=Chuo-ku/O=Example, Inc./OU=Example Division/CN=John Doe"
sockopt keepalive

FILES

/usr/local/gfarm/etc/gfarm2.conf

$HOME/.gfarm2rc

SEE ALSO

gfmd(8), gfsd(8), setsockopt(2)