GRASS logo

Note: This addon document is for an older version of GRASS GIS that will be discontinued soon. You should upgrade your GRASS GIS installation, and read the current addon manual page.

NAME

g.cloud - Connects a local GRASS GIS session to another one in a cluster system.

KEYWORDS

general, cloud computing

SYNOPSIS

g.cloud
g.cloud --help
g.cloud [-cka] config=string server=name [grass_script=name] [qsub_script=name] [variables=name] [raster=name[,name,...]] [vector=name[,name,...]] [mail=name] [reconnect=name] [path=name] [--help] [--verbose] [--quiet] [--ui]

Flags:

-c
Cycle through all variables
-k
Keep temporal files and mapsets
-a
Use ssh-add for faster access (requires to launch ssh-agent beforehand)
--help
Print usage summary
--verbose
Verbose module output
--quiet
Quiet module output
--ui
Force launching GUI dialog

Parameters:

config=string [required]
Path to ASCII file containing authentication parameters
"-" to pass the parameters interactively
server=name [required]
Name or IP of server to be connected
grass_script=name
Path to the input GRASS script
qsub_script=name
Path to the input qsub script
variables=name
Python dictionary with the variables to pass to the GRASS script via qsub
Example: "{'year':[2010,2011]}" will call 'qsub -v year=2010 ...' and 'qsub -v year=2011 ...'
raster=name[,name,...]
Name of input raster map(s) used by GRASS script
vector=name[,name,...]
Name of input vector map(s) used by GRASS script
mail=name
Mail address to send emails at the end of jobs
reconnect=name
Reconnect with old job to see if it is finished
path=name
Path for temporal for g.cloud operations
The directory must be visible on the network to the blades
Default: $HOME

Table of contents

DESCRIPTION

g.cloud is a module to connect a personal GRASS GIS session to a remote GRASS GIS installation on a much faster cluster facility (high performance computing system). It currently only supports the Grid Engine system. The general aim is to offer the possibility to a user to perform high speed calculations, which are hard to compute on a personal computer by connecting to an external, powerful server or cloud computing system.

The g.cloud environment consists of a Python module. It checks if all program requirements are fullfilled on both the client and the server side. If everything is ok, it connects the current session to the cloud server; it transfers the user and other required scripts as well as the GRASS data to the server and eventually executes the remote GRASS GIS job. Subsequently, it transfers back the resulting maps and data and cleans up the server session.

For a job execution, the script requirements are two: on the client-side the user prepared GRASS GIS script which performs the calculations and on the server-side the qsub control script for Grid Engine to launch the GRASS GIS script job.

NOTES

It is highly recommended to use "ssh-add" in order to avoid the authentication to the cluster via password file.

EXAMPLE

North Carolina example, calculation of texture maps:

g.cloud config=$HOME/.gc_loginfile.txt server=giscluster \
  grass_script=$HOME/g.cloud/test/test_onevariable_raster.sh \
  qsub_script=$HOME/g.cloud/test/launch_SGE_grassjob.sh \
  variables="{'TEXTURE' : ['asm','corr','entr','se','var']}" \
  raster=lsat7_2002_40

North Carolina example, calculation of daily sun radiation:

g.cloud config=$HOME/.gc_loginfile.txt server=giscluster \
  grass_script=$HOME/g.cloud/test/test_onevariable_sun.sh \
  qsub_script=$HOME/test/launch_SGE_grassjob.sh \
  variables="{'DOY' : [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325]}" \
  raster=elevation

REQUIREMENTS

The software requirements are GRASS GIS 7 and ssh/scp on the client side; while on the server side the Grid Engine software or compatible software needs to be present to launch the GRASS jobs. Furthermore, python-pexpect is required to be installed on the client system.

REFERENCES

FOSS4G 2011 presentation

SEE ALSO

Parallel GRASS jobs from GRASS User Wiki

AUTHORS

Luca Delucchi, Markus Neteler, Fondazione Edmund Mach, Research and Innovation Centre, Department of Biodiversity and Molecular Ecology, GIS and Remote Sensing Unit

SOURCE CODE

Available at: g.cloud source code (history)

Latest change: Mon Jun 28 07:54:09 2021 in commit: 1cfc0af029a35a5d6c7dae5ca7204d0eb85dbc55


Note: This addon document is for an older version of GRASS GIS that will be discontinued soon. You should upgrade your GRASS GIS installation, and read the current addon manual page.

Main index | General index | Topics index | Keywords index | Graphical index | Full index

© 2003-2023 GRASS Development Team, GRASS GIS 7.8.8dev Reference Manual