mmcloudgateway command

Creates and manages the cloud storage tier.

Synopsis

mmcloudgateway account pre-test --cloud-type {S3 |SWIFT | SWIFT-K EYSTONE | SWIFT3 | CLEVERSAFE | CLEVERSAFE-NEW}
                                              --username UserName  [ --pwd-file PasswordFile]
                                              [--cloud-url CloudURL] [--tenant-id TenantID] [--location Location]
                                              [--object-size ObjectSize] [--server-cert-path ServerCertPath]

mmcloudgateway account create --cloud-nodeclass CloudNodeClass --cloud-name CloudName
                               --cloud-type {S3 |SWIFT | SWIFT-KEYSTONE | SWIFT3 | CLEVERSAFE | CLEVERSAFE-NEW} 
                               --username UserName  [ --pwd-file PasswordFile]
                               --enable {TRUE | FALSE} [--cloud-url CloudURL] [--tenant-id TenantID]
                               [--location Location] [--meta-location MetaLocation] [--mpu-parts-size MPUPartsSize] 
                               [--server-cert-path ServerCertPath] [--enc-enable{ TRUE | FALSE }]
                               [--etag-enable {TRUE | FALSE }]

mmcloudgateway account test --cloud-nodeclass CloudNodeClass --cloud-name CloudName

mmcloudgateway account update --cloud-nodeclass CloudNodeClass --cloud-name CloudName 
                                { [--pwd-file PasswordFile ] |
                                [--mpu-parts-size MPUPartsSize] |
                                [--location Location{ --cloud-type 
                                {S3 |CLEVERSAFE |CLEVERSAFE-NEW}}]}

mmcloudgateway account delete --cloud-nodeclass CloudNodeClass --cloud-name CloudName

mmcloudgateway account list --cloud-nodeclass CloudNodeClass [ --cloud-name CloudName ]

mmcloudgateway config set --cloud-nodeclass CloudNodeClass 
                         [--port Port ] 
                         [--slice-size SliceSize]
                         [--migrate-threadpool-size MigrateThreadpoolSize]
                         [--recall-threadpool-size RecallThreadpoolSize]
                         [--tracing-enable {TRUE | FALSE }]
                         [--tracing-level {comp=level [,comp=level  ...]}]
                         [--audit-enable { TRUE | FALSE }] [--rotate-key]
                         [--rkm-enable { TRUE --rkm-servername RKMServerName --rkm-port RKMPort 
                          --rkm-username RKMUserName 
                         [--pwd-file PasswordFile] | FALSE}]

mmcloudgateway config unset --cloud-nodeclass CloudNodeClass  
                             {[--slice-size] | [--migrate-threadpool-size ]
                             [--recall-threadpool-size ] | [ --tracing-enable ] |
                             [ --tracing-level ] | [ --audit-enable ]}

mmcloudgateway config list [--cloud-nodeclass CloudNodeClass]

mmcloudgateway filesystem create --cloud-nodeclass CloudNodeClass
                                 --file-system FileSystem --container-prefix
                                  ContainerPrefix [--override-container-name]

mmcloudgateway filesystem delete --cloud-nodeclass CloudNodeClass --file-system FileSystem

mmcloudgateway filesystem list --cloud-nodeclass CloudNodeClass

mmcloudgateway node list

mmcloudgateway service start [-N { Node [,Node ...] | NodeFile | NodeClass }]

mmcloudgateway service stop [-N { Node [,Node ...] | NodeFile | NodeClass }]

mmcloudgateway service status [-N { Node [,Node ...] | NodeFile | NodeClass }]

mmcloudgateway files migrate [-v] [--] File [File ...]

mmcloudgateway files recall [-v] [--co-resident-state]  [--] File [File ...]

mmcloudgateway files restore [-v] [--overwrite]
                                  { -F FileListFile | [--dry-run] [--restore-location RestoreLocation]
                                  [ --id ID] [--] File}

mmcloudgateway files delete {-delete-local-file | -recall-cloud-file | --require-local-file}
                            [--keep-last-cloud-file] [--] File [File ...]

mmcloudgateway files reconcile {Device | all}

mmcloudgateway files cloudList {--path Path [ --recursive  [--depth Depth]] [--file File] | 
                               [--file-versions File] | --files-usage --path Path [ --depth Depth]
                               | --reconcile-status --path Path }

mmcloudgateway files rebuildDB Device

mmcloudgateway files list File [File ...]

Availability

Available with IBM Spectrum Scale™ Advanced Edition.

Description

Use the mmcloudgateway command to manage and administer the Transparent Cloud Tiering feature.

Parameters

account

Manages cloud storage accounts with one of the following actions:

pre-test

Verifies that the cloud storage account settings are correct before the account is actually created. If no errors, you can go ahead and create a cloud storage.

--cloud-type CloudType

Specifies the name of the object storage provider. The following object storage providers are supported:

Amazon Simple Storage Service (S3)
OpenStack Swift (SWIFT)
OpenStack Swift with Keystone authentication (SWIFT-KEYSTONE)
S3 API layer on Swift (SWIFT3)
IBM Cloud Object Storage version earlier than 3.7.2. To use this cloud type, select the "Cleversafe" cloud type.
IBM Cloud Object Storage version 3.7.2 and above. To use this cloud type, select the "Cleversafe-New" cloud type.
Note: Incorrect cloud type affects migration of large files that are transferred using multipart upload (MPU) technology.

--username UserName: Specifies the user name of the cloud object storage account.
Note: For Amazon S3 and IBM Cloud Object Storage, it represents the access key.

--pwd-file PasswordFile: Specifies a file that includes the password. For Amazon S3 and IBM Cloud Object Storage, password represents the secret access key.

--cloud-url CloudURL: Specifies the URL of the cloud storage provider.
Note: Optional for S3, but mandatory for Swift, Swift-Keystone, and IBM Cloud Object Storage. For object storage providers that require an end-point URL, the end-point URL need not have a container/vault name.

--tenant-id Tenant ID: Specifies the tenant ID for the cloud storage provider account.
Note: Optional for cloud type “S3” but mandatory for cloud types “Swift” and “Swift-Keystone”

--location Location

Specifies the preferred location of the object cloud storage provider. The valid values for S3 are the following:

us-west-1
us-west-2
EU or eu-west-1
eu-central-1
apsoutheast-1
ap-southeast-2
ap-northeast-1
ap-northeast-2
sa-east-1

For IBM Cloud Object Storage, use the provisioning code for vault provisioning template. Do not use the name of the provisioning template. If a location is not used to provide provisioning code, then the default provisioning template is used.

--object size ObjectSize: Size of the objects to be transferred to the cloud for the pre-test.

create

Creates a cloud account definition to access the remote cloud.

--cloud-nodeclass CloudNodeClass: Specifies the node class that was created by using the mmcrnodeclass command.

--cloud-name CloudName

Specifies a name that uniquely identifies the cloud object storage account on the node.

Note: No special characters are allowed in the name except for “-” and “_”.

--cloud-type CloudType

Specifies the name of the object storage provider. The following object storage providers are supported:

Amazon Simple Storage Service (S3)
OpenStack Swift (SWIFT)
OpenStack Swift with Keystone authentication (Swift-Keystone)
S3 API layer on Swift (Swift3)
IBM Cloud Object Storage version earlier than 3.7.2
IBM Cloud Object Storage version 3.7.2 and above

--username Name: Specifies the user name of the cloud object storage account.
Note: For Amazon S3 and IBM Cloud Object Storage, it represents the access key.

--pwd-file PasswordFile: Specifies a file that includes the password.

--enable {TRUE | FALSE}: Enables or disables the added cloud object storage account for Transparent Cloud Tiering node.
Note: This property is always enabled in the 4.2.1 release.

--cloud-url CloudURL: Specifies the URL of the cloud storage provider.
Note: Optional for S3, but mandatory for Swift, Swift-Keystone, and IBM Cloud Object Storage. For object storage providers that require an end-point URL, the end-point URL need not have a container/vault name.

--tenant-id Tenant ID: Specifies the tenant ID for the cloud storage provider account.
Note: Optional for cloud type “S3” but mandatory for cloud types “Swift” and “Swift-Keystone.”

--location Location

Specifies the preferred location of the object cloud storage provider. The valid values for S3 are the following:

us-west-1
us-west-2
EU or eu-west-1
eu-central-1
apsoutheast-1
ap-southeast-2
ap-northeast-1
ap-northeast-2
sa-east-1

--meta-location MetaLocation: For IBM Cloud Object Storage, two containers are created on the cloud storage tier, one for data and one for metadata. You can use this attribute to specify different vault provisioning codes for metadata containers. Data container should have indexing disabled, whereas metadata container should have indexing enabled.

--mpu-parts-size MPUPartsSize: Specifies multi-part upload size in MB. Value that is allowed is 5 - 32, default being 32.

--server-cert-path ServerCertPath: Specifies the certificate path for the self-signed certificates that are presented by the private object storage servers. This is required only when the cloud URL uses https.

--enc-enable [TRUE | FALSE]: Specifies whether you want to enable encryption on the data that is transferred to the object storage.

--etag-enable [TRUE | FALSE]: Specifies whether you want to enable an integrity check on the data that is migrated or recalled to or from the cloud storage.

test

Verifies the validity of the cloud account.

--cloud-nodeclass CloudNodeClass: Specifies the node class that was created by using the mmcrnodeclass command.

--cloud-name CloudName: Specifies the unique name that was provided to the cloud object storage account on the Transparent Cloud Tiering node.

update

Updates the cloud account attributes (except the cloud type) that are provided while you create a cloud account. To update the cloud type, you must delete the cloud account and create a new one.

--cloud-nodeclass CloudNodeClass: Specifies the node class that was created by using the mmcrnodeclass command.

--cloud-name CloudName: Specifies the unique name that was provided to the cloud object storage account on the node.

--pwd-file PasswordFile: Specifies the file that includes the password.

--mpu-parts-size MPUPartsSize: Specifies multi-part upload size in MB. Value that is allowed is 5 - 32, default being 32.

--location Location: Specifies the preferred location that was provided while you created the object cloud storage tier.

--cloud-type CloudType: Specifies the name of the object storage provider.

delete

Deletes the cloud storage tier that is created by using the mmcloudgateway account create command.

--cloud-nodeclass CloudNodeClass: Specifies the node class that was created by using the mmcrnodeclass command.

--cloud-name CloudName: Specifies the cloud account that you want to delete.

list

Lists the registered cloud accounts. Displays more information about the configured cloud account such as the cloud provider name, cloud provider tenant ID, cloud provider URL.

--cloud-nodeclass CloudNodeClass: Specifies the node class that was created by using the mmcrnodeclass command.

--cloud-name CloudName: Specifies the cloud account that you want to delete.

config

Configures and tunes the Gateway node parameters with one of the following actions:

set

Sets the following system parameters, overriding the default values:

--cloud-nodeclass CloudNodeClass: Specifies the node class that was created by using the mmcrnodeclass command.

--port Port: Specifies the port the Transparent Cloud Tiering service listens on incoming configuration and data migration/recall commands.

--slize-size SliceSize: Specifies the internal unit of transferring data within Transparent Cloud Tiering modules. Higher slice size indicates better performance. Default value is 256 KB.

--migrate-threadpool-size MigrateThreadpoolSize: Enables administrators to increase the parallelism for data migration within the Transparent Cloud Tiering service. Higher threadpool-size indicates better performance. Its value can be within 1 to 32, with default being 16.

--recall-threadpool-size RecallThreadpoolSize: Enables administrators to increase the parallelism for data migration within the Transparent Cloud Tiering service. Higher threadpool-size indicates better performance. Its value can be within 1 to 32, with default being 16.

--tracing-enable {TRUE | FALSE}: Tracing level is to set non-default tracing levels for various Transparent Cloud Tiering internal components to generate more debug data if any problems occur.

--audit-enable {TRUE | FALSE}: Enables administrator to change the default audit behavior to be able to record important events within the Transparent Cloud Tiering service.

--rotate-key: Enables administrator to generate a new key as and when needed according to the security requirements.

--rkm-enable: Enables the IBM® Security Key Lifecycle Manager.

--rkm-servername RKMServerName: Specifies the host name or IP address of the IBM Security Lifecycle Manager server.

--rkm-port RKMPort: Specifies the port number on which the IBM Security Key Lifecycle Manager server listens for requests. Default value is 9080.

--rkm-username RKMUserName: Specifies the user name of the IBM Security Key Lifecycle Manager server REST Global Admin. Default value is SKLMAdmin.

--pwd-file PasswordFile: Specifies the password file of the IBM Security Key Lifecycle Manager server REST Global Admin.

unset

Unsets the configured attributes and retains the default values.

--cloud-nodeclass CloudNodeClass: Specifies the node class that was created by using the mmcrnodeclass command.

--slize-size SliceSize: Specifies the internal unit of transferring data within modules. Higher slice size indicates better performance. Default value is 256 KB.

--recall-threadpool-size: Enables administrators to increase the parallelism for data recall within the service. Higher threadpool-size indicates better performance. Its value can be within 1 to 32, with default being 16.

--tracing-enable: Tracing level is to set non-default tracing levels for various internal components to generate more debug data if any problems occur.

--tracing-level: Enables administrators to print trace messages of the internal components in a file.

--audit-enable: Enables administrator to change the default audit behavior to be able to record important events within the service.

list

Lists the current configurations such as IP address, port number, thread-pool size, tracing level, slice size.

--cloud-nodeclass CloudNodeClass: Specifies the node class that was created by using the mmcrnodeclass command.

filesystem

Enables the mapping of a file system to a node class.

create

Specifies the node class and a container name.

--cloud-nodeclass CloudNodeClass: Specifies the node class that was created by using the mmcrnodeclass command.

--file-system FileSystem: Specifies the file system.

--container-prefix ContainerPrefix: Specifies the cloud container.

--override-container-name: Overrides the container name and make it unique by attaching the file system ID to the container prefix.

delete

Deletes the cloud file system association to the node class.

--cloud-nodeclass CloudNodeClass: Specifies the node class that was created by using the mmcrnodeclass command.

--file-system FileSystem: Specifies the file system that is attached to the cloud storage tier.

list: Lists the table of file systems associated with the specified node class.

node

Enables administrators to manage registration of Transparent Cloud Tiering nodes within a cluster and also display the node class the nodes are part of.

list

Lists the Transparent Cloud Tiering nodes that are registered to the cluster.

--cloud-nodeclass CloudNodeClass: Specifies the node class that was created by using the mmcrnodeclass command.

service

Manages the Transparent Cloud Tiering service with these options:

start

Starts the Transparent Cloud Tiering service for a node or set of nodes and make the service available for file movement.

-N: Specifies the nodes.

Node[,Node...]: Specifies the list of nodes where the service needs to be started.

NodeFile: Specifies a file, containing the list of nodes where the service needs to be started.

NodeClass: Specifies the node class.

stop

Stops the Transparent Cloud Tiering service for a node or set of nodes.

-N: Specifies the nodes.

Node[,Node...]: Specifies the list of nodes where the service needs to be stopped.

NodeFile: Specifies a file, containing the list of nodes where the service needs to be started.

NodeClass: Specifies the node class.

status

Displays detailed status of the Transparent Cloud Tiering service including running state of the daemon service, cloud account name, and its connectivity status.

-N: Specifies the nodes.

Node[,Node...]: Specifies the list of nodes where the status of the service needs to be checked.

NodeFile: Specifies a file, containing the list of nodes where the status of the service needs to be started.

NodeClass: Specifies the node class.

files

Manages the resident and co-resident files on the cloud tier, with the following options:

migrate

Migrates the specified files or file sets to the cloud storage tier.

-v: Specifies the verbose message.

--File [File …]: Specifies multiple files that need to be migrated to the cloud storage tier. This parameter must be a complete file name. It cannot be a fragment of a file name and it cannot be a path.

recall

Recalls the specified files or file sets from the cloud storage tier.

-v: Specifies the verbose message.

--co-resident-state: Indicates that the files are co-resident on the cloud storage tier.

--File [File …]: Specifies multiple files that need to be recalled from the storage tier. This parameter must be a complete file name. It cannot be a fragment of a file name and it cannot be a path.

restore

Restores a file or list of files from the cloud storage tier when the local files are lost.

-v: Specifies the verbose message.

--overwrite: Overwrite the files if needed. If this option is not set, files will not be overwritten, and the files that are retrieved from the cloud will remain in temporary locations.

-F: Loads file arguments from the given filename.

--dry-run: Queries the local database and prints what would have been sent to the server. Does not contact the server. This is intended for debugging.

--restore-location RestoreLocation: Specifies the target location of the files to be restored.

--id Id: Specifies the version ID of a file if the file has multiple versions.

File: Specifies the files to be restored.

delete

Deletes the specified files or file sets.

--delete-local-file: Deletes the local files and the corresponding cloud objects.

--recall-cloud-file: Recalls the files from the cloud before they are deleted on the cloud. The status of local files becomes resident after the operation.

--require-local-file: Removes the extended attributes from a co-resident file and makes it resident, without deleting the corresponding cloud objects. The option requires the file data to be present on the file system and will not work on a non-resident file.

--keep-last-cloud-file: This option deletes all the versions of the file except the last one from the cloud. For example, if a file has three versions on the cloud, then versions 1 and 2 are deleted and version 3 is retained.

--File [File …]: Specifies multiple files. This parameter must be a complete file name. It cannot be a fragment of a file name and it cannot be a path.

reconcile

Reconciles files between your file system and the cloud storage tier.

Device: Specifies the device name associated with the file system.

all: Specifies all the device names.

cloudList

Lists the files on the cloud.

--path Path: Lists files and directories under the specified path.

--recursive: List all files in all directories under the current directory.
--depth Depth: List directories up to the specified depth under the specified path. Default is to list up to the full depth. Specify 0 to list only the current directory.

--file [File]: Specifies the names of the files that need to be listed. This parameter must be a complete file name. It cannot be a fragment of a file name and it cannot be a path.

--file-versions File: Displays information about all versions of the files specified by the full path.
--files-usage --path Path: Displays cloud data and metadata space usage under the specified path.

--reconcile-status --path Path: Displays the progress of the reconcile operation.

rebuildDB

Rebuilds the database manually.

Device: Specifies the device name associated with the file system whose database is corrupted and which is in need of manual recovery.

list

Lists the files and the associated states.

--File [File …]: Specifies the names of the files that need to be listed. This parameter must be a complete file name. It cannot be a fragment of a file name and it cannot be a path.

Security

You must have root authority to run the mmcloudgateway command.

The node on which the command is issued must be able to execute remote shell commands on any other node in the cluster without the use of a password and without producing any extraneous messages. For more information, see Requirements for administering a GPFS file system.