Logistics#

Representing locations#

Metasmith provides abstractions to move XGDBs between different compute platforms using a variety of methods. The Logistics class facilitates transfers between locations specified using the Source class. Files and folders not managed by Metasmith can also be moved using Logistics.

1from metasmith.python_api import Logistics, Source
2mover = Logistics()
3mover.QueueTransfer(
4    src = Source.FromLocal("./example.xgdb"),
5    dest = Source.FromLocal("path/to/destination/example.xgdb"),
6)
7result = mover.ExecuteTransfers()

A DataInstanceLibrary can be moved directly. The above is equivalent to:

1xgdb.SaveAs(Source.FromLocal("path/to/destination/example.xgdb"))

A Source need not be local.

Locations by method of access#

SSH#

Transfers can be made to and from remote machines via SSH.

 1from metasmith.python_api import SshSource
 2
 3remote = SshSource(
 4    host = "remote_name",
 5    path = "/path/on/remote/machine/example.xgdb",
 6).AsSource()
 7local = Source.FromLocal("path/to/local/example.xgdb")
 8
 9mover = Logistics()
10mover.QueueTransfer(remote, local)
11mover.QueueTransfer(local, remote)
12mover.ExecuteTransfers()

HTTP(S)/FTP#

Files accessible via a url can be downloaded. Upload is currently not supported.

1from metasmith.python_api import HttpSource
2remote = HttpSource.Parse("https://www.website.com/path/to/example.xgdb").AsSource()

Globus#

Metasmith can interface with Globus using the globus-cli to queue transfer tasks between endpoints.

1from metasmith.python_api import GlobusSource
2remote = GlobusSource(
3    endpoint = "01234567-89ab-cdef-0123-456789abcdef", # endpoint ID
4    path = "/path/on/endpoint/example.xgdb",
5).AsSource()

If using the globus file manager, links to folders and files can be parsed directly.

1remote_folder = GlobusSource.Parse(
2    "https://app.globus.org/file-manager?origin_id=01234567-89ab-cdef-0123-456789abcdef&origin_path=%2Fpath%2Fon%2Fendpoint",
3).AsSource()
4
5
6remote_file = GlobusSource.Parse(
7    "https://g-012345.6789a.bcde.data.globus.org/path/on/endpoint/example.file",
8).AsSource()