Using the Globus file transfer service

Summary

How to use the Globus file transfer service to copy and share files between on- and off-campus systems.

Environment

The Globus file transfer service can copy and share files stored on the following systems:

  • Data Den
  • Locker
  • Turbo
  • Stand-alone platforms and laptops

Directions 

Getting Started with Globus

  1. Set up your Globus identity
    Step-by-step guide

Transferring Data with Globus

  1. Browse to Globus 
  2. Log in at LOG IN icon at upper right
  3. On the File Manager pane (select File Manager on left if necessary) search for the data source collection in the left Collection box.  ARC provides endpoints for the following collections:
    ARC Service Globus Name
    Great Lakes cluster umich#greatlakes
    Lighthouse cluster umich#lighthouse
    Armis2 cluster umich#armis2 v2
    Sensitive Data Turbo Volumes UMich ARC Sensitive Turbo Volume Collection
    Non-Sensitive Data Turbo Volumes UMich ARC Non-Sensitive Turbo Volume Collection
    Non-Sensitive Data Locker Volumes    UMich ARC Non-Sensitive Locker Volume Collection
    Non-Sensitive Data Den UMich ARC Non-Sensitive Data Den Volume Collection
    Sensitive Data Den UMich ARC Sensitive Data Den Volume Collection

    Note:  the latter UMich ARC  data-centric endpoints are preferred.  The former cluster-centric endpoints can also be used, but you have to search for the right one in the list that appears, and they will be phased out when ARC migrates to future clusters.  /scratch is only accessible from a cluster-centric endpoint, and some Turbo volumes are accessible in the /nfs/turbo directory there as well.
  4. Select the collection from the list that appears.
  5. Select the volume from the list that appears in the pane, and click the > chevron to its right.
    Note:  if you are a member of more than 16 Posix groups on the clusters you may be denied access.  There is no workaround except to reduce the number of Posix groups you are in or authenticating using another U-M identity that is under the Posix limit.
  6. Navigate to the file or directory you would like to transfer, and click to highlight it.  You can also type pathnames directly into the Path box, but be aware only data volumes are reachable through a data-centric endpoint.
  7. Repeat the collection and directory selection process for the right-hand pane, selecting the directory you would like to transfer to.  You can use the same endpoint for both sides of the copy.
  8. If desired, select Transfer & Timer options in the upper center and enable the desired options for the transfer.  More information on each option is available via the "i" icon.  You can also set a delayed start for the transfer, or specify an interval for repetitive transfers.
  9. Click the blue Start button to start or schedule the transfer.   You can view details of the transfer by clicking the popup that appears, or selecting Activity on the left pane and clicking the > chevron for the appropriate transfer.

You can click Activity in the left pane at any time to view your transfers; click the > chevron to the right of a given transfer to view an overview including transfer status and progress.  Click the Event Log tab to see a list of each segment of data transferred; more details are available for each segment by clicking the v chevron.

You can transfer small numbers of files or directories using the Upload button in the center column.

Sharing Data with Globus

ARC subscribes to Sharing or Guest collections, which allow users to create new Collections that are subsets of existing Collections and can be shared with other users. 

  1. Login to Globus and select a directory you would like to share
  2. Select Share in the middle column.
  3. On the Collections page that appears, click Add Guest Collection or select an existing Guest Collection from the list.  You can check ADMINISTERED BY ME to see only your own collections.
  4. Create your collection on the page that appears.  You can share the entire directory you specified in Step 1 or any subdirectory of it with the Browse button.  Setting keywords will help you to search for this collection later.  Checking Force Encryption will not allow encryption to be turned off in Transfer & Timer for any transfers to or from this collection.  Click Create Collection to create it.
  5. A page appears showing your collection and that you have read, write, and admin access to it.
  6. Allow access to this collection for other users by clicking Add Permissions - Share With on the upper right, and selecting:
    1. Again, you can give the new user access to the entire directory you specified in Step 4 or any subdirectory of it. 
    2. Choose whom to share with.  Ensure any data privacy requirements are met.  It is not recommended that you give write access to the all users or public categories.
    3. Search for the user by name or email, and select from the list.  They will receive an email at the indicated address; you can add message text here.
    4. Set write access if desired (read access will always be permitted).
  7. Click Add Permission.  The user will receive an email with a linkt to the collection and information describing how they can access it.
  8. You can repeat the process to allow another user to access the collection or click  Done to finish.

You can click Collections in the left pane at any time to view your collections  (check ADMINISTERED BY YOU to see only your collections; click the > chevron to the right to inspect or delete your collection.

Globus Connect Personal

Users can install an endpoint on their own machines.  Users can install it themselves without administrative oversight and will only be able to authenticate as the user who installed the endpoint.  Once installed, the endpoint will be visible on the Collections page for that user and can be used like any other Collection managed by Globus.

Globus Connect Server

Administrators can install an endpoint on a lab or other shared machine.  Send an email to arc-support@umich.edu for assistance in installation.  Once installed, the endpoint will be visible on the Collections page for all users and can be used by all users like any other Collection managed by Globus.

See also ARC documentation.

Managing Sensitive Data with Globus

Users are responsible for security and compliance related to sensitive code and/or data. Security and compliance are shared responsibilities. If you process or store sensitive university data, software, or libraries on your storage volume, you are responsible for understanding and adhering to any relevant legal, regulatory or contractual requirements.

See the ARC documentation for information on the Sensitive Data Turbo Collection and Sensitive Globus Connect Personal terms of usage.  Note: It is our responsibility, not ARC’s, to be aware of and comply with all applicable laws, regulations, and university policies (e.g., ITAR, EAR, HIPAA) as part of any research activity that may raise compliance issues under those laws.

External resources

Using Globus (complete document, portions referenced above)

Details

Article ID: 10585
Created
Mon 8/7/23 10:33 AM
Modified
Fri 3/8/24 10:41 AM