We will provide an overview of the Globus platform, and demonstrate several data management features. Serving as an introductory session suitable for all current and prospective users, we’ll use the Globus web app to show data transfer and sharing, and connect your laptop/desktop to Globus. We will also introduce the Globus Command Line Interface. This introduction will provide important context for subsequent sessions.

We will present an overview of Globus services for automating research computing and data management tasks, to accelerate research process throughput. This session is aimed at researchers who wish to automate repetitive data management tasks (such as backup and data distribution to collaborators), as well as anyone working with instruments (cryoEM, next-gen sequencers, fMRI, etc.) who wishes to streamline data egress, downstream analysis, and sharing at scale. The material in this session will serve as an introduction to the more advanced concepts that will be covered in detail on day two of the workshop.

We will review the Globus Connect Server architecture and deployment model, and describe how system administrators can create a Globus endpoint to access on-premises and cloud-hosted storage systems. You will experiment with installing Globus Connect Server, and configuring a number of common options on the endpoint. We will also demonstrate how to monitor and manage user activity, and options for optimizing file transfer performance.

We will present various use cases for applying the Globus data sharing capability, and discuss use of service accounts for automating data access and transfer tasks.

We will provide a brief introduction to the Globus Platform-as-a-Service, with emphasis on building simple web applications for data distribution and discovery. We will describe how to register an application with Globus and access platform APIs using the Globus Python SDK and a Jupyter Notebook. We will also introduce the Globus Search service and demonstrate how it is used by an open source web portal framework that can jumpstart research application development. The objective here is to survey the breadth of Globus services and tools that are available, to better inform research software engineers making design and implementation decisions.

Rick Wagner, SDSC CTO, will describe examples of research data management and computation enabled by the supercomputing center, and identify the common patterns that emerge, as well as opportunities for automation.

With the use cases above as context, we will dive deeper into the Globus automation platform and describe how common instrument-based scenarios may be streamlined. We will examine the various components of a Globus flow that takes data from the point of capture on an instrument through to distribution of analyzed data to collaborators, and even broader publication of result data.

We will start with a simple series of tasks and progressively work through more complex activities that incorporate computation as well as data description, publication and discovery. In the course of automating these more advanced scenarios we will describe and demonstrate the Globus Compute, Globus Search and Globus Flows services.

This session—and the remainder of the day—will be spent defining various automation-enabling services in more detail and progressively augmenting the solution. We will engage in lots of hands-on exercises to familiarize you with many new capabilities and how they may be leveraged using SDSC storage and compute resources.

We will be available for 1:1 discussions and consultation on specific use cases.