User Guide#

Welcome to Elm!#

Elm is a high-capacity, cold-storage data service designed to meet the massive storage needs of the Stanford research community. With the ability to store over 200 Petabytes, Elm is designed for large volumes of data that need to be stored cost-effectively over long periods, but accessed infrequently.

If you have significant amounts of data that require long-term, inexpensive storage, Elm is an ideal solution.

Accessing Elm storage#

MinIO Console#

New Elm users need to connect at least once to the MinIO Console (web interface to MinIO) to generate an Access Key for data transfers using the S3 protocol. The MinIO Console also features a bucket browser and supports basic file transfers over HTTPS, making it convenient for managing smaller files.

For the campus instance, see the Elm Workgroup & URL Helper

S3 API Endpoint#

Example Use Case: Using `aws-cli` to transfer data between Elm and your computer.#

Authorized Elm users can use S3-compatible tools to access their data. Examples of such tools, already installed on Sherlock, include aws-cli, rclone, and s5cmd.

Disk space#

Elm quotas are measured in allocations of 1 TiB. Your quota is a hard limit that you set, not something that is determined by how much data you upload to Elm, so you'll need to allocate storage space before you upload to Elm.

NOTE: Elm is NOT backed up. While Elm has redundancy features like erasure coding, it’s still located on campus. This means Elm should be considered a single copy of your data. If there’s a big disaster (think earthquakes), there’s a chance you might lose access. For extra-important data, it’s a good idea to keep a second copy on another system.

Tracking usage#

The MinIO console provides basic usage tracking functionality. To access it, head to the approiate console for your bucket, depenind on your data risk classification:

Low or Moderate-risk: https://campus.elm.stanford.edu:9001/
High-risk (PHI & Non-PHI): https://campus.elm-hr.stanford.edu:9001/

Once you're logged into the console, click on Adminstrator -> Buckets to see your bucket's usage (volume) and number of objects stored.

Email notification for system maintenance and issues#

Please note that your email will be subscribed to the mailing list elm-announce (very low traffic for official announcements such as planned maintenance). Your group members will also be automatically subscribed. For more information, please see the following page:

Elm-announce List Information

Getting data in and out of Elm#

Elm is a stand-alone storage system that provides severl methods of access.

S3 Protocol
- Popular for use with tools like awscli, rclone, restic, and others.
MinIO Console
- A web-based user interface that allows for basic data management tasks.
Globus Endpoint
- A powerful data transfer tool that can be used for both simple and large scale transfers.

Service maintenance#

The first Tuesday of each month, from 8 a.m. to 2 p.m., is designated for scheduled maintenance. Clients will be notified by email at elm-announce@lists.stanford.edu two weeks prior to any scheduled work that may impact system availability, including the monthly maintenance.

Support and feedback#

Your feedback is very important. Please tell us about any issues you run into and successes you have. The best way to contact us is by email to SRCC Support.

Support limitations#

Stanford Research Computing will resolve Elm-specific hardware and file-system problems. Stanford Research Computing is not able to address firewall, network, remote connectivity, or desktop issues. For those issues, you can get help at HelpSU.

SRCC Slack workspace#

Stanford Research Computing has a workspace on the Stanford Slack Grid! This allows us to communicate system-wide announcements and provides our users a platform to connect and discuss directly, ask questions, recommend best practices, or share tips with the community.

In particular, we encourage you to join these two channels:

#elm-announce, for announcements related to Elm storage (read-only)
#elm-users, for discussions among the community of Elm storage users

Please note that while SRCC staff will monitor these channels, they are not intended as official support channels, and the recommended way to get support is still to email us at SRCC Support.

For more details about the SRCC Slack Workspace and instructions on how to join this workspace and its channels, please see:

SRCC Users Slack Workspace.