|
 |
The various filesystems available on the Fermilab LQCD Computing Facility are listed the table below
|
Area
|
Description
|
/project/xxxxx
|
- Area typically used for approved projects.
- Visible on all cluster worker nodes via NFS file-system.
- Backups nightly. Suitable for output logs, meson correlators and other small data files. NOT suitable for fields e.g configs, quark propagators.
- Go to Allocations page to check you /project disk usage.
|
/home/<username>
|
- Home area. Backups nightly. Visible on all cluster worker nodes via NFS.
- Not suitable for configs or props.
- Can be used as "run" directory for light production or testing.
- Quota of about 6 to 10 GB per home directory.
- Run zfsquota on lq.fnal.gov to check your home area disk usage.
|
/pnfs/lqcd
|
- Enstore Tape storage.
- Visible on cluster login head nodes only.
- Ideal for permanent storage of parameter files and results.
- Must use special copy command: 'dccp'
|
/lustre1
|
- Lustre storage. NO backups.
- Visible on all cluster worker nodes.
- Ideal for temporary storage (~month) of very large data files. NOT suitable for large number of small files. Disk space usage monitored and disk quotas enforced.
- Go to Allocations page to check you /lustre1 disk usage.
|
/scratch
|
Local disk on each worker node. NO backups. 892GB in size. Suitable for writing all sorts of data created in a batch job. Any precious data needs to be copied off of this area before a job ends since the area is automatically purged at the end of a job.
|
|
 |
/home Filesystem
|
 |
Each user has a home directory on the LQ1 cluster. This home directory is mounted over NFS on all the worker nodes. For most part users do not have to worry about home directories in general. The user quota on these disks are several GBs and this space is backed up nightly.
|
Lustre Filesystem
|
Lustre is a Lustre is a scalable, secure, robust, highly-available cluster file system. The current Lustre Portable Operating System Interface (POSIX) is comparable with NFS and Lustre filesystem supports MPI-IO as well.
The Lustre Filesystem is mounted at /lustre1 and your project will have a sub directory under the main mountpoint depending on your disk allocation. This filesystem is not backed up and is meant to be used as a volatile storage space. Quotas are enforced on the file system and count against your disk allocation. Please send email to lqcd-admin@fnal.gov to apply for a top level directory for your project and for quota maintenance.
To transfer between Lustre using Globus Online please follow the steps listed on this page.
Please refer to the following page for more details about the various Lustre user commands. Our Lustre version for /lustre1 is 2.12.2.
|
/project Filesystem
|
This limited storage area is typically used for approved projects. This filesystem is accessible from all cluster worker nodes via NFS as /project using IP-over-Infiniband and is automatically backed up every night. This storage space is suitable for output logs, meson correlators, and other small data files and it should NOT be used for storing fields e.g configurations, quark propagators.
|
/pnfs/lqcd Filesystem
|
|
/pnfs/lqcd is the central disk cache with tape backup facility and is intended for permanent storage of parameter files and results. Although this appears to be a standard disk area, it is not. Commands to manipulate file and directory metadata (for example, rm, mv, chmod, mkdir, rmdir) will work here, but commands like cp or cat will not. Instead, you will need to use dccp to copy files in and out of this area. dccp has syntax like cp, i.e.
dccp source destination
|
/scratch Filesystem
|
Each worker node has a private locally mounted filesystem called /scratch which is available to a batch job. Each LQ1 worker has 892 GB of space onn /scratch. Scratch is the recommended place to write data of all sorts created in a batch job. Any precious data needs to be copied off of /scratch before a job ends since the area is automatically purged at the end of a job.
|
- Fermi National Accelerator Laboratory
- Managed by Fermi Research Alliance, LLC
- for the U.S. Department of Energy Office of Science
|
|
 |
- Security, Privacy, Legal
|
|
 |
|
|