Difference between revisions of "NAS"
| Line 60: | Line 60: | ||
* <code>-d</code>: Create required directories that don't already exist. Equivalent of the <code>-p</code> flag for <code>mkdir</code> | * <code>-d</code>: Create required directories that don't already exist. Equivalent of the <code>-p</code> flag for <code>mkdir</code> | ||
* <code>--sync</code>: Only copy over "new" files, where "new" are any changes to the modification time or file size. | * <code>--sync</code>: Only copy over "new" files, where "new" are any changes to the modification time or file size. | ||
| + | ** If a file exists on destination (<code>.</code>), but not source (<code>STFM_Tet_dz4-10_dx15</code>), it will not be copied back to source nor will it be deleted to match the state of source. | ||
Once this command is submitted, the transfer process will be backgrounded. Progress can be viewed by running <code>shiftc --monitor</code>. Additionally, you will recieve an email with the transfer job is completed. | Once this command is submitted, the transfer process will be backgrounded. Progress can be viewed by running <code>shiftc --monitor</code>. Additionally, you will recieve an email with the transfer job is completed. | ||
More documentation for <code>shiftc</code> can be found in its man page (<code>man shiftc</code>) and on [https://www.nas.nasa.gov/hecc/support/kb/shift-transfer-tool-overview_300.html NAS's documentation website]. | More documentation for <code>shiftc</code> can be found in its man page (<code>man shiftc</code>) and on [https://www.nas.nasa.gov/hecc/support/kb/shift-transfer-tool-overview_300.html NAS's documentation website]. | ||
Revision as of 09:09, 28 July 2020
Wiki for information related to the NASA Advanced Supercomputing (NAS) facility.
Contents
Overview
| Key | Value | Notes |
|---|---|---|
| Machines | Pleiades | Compute |
| Lou | Storage and Analysis | |
| Electra | Compute | |
| Endeavour | Compute | |
| Merope | Compute | |
| Job Submission System | PBS | |
| Facility Documentation | Support Knowledgebase |
How-To's
How-To's in Separate Wiki's
Backup Data from Scratch Directories
This is done simply by copying data from the /nobackup/$USER directories to your home directory on Lou (lfe). The /nobackup/$USER directories are mounted onto lfe, so transfers should be done on lfe.
This can be done with scp, but is recommended to use NASA's in-house utility shiftc. shiftc will automatically perform parallel file transfers, data integrity checks and repairs, and syncing features similar to rsync.
Commands:
jrwrigh7@lfe7: shiftc -r -d --sync /nobackup/jrwrigh7/models/STGFlatPlate/STFM_Tet_dz4-10_dx15 .
This will copy the directory STFM_Tet_dz4-10_dx15 to the current location. The flags do as follows
-
-r: Recursively copy files from destination -
-d: Create required directories that don't already exist. Equivalent of the-pflag formkdir -
--sync: Only copy over "new" files, where "new" are any changes to the modification time or file size.- If a file exists on destination (
.), but not source (STFM_Tet_dz4-10_dx15), it will not be copied back to source nor will it be deleted to match the state of source.
- If a file exists on destination (
Once this command is submitted, the transfer process will be backgrounded. Progress can be viewed by running shiftc --monitor. Additionally, you will recieve an email with the transfer job is completed.
More documentation for shiftc can be found in its man page (man shiftc) and on NAS's documentation website.
