Difference between revisions of "Janus Cheat Sheet"

From PHASTA Wiki
Jump to: navigation, search
(→‎Allocation Status)
 
(6 intermediate revisions by 4 users not shown)
Line 12: Line 12:
 
   use Subversion
 
   use Subversion
  
 +
Viewing the list of dotkit modules (software) that you have loaded
 +
  echo $_dk_inuse
 
available nodes/cores:
 
available nodes/cores:
  
Line 45: Line 47:
  
 
Interactive, for debugging (with totalviewer)
 
Interactive, for debugging (with totalviewer)
 +
 
 +
  ssh -Y "user_id"@janus.rc.colorado.edu
  
−
   qsub -X -I -l nodes=1:ppn=12,walltime=1:00:00 -q janus-debug
+
   qsub -I -X -l nodes=1:ppn=12,walltime=1:00:00 -q janus-debug
  
 
Non Default allocation:
 
Non Default allocation:
Line 56: Line 60:
  
 
== Debugging PHASTA ==  
 
== Debugging PHASTA ==  
−
  If you are getting hanging codes or crashing codes, compile with the pgi compilers and try the following flags on  C, C++ and Fortran:
+
If you are getting hanging codes or crashing codes, compile with the pgi compilers and try the following flags on  C, C++ and Fortran:
 
    
 
    
 
       -Ktrap=fp -Mbounds -Kieee
 
       -Ktrap=fp -Mbounds -Kieee
 
    
 
    
−
   Basically this will make your code crash where you get floating point exceptions, go out of bounds with arrays, or have ieee exceptions. Then if you type
+
For PHASTA with cmake, you need to:
 +
   
 +
    CC=pgcc CXX=pgCC FC=pgfortran ccmake ../phasta
 +
    
 +
Then, add the compilation flags above to the following variables:
 +
 
 +
    CMAKE_CXX_FLAGS and CMAKE_C_FLAGS and CMAKE_Fortran_FLAGS variables
 +
 
 +
Basically this will make your code crash where you get floating point exceptions, go out of bounds with arrays, or have ieee exceptions. Then if you type
 
    
 
    
 
       set catchDebugger=1
 
       set catchDebugger=1
 
    
 
    
−
  and then  
+
and then  
 
    
 
    
 
       mpirun  <exec>
 
       mpirun  <exec>
 
    
 
    
−
  it will launch debug windows which when you release them will stop at the point of your "foul" which REALLY helps.
+
it will launch debug windows which when you release them will stop at the point of your "foul" which REALLY helps.
 
   More info: http://www.pgroup.com/userforum/viewtopic.php?t=1898&postdays=0&postorder=asc&start=0&sid=719a9d9871bf16add8565a398ba548f3
 
   More info: http://www.pgroup.com/userforum/viewtopic.php?t=1898&postdays=0&postorder=asc&start=0&sid=719a9d9871bf16add8565a398ba548f3
  
 
== Allocation Status ==  
 
== Allocation Status ==  
−
   use Crc-allocations
+
   module load utils/Crc-allocations
−
   check_allocation.py
+
   check_allocation
−
   use Moab
+
   module load Moab
 
   showstats -u $USER
 
   showstats -u $USER
  

Latest revision as of 17:11, 24 June 2013

Login

 ssh your_identikey_user@login.rc.colorado.edu
 [pin][token]

Running Jobs

Environment Setup: 

 use Torque
 use Moab
 use Git
 use .openmpi-1.4.3_intel-12.0_ib
 use Graphviz
 use Subversion

Viewing the list of dotkit modules (software) that you have loaded 

 echo $_dk_inuse

available nodes/cores: 

 pbsnodes -a
 showbf

Jobscript Template: 

 #!/bin/bash
 #PBS -N name_of_job
 #PBS -l walltime=0:20:00
 #PBS -l nodes=1:ppn=12
 . /curc/tools/utils/dkinit
 reuse .openmpi-1.4.3_intel-12.0_ib
 
  mpirun /path/to/executable

Submission: 

 qsub -q janus-debug jobscript.sh

Status 

 checkjob job_id
 showq -u $USER

Cancel: 

 qdel job_id

Large Memory, Interactive: 

 qsub -I -l nodes=1:ppn=32,walltime=1:00:00 -q himem

Interactive, for debugging (with totalviewer) 

 ssh -Y "user_id"@janus.rc.colorado.edu

 qsub -I -X -l nodes=1:ppn=12,walltime=1:00:00 -q janus-debug

Non Default allocation: 

 qsub -A UCBXXXXXXX job.sh

or in the job script: 

 #PBS -A UCBXXXXXXX

Debugging PHASTA

If you are getting hanging codes or crashing codes, compile with the pgi compilers and try the following flags on C, C++ and Fortran: 

     -Ktrap=fp -Mbounds -Kieee

For PHASTA with cmake, you need to: 

    CC=pgcc CXX=pgCC FC=pgfortran ccmake ../phasta

Then, add the compilation flags above to the following variables: 

   CMAKE_CXX_FLAGS and CMAKE_C_FLAGS and CMAKE_Fortran_FLAGS variables

Basically this will make your code crash where you get floating point exceptions, go out of bounds with arrays, or have ieee exceptions. Then if you type 

     set catchDebugger=1

and then 

     mpirun  <exec>

it will launch debug windows which when you release them will stop at the point of your "foul" which REALLY helps. 

 More info: http://www.pgroup.com/userforum/viewtopic.php?t=1898&postdays=0&postorder=asc&start=0&sid=719a9d9871bf16add8565a398ba548f3

Allocation Status

 module load utils/Crc-allocations
 check_allocation
 module load Moab
 showstats -u $USER

GridFTP

 export MYPROXY_SERVER_DN=$MYPROXY_SERVER_DN"/C=US/O=Globus Consortium/OU=Globus Connect Service/CN=842a610a-4de7-11e1-9674-123138151443"
 myproxy-logon -T -b -s gridftp-00.rc.colorado.edu -v -l bema1643

 globus-url-copy -v -r -cd -rst -rst-retries 0 -fast -vb -p 64 -stripe -tcp-bs 4M -g2 -ss '/C=US/O=Globus Consortium/OU=Globus Connect Service/CN=842a610a-4de7-11e1-9674-123138151443' gsiftp://gridftp-00.rc.colorado.edu/lustre/janus_scratch/bema1643/test.img sshftp://matthb2@jumpgate-phasta.colorado.edu/scratch/matthb2/foo/

globus-url-copy -v -r -cd -rst -rst-retries 0 -fast -vb -p 64 -stripe -tcp-bs 4M -g2 -ds '/C=US/O=Globus Consortium/OU=Globus Connect Service/CN=842a610a-4de7-11e1-9674-123138151443' sshftp://matthb2@jumpgate-phasta.colorado.edu/scratch/mrasquin/416M/ gsiftp://gridftp-00.rc.colorado.edu/lustre/janus_scratch/bema1643/

(note: -ss to specify that the source server has a non-standard DN, -ds for the destination)

Queues/Policy

https://www.rc.colorado.edu/crcdocs/queues https://www.rc.colorado.edu/policies/janus-jobs

Retrieved from "https://fluid.colorado.edu/wiki/index.php?title=Janus_Cheat_Sheet&oldid=323"