Warning: no access to tty (Bad file descriptor).
Thus no job control in this shell.
set sim = 1PO5_tmi/angle
set mpirun = mpirun -np 32 -machinefile /uufs/updraft.arches/host/var/torque/aux//37164.uprm.privatearch.arches
set pmemd = ~/pmemd-a10-updraft 
set cur = 13
set pre = 12
cd ~/biodata2/P450/1PO5_tmi/angle
mpirun -np 32 -machinefile /uufs/updraft.arches/host/var/torque/aux//37164.uprm.privatearch.arches /uufs/chpc.utah.edu/common/home/u0181375/pmemd-a10-updraft -O -i mdin -o mdout.13 -p prmtop -c restrt.12 -r restrt.13 -x traj.13 -e ener.13
MPIRUN.up012: MPI progress Quiescence Detected after 900 seconds.
MPIRUN.up012: 32 out of 32 ranks showed no MPI send or receive progress.
MPIRUN.up012: Per-rank details are the following:
MPIRUN.up012: Rank    0 (up012           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank    1 (up012           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank    2 (up012           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank    3 (up012           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank    4 (up012           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank    5 (up012           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank    6 (up012           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank    7 (up012           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank    8 (up049           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank    9 (up049           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   10 (up049           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   11 (up049           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   12 (up049           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   13 (up049           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   14 (up049           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   15 (up049           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   16 (up050           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   17 (up050           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   18 (up050           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   19 (up050           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   20 (up050           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   21 (up050           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   22 (up050           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   23 (up050           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   24 (up051           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   25 (up051           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   26 (up051           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   27 (up051           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   28 (up051           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   29 (up051           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   30 (up051           ) caused MPI progress Quiescence.
MPIRUN.up012: Rank   31 (up051           ) caused MPI progress Quiescence.
MPIRUN.up012: both MPI progress and Ping Quiescence Detected after 900 seconds.
MPIRUN.up012: 32 out of 32 ranks showed no MPI send or receive progress.
MPIRUN.up012: 32 out of 32 ranks could not return a ping request.
MPIRUN.up012: Per-rank details are the following:
MPIRUN.up012: Rank    0 (up012           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank    1 (up012           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank    2 (up012           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank    3 (up012           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank    4 (up012           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank    5 (up012           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank    6 (up012           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank    7 (up012           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank    8 (up049           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank    9 (up049           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   10 (up049           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   11 (up049           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   12 (up049           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   13 (up049           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   14 (up049           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   15 (up049           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   16 (up050           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   17 (up050           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   18 (up050           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   19 (up050           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   20 (up050           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   21 (up050           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   22 (up050           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   23 (up050           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   24 (up051           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   25 (up051           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   26 (up051           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   27 (up051           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   28 (up051           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   29 (up051           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   30 (up051           ) caused both MPI progress and Ping Quiescence.
MPIRUN.up012: Rank   31 (up051           ) caused both MPI progress and Ping Quiescence.


**** Segmentation fault!  Fault address: (nil)

This is likely to have been caused by either a null pointer dereference or a general protection fault.

pmemd-a10-updraft:7021 terminated with signal 6 at PC=2a9604626d SP=403ff5c8.  Backtrace:
/lib64/tls/libc.so.6(gsignal+0x3d)[0x2a9604626d]
/lib64/tls/libc.so.6(abort+0xfe)[0x2a96047a6e]
/uufs/arches/sys/pkg/pscale/3.2/lib/3.2/libpathfortran.so.1(__f90_segv+0x289)[0x2a95999f3b]
/lib64/tls/libc.so.6[0x2a96046300]
/lib64/tls/libc.so.6(_IO_vfprintf+0xcb)[0x2a960578cb]
/lib64/tls/libc.so.6(vsprintf+0x79)[0x2a96075319]
/uufs/arches/sys/pkg/pscale/3.2/lib/3.2/libpathfortran.so.1(_lmessage+0x109)[0x2a95af2c72]
echo Done with 1PO5_tmi/angle...
Done with 1PO5_tmi/angle...