Running Ansys Electronics Desktop 2020 R2 using an LSF Job Scheduler (command line; without GUI)
More information about our Load Sharing Facility (LSF) job scheduler is given here.
The CAD Compute Cluster nodes can be displayed by typing lshosts at a command prompt in the environment as shown in Figure 1. You will see four login nodes (uwlogin*), two management nodes (uwlsfm*) and eight compute nodes in the list (uwmhpc*).
Figure 1. The nodes in the CAD Compute Cluster
When you run a simulation on our Cluster, you will submit your simulation request to an LSF queue. The job scheduler will try to assign the hardware (CPU cores and RAM) you want to your simulation run. Use the bqueues command to discover queue names.
Figure 2. The queues in the CAD Compute Cluster.
Summary: It is possible to run Ansys Electronics Desktop (EDT) software two different ways in a LINUX multi-core environment. One way is by using its Desktop interface. A second way is to create a script containing all the commands that are to be given to Ansys, and then submit this script to a job scheduler.
Here are the steps we have tested in our Cluster to demonstrate the second option.
- Test file: bp_filter.aedt example (bandpass filter). MPI software tested: Intel MPI w/Ansys 2020 R2.
- Log into the Cluster using one of our VCAD Cluster instances. Use the “module load ansys/20.2.cadconnect” declaration in your terminal window. Then type “module list” to see if the software has been loaded into memory.
- Use “mpitest” command to verify message-passing interface (MPI) software function.
- Use the setsshkey utility under /home/scripts to generate an SSH passcode. For simplicity, do not enter a password and leave the result in the default location; this is usually your home directory on a Cluster login node.
- Create a shell file to enter LSF batch submission (bsub) and the Ansys batch commands. See appendix for an example (“my_ansys_hfss.csh”).
- Upload user files to your working directory. We provide instructions for using MobaXTerm to perform a secure file transfer (sftp) upload.
- Upload your shell file (e.g. “my_ansys_hfss.sh”) to same directory. Ensure that the body of your shell file does not contain extraneous Windows editor characters (e.g. a carriage return).
- Use bsub < my_ansys_hfss.sh command at a prompt to run a simulation.
For this example, we use a prepared example (a bandpass filter, with filename bp_filter.aedt) supplied by Ansys support staff. You will find it in the list of examples supplied with your local Ansys EDT installation.
- Set the number of cores to a maximum of 32.
- Note that you will have to define the use of Ansys HPC pool licences in a registry.txt file in the batchoptions segment of the script. See Appendix A below for an example.
Use commands such as bjobs and bhosts during simulation runs to monitor simulation progress. For example:
~$ bjobs -l
Any text after a single # is visible to the LSF Job Scheduler and will be read as a possible command by LSF. Any command after ## is treated as a comment by both LSF and ANSYS.
## embedded options to bsub – start with #BSUB
# — Name of the job —
#BSUB -J ansys_HFSS_example
## — specify queue —
#BSUB -q adept
## — specify wall clock time to run as hh:mm —
##BSUB -W 04:00
# — specify the number of processors —
#BSUB -n 32
## — specify the number of nodes —
##BSUB -R “span[hosts=]”
## — user e-mail address —
##BSUB – email@example.com or other address
## — mail notification —
## — at start —
## — at completion —
## — Specify the output and error files. %J is the job ID —
## — -o and -e mean append, –oo and-eo mean overwrite —
#BSUB –oo BPFilter_%J.out
#BSUB –eo BPFilter_%J.err
# — example of an ansys command line call —
/CMC/tools/ansys/ansys.2020r2/AnsysEM20.2/Linux64/ansysedt -distributed -machinelist numcores=32 -auto -monitor -ng –useElectronicsPPE –batchoptions registry.txt –batchsolve /home/<user_name>/bp_filter.aedt
The registry.txt file included above contains a direct reference to the anshpc feature ‘pool’. If using this as an example, insert your assigned CMC Cluster user name in place of <user_name> below. Here is a copy of the file contents:
## — include other desired registry entries here —
Use of a registry.txt file is described in the ANSYS 2020 R2 help files.