D: Using CASINO with external programs
Note this sort of advice is likely to date quickly.
UK Hector machine (PWSCF current SVN version 26/11/2011)
None of the four compilers works by default – I (MDT) eventually
managed to make the GNU one work. There may be other ways.
Check current default module with ‘module list‘ – if PrgEnv-gnu is not
listed then run the relevant one of the next three lines’
module unload PrgEnv-path
module unload PrgEnv-cray
module unload PrgEnv-pgi
then
module load PrgEnv-gnu
then in the espresso base directory
./configure ARCH=crayxt
Edit the make.sys file that gets produced, changing the CFLAGS line from
CFLAGS = -fast $(DFLAGS) $(IFLAGS)
to
CFLAGS = -O3 $(DFLAGS) $(IFLAGS)
Then type ‘make pw‘.
JaguarPF
Having loaded module PrgEnv-pgi:
./configure ARCH=crayxt4
make pw
UK Hartree Centre – Blue Joule Blue Gene/Q (Espresso version 5.02 Feb 2013)
This requires a bit of hacking. Not all of the following may be necessary, or even the right thing to do (you’re supposed to fiddle with some preliminary files in the install directory) but this is a recipe that
worked for me.
module load scalapackcd ~/espresso-
./configure - This will create
~/espresso/make.sys, in which you should change the
following things (again, these may not all be necessary but I couldn’t be bothered to check):MPIF90 = mpixlf90to
#F90 = /opt/ibmcmp/xlf/bg/14.1/bin/bgxlf90_r
CC = /opt/ibmcmp/vacpp/bg/12.1/bin/bgxlc_r
F77 = /opt/ibmcmp/xlf/bg/14.1/bin/bgxlf_rMPIF90 = mpixlf90_r
CC = mpixlc_r
F77 = mpixlf77_r
LD = /opt/ibmcmp/vacpp/bg/12.1/bin/bgxlc_r -qarch=qp -qtune=qptoLD = mpixlf90_r -qarch=qp -qtune=qpLD_LIBS =toLDLIBS = -L/opt/ibmcmp/xlf/bg/14.1/lib64 -lxlopt -lxl -lxlf90_r -lxlfmath– and the Blas/Lapack libs should be changed from whatever they are toBLAS_LIBS = -L/bgsys/ibm_essl/prod/opt/ibmmath/essl/5.1/lib64/ -lesslbg
BLAS_LIBS_SWITCH = externalLAPACK_LIBS = -L/bgsys/ibm_essl/prod/opt/ibmmath/essl/5.1/lib64/ \Note that the 3.4.2 version of the lapack directory might have changed by the time you come to read this – check this exists and has
-L/gpfs/packages/ibm/lapack/3.4.2/lib -lesslbg -llapack
LAPACK_LIBS_SWITCH = externalliblapack.ain it. - Then type ‘
make pw‘ in~/espresso.
Various versions of PWSCF from summer 2011 produce an apparently miscompiled executable, which when told to print out a list of k points in the output file (or in xwfn.data), just prints out a string of zeroes i.e. all k points are listed as (0.0 0.0 0.0).
Having dug around in the PWSCF source code, it seems that the k point grid was being defined before the reciprocal lattice vectors, hence all the k points really were zero. I have no idea why no-one noticed this – you would think it would be pretty fundamental. Looking at the latest (26/11/2011) this has now been fixed.
Solution : upgrade your PWSCF.
EDIT: later investigation showed this problem was extant between commit 8051 and commit 8121.
The casino2upf utility marks any UPF files it creates as having been generated using Hartree-Fock (since they generally are). If you do not supply a value for the ‘input_dft‘ keyword in the `system’ section of the PWSCF input file, then PWSCF will attempt to use the functional specified in the pseudopotential file i.e. it will try to do a Hartree-Fock calculation, and — given that this is only possible with PWSCF if you compiled it having invoked ‘configure‘ with the ‘--enable-exx‘ flag — then the code may stop and whine about not having been compiled with support for hybrid functionals. This can be confusing. Solution: specify ‘input_dft‘ in the input file.
According to CRYSTAL author Roberto Orlando:
“There was a stupid restriction in the input for pseudopotentials that limited the maximum allowed angular quantum number to L=4, even if algorithms are general. Thus, we have extended it to L=5. Unfortunately, this implies the addition of one datum in the record below INPUT. This change is reported in the manual, but everybody using pseudopotentials now fails. Maybe we should change the error message …”
There are clearly backwards compatible ways in which this could have been done but anyhow, the point is that from CRYSTAL09 onwards, all input decks constructed from pseudopotentials obtained from the CASINO pseudopotential library before Feb 2012 will fail.
The solution is to add an extra zero to end of the second line of each pseudopotential (effectively stating that your pseudo contains no g functions). Thus:
INPUT
1.000 8 8 8 0 0
51.12765602 1.00000000 -1
38.05848906 -860.41240728 0
etc..
becomes
INPUT
1.000 8 8 8 0 0 0
51.12765602 1.00000000 -1
38.05848906 -860.41240728 0
etc..
On 14/2/2012 MDT converted all the files in the CASINO pseudopotential library and in the examples to reflect this change so everything should now work with CRYSTAL09.
The converters in utils/wfn_converters/crystal0[6,9] were broken for a period of several years. The error was introduced in patch 2.4.41 and fixed in 2.11.5 (and backfixed into the official 2.10 release).
The essence of the problem was that the block of orbital coefficients for the down-spin orbitals was incorrectly just a copy of those for the up-spin-orbitals, rather than being the correct down-spin ones. Apologies for our having taken so long to notice this.
To fix this problem manually:
Fine line 466 in the following two files:
CASINO/utils/wfn_converters/crystal06/casino_interface.f90
CASINO/utils/wfn_converters/crystal09/casino_interface.f90
In the incorrect version of these files, this should read:
ak_pointer(1,:)=0
Change this to:
ak_pointer(1,1)=0
ak_pointer(1,2)=ndfrf
Like runqmc, the runpwscf script uses the CASINO architecture scheme in order to know how to run calculations on any given machine. The data files containing the machine definition is in CASINO/arch/data/xx.arch (the ‘arch file’).
If your CASINO arch file defines a command for running CASINO and PWSCF (such as SCRIPT_RUN: mpirun -np &NPROC& &BINARY&), then it must include a tag &BINARY_ARGS& following the &BINARY& tag. This is because the PWSCF executable takes command line arguments such as ‘-pw2casino‘, ‘-npool‘ and ‘< in.pwscf >> out.pwscf‘ etc. which are not required by CASINO. For this reason an arch file which works for CASINO may not necessarily work for PWSCF without this modification. To check this on a batch machine, type ‘runpwscf --qmc --check-only‘ and examine the resulting ‘pw.x‘ batch file. If the line containing e.g. the mpirun command above does not have ‘-pw2casino‘ following ‘pw.x‘, then you will need to add &BINARY_ARGS& to your arch file.
Use VNC (http://en.wikipedia.org/wiki/Virtual_Network_Computing).
Here is a recipe that I (MDT) got to work on my machines (at work and at home) in connection with the Cetus machine at Argonne.
(1) If you are trying to do this at home without a static IP address, then subscribe to a free dynamic DNS service like http://www.noip.com/ . You end up with a daemon permanently running on your home machine that sends your current IP address (whenever it changes) to Noip who keep it linked with a name like ‘mikes_house.no-ip.biz‘. Then you can do ‘ssh mikes_house.no-ip.biz‘ from anywhere and it supposedly will figure out the current correspinding IP address.
Also make sure you instruct your router to open the ssh port and to forward any ssh traffic to the machine you want (which you will have setup with a static IP like 192.168.0.42 or whatever on your internal network).
You might also need to tell your home machine to open the correct ports for VNC (with OpenSuse Linux there’s an option for this in Yast2).
(2) On Cetus, type:
vncserver -geometry 1920x1080
It will respond
New 'cetuslac1:2 (your_user_name)' desktop is cetuslac1:2
Keep a note of the ‘:2‘ – this is the port number that you will need (and this might be different every time you do this).
(3) On Cetus, type:
ssh -c blowfish -C -f -R5902:localhost:5902 mdt26@mikes_house.no-ip.biz
vncviewer -display :0 localhost:2
where all occurrences of :2 and 02 refer to the port number from step (2).
The Totalview user support guy told me to add
-encodings '"tight zlib copyrect hextile rre corre raw"'
to the list of arguments to ssh, but that either corrupted my display, or had no effect. I’m not sure what this is supposed to do (and can’t be bothered to look it up – feel free).
(4) You should then get a big window containing a desktop (which will either be a fully functioning clone of the desktop on your home machine, or a partially functioning black desktop where you can’t e.g. resize windows, depending on which machine you use. I don’t know how to fix the black case – nobody said this is easy!).
(5) Inside the desktop should be a terminal window with which you can issue commands on Cetus. On Cetus, you would then do
casinoarch bluegene-xlf-cobalt-parallel.cetus_totalview.arch
then, e.g.
runqmc -n 1 --ppn=2 -T 1h -s --user.shmemsize=1000 --debug
Totalview will then launch inside the virtual desktop, and when you press a button it should respond quickly, rather than taking half an hour over it, as would happen with ‘ssh -X‘.
Then, assuming you’ve studied the Totalview manual for several days, you ought to be able to debug CASINO. Various bizarre behaviours may be encountered if you try to do this, but it essentially works.
(6) Once you’ve finished with it, close the Totalview window, then issue the following command on Cetus:
vncserver -kill :2
where :2 is the port number from step (2).
Note finally that Totalview also produce a ‘Remote Display Client’ which will supposedly do all this quasi-automatically, and which you can download and install this on your personal machine. However, not even the Totalview user support guys can make it work with Cetus (and I did ask).
Unmodified versions 5.1 or 5.1.1 of the PWSCF DFT code (extant from April –> December 2014) contain a bug introduced by the developers that affected the CASINO converter routine. SVN development versions from December 2014 contain a fix for this bug, as will subsequent releases.
To fix the bug manually, then replace line 366 of PW/src/pw2casino_write.f90
CALL get_buffer (evc, nwordwfc, iunwfc, ikk )
with
IF( nks > 1 ) CALL get_buffer (evc, nwordwfc, iunwfc, ikk )