A: Installing CASINO
You’ll need a Fortran 90 compiler, a UNIX environment with the bash shell, the [t]csh shell, and, if you have a parallel machine, an MPI library. Optionally, if you plan to use the provided plotting utilities, you should install xmgrace and gnuplot.
These dependencies can be installed automatically by the ‘install’ script, so go to A2 below. Keep reading if you would rather do this by hand.
Here are full setup commands for popular Linux distributions, which you can copy and paste into your terminal:
- Ubuntu 10.04+, Linux Mint 9+:
sudo apt-get install make gcc gfortran g++ tcsh openmpi-bin libopenmpi-dev grace gnuplot - Debian Lenny+ (5.0+):
su -c "apt-get install make gcc gfortran g++ tcsh openmpi-bin libopenmpi-dev grace gnuplot" - Fedora 9+, CentOS:
su -c "yum install make gcc gcc-gfortran gcc-c++ tcsh openmpi openmpi-devel grace gnuplot" - openSUSE 11.3+:
sudo zypper install make gcc gcc-fortran gcc-c++ tcsh openmpi openmpi-devel xmgrace gnuplot - Mandriva 2010.2+:
su -c "gurpmi make gcc gcc-gfortran gcc-c++ tcsh openmpi grace gnuplot" - Gentoo:
su -c "emerge make gcc tcsh openmpi grace sci-visualization/gnuplot" - Arch Linux:
su -c "pacman -S make gcc gcc-fortran tcsh openmpi grace gnuplot" - Slackware:
This distribution has no official package manager with automated dependency resolution. Slackware users, have fun. Both of you.
Notes:
- For openSUSE (up to at least version 11.3), after installing the packages above and before trying to compile the code, you may need to log out and back in for changes to take effect.
- As a bleeding-edge rolling-release distro, Arch Linux as of May 2011 is the first to hit a compilation problem with the default gfortran 4.6.0. This will get fixed eventually on gfortran’s side, or worked around on ours.
- Installing OpenMPI under Ubuntu 10.10 and 11.04 will pull a package called
blcr-dkmsas a ‘recommends’, but it does not compile against the Linux kernel versions distributed in either of these releases. Ignore the error message after installing the above packages, and runsudo apt-get remove --purge blcr-dkmsto remove the problematic package. It is not needed for normal operation. Ubuntu bug: https://bugs.launchpad.net/bugs/700036
.
After this, go to A2 below. Your CASINO_ARCH is ‘linuxpc-gcc‘ for the non-parallel version and ‘linuxpc-gcc-parallel‘ for the parallel one. Fedora 12 and later versions on multi-core systems should use ‘linuxpc-gcc-parallel.fedora‘ instead.
Comprehensive instructions here. Brief summary below.
Change into the CASINO directory and type ‘./install‘ then follow the prompts. This script helps you find the correct CASINO_ARCH for your machine, or to create a new one if you need to.
The script allows you to configure several CASINO_ARCHs for the same installation, e.g., allowing you to keep binaries from different compilers, and to share the installation on machines with different architectures.
The script creates a ~/.bashrc.casino file which is loaded on login and defines a function called casinoarch, so that at any point you can type ‘casinoarch‘ at the prompt to switch to the configuration that you choose.
You can run the install script as many times as you like, to do things like adding/removing CASINO_ARCHs or reordering them by preference.
Should the abilities of the script fail to satisfy your needs, you can always define a CASINO_ARCH by creating the file CASINO/arch/data/<CASINO_ARCH>.arch The syntax of .arch files is explained in CASINO/arch/README and in Appendix 5 of the manual.
In our tests the Intel Fortran compiler ifort often generates the fastest executables for x86-based processors (NB, the PathScale Fortran compiler is tied in speed with ifort on AMD processors). The Gnu GCC gfortran compiler is free, widely available and also often very fast. Basically, if you have access to multiple compilers on any given machine, you should just try each one and see what happens.
Here is a sample set of timings on 16 processors of a Cray XK6 system (Jaguar) in December 2011 (MDT). Again – Intel ifort wins, with Gnu gfortran a close second.
examples/crystal/blips/silicon
with the following keywords changed:
vmc_nstep : 800 #*! Number of steps (Integer)
vmc_nconfig_write : 800 #*! Number of configs to write (Integer)
dmc_equil_nstep : 20 #*! Number of steps (Integer)
dmc_stats_nstep : 20 #*! Number of steps (Integer)
dmc_target_weight : 800.d0 #*! Total target weight in DMC (Real)
Total CASINO CPU time DMC energy
Ifort : 54.6394 seconds -63.253019810592 +/- 0.007612017376
Gnu : 62.6400 seconds -63.253019816018 +/- 0.007612049672
Pathscale : 67.8962 seconds -63.259034999613 +/- 0.011000738147
PGF : 72.9000 seconds -63.245897141026 +/- 0.009693089487
Cray : 84.0300 seconds -63.192701255463 +/- 0.006393938977
NB: The pathscale compiler is being deprecated on Crays and is no longer
supported on Jaguar (as of June 2012).
Here is another set of runs for H on graphene (MDT 1.2013) on a Cray XK7 (Titan).
VMC
Gnu 79.65 seconds -282.890716861446 +/- 0.160222570613
PGF 83.74 seconds -282.895599950580 +/- 0.121131726135
Ifort 84.48 seconds -282.970112923443 +/- 0.128262416613
Cray 87.28 seconds -282.890716855184 +/- 0.160222562116
DMC
Gnu 397.20 seconds -284.256578870330 +/- 0.048408651743
Cray 456.63 seconds -284.256578740638 +/- 0.048408642717
Ifort 477.75 seconds -284.252588591933 +/- 0.117686071014
PGF 490.10 seconds -283.915347739145 +/- 0.077180929508
Moral: use the GNU compiler (CASINO_ARCH = linuxpc-gcc-pbs-parallel.titan).
Note: GNU and Cray answers essentially agree with each other, but Ifort (VMC) and PGF (DMC) are giving significantly different answers. This needs to be investigated.
Use the install script to create a new CASINO_ARCH. If you choose to edit the compiler options when prompted, you will be offered the choice of enabling external BLAS and LAPACK libraries (as opposed to compiling the BLAS and LAPACK source code included in the CASINO distribution). You will need to provide the linker flags required to link to these libraries (this depends on your setup, ask you system administrator or read your library’s documentation). For example, the Intel MKL is linked (at least in the Cambridge TCM group) by setting:
BLAS: -lmkl lguide -lpthread
LAPACK: -lmkl_lapack -lmkl -lguide -lpthread
NB: CASINO does not make particularly intensive use of linear algebra routines, so the improvements from doing this would normally be expected to be small. However, in favourable cases – and cases become more favourable as the system size increases – improvements of up to 20% have been observed. You should feel free to experiment; beware that doing this can make things slower as well..
Recent Linux distributions offer ready-to-use MPI-enabled compilers. Installing the relevant packages should provide you with an ‘mpif90‘ executable, an ‘mpirun‘ launcher, libraries etc, and you would not need to go through the instructions below (see A1 above instead).
If you want a different compiler, or if your distribution does not provide said package(s), follow these instructions:
- Install the Fortran compilers you want to use (
ifort,g95,gfortran, etc). - Download OpenMPI from
www.open-mpi.org. (NB, we use OpenMPI in this example, but you can choose to use any other MPI implementation so long as it fully supports MPI-2 [LAM/MPI can not be used].). - Extract from the archive and change into the newly created directory.
- Choose a directory for the installation. We will refer to this directory as (e.g. /opt/openmpi).
- For each compiler you want OpenMPI to work with, let:
<fc>= name of Fortran compiler binary,
<f77>= name of Fortran 77 compiler (may be the same as ),
<cc>= name of C compiler, and
<c++> = name of C++ compiler
<comp>= name under which we will refer to this setup.
(e.g., <fc>=<f77>=<comp>=ifort,<cc> =icc <c++>=icpc).
Configure OpenMPI with:
./configure CC=<cc> CXX=<c++> FC=<fc> F77=<f77> --prefix=<install-dir>/<comp>
Then run:
make all
sudo make install
make clean
Repeat this step with the next compiler. - Add <install-dir>/<comp>/bin to your PATH, and <install-dir>/<comp>/lib
to your LD_LIBRARY_PATH. Use the compiler as mpif90 / mpicc. - Refer to A2 above to set up CASINO for this compiler.
- To switch compilers, change your
PATH,LD_LIBRARY_PATHandCASINO_ARCH. Simple bash functions (which can be put in your.bashrcfile) can be written to simplify this task.
Short answer: no.
Long answer: you shouldn’t unless you know what you’re doing. CASINO is designed to be installed under the user’s home directory. If you want to do a system-wide installation we can provide no help, as we haven’t ever done this. A future version of CASINO might support this.
Contact your system administrator to check you have done the right setup. If you have, ask a question on the discussion forum.
a) In the linking stage, I get something like:
1586-346 (S) An error occurred during code generation.
The code generation return code was -1.
ld: 0706-005 Cannot find or open file: /tmp/ipajR8xEe
ld:open(): No such file or directory
1586-347 (S) An error occurred during linking of the object
produced by the IPA Link step.
The link return code was 255.
Your machine probably has a small CPU-time limit for interactive commands set by default. Type ‘limit cputime 10:00’ (or a larger value) and type ‘make’ again.
b) Why does the utilities compilation die on an IBM SP3 sometimes?
If you have a ‘TMPDIR‘ environment variable set, unset it. The compiler uses it and may be confused by it having a value already.
Use something like ‘jlimit processes 24‘ to increase the maximum
number of concurrent processes.
If you can get root privileges and xmgrace is available for your distribution in .rpm or .deb form, go for that.
The following are instructions for installing xmgrace as a non-root user when motif-compatible libraries are not available:
- Get the SOURCES of
grace(http://plasma-gate.weizmann.ac.il/Grace/) andlesstif(http://lesstif.sourceforge.net/), and untar them in a temp directory. - Go into the
lesstifdirectory and installlesstifby issuing:
$ ./configure --prefix $HOME/misc/lesstif
make all install - Go into the
gracedirectory and install it by issuing:
$ ./configure --prefix $HOME/misc --with-extra-incpath="$HOME/misc/lesstif/include:/usr/X11R6/include" --with-extra-ldpath="$HOME/misc/lesstif/lib:/usr/X11R6/lib: /usr/X11R6/lib64" --with-motif-library="-lXm -lXt -lXft -lXrender -lXext -lX11"
$ make ; make install - Then link
$HOME/misc/grace/bin/xmgraceinto your path (e.g.,$HOME/bin) for convenience. You should be able to startxmgraceby typing
‘xmgrace‘ at the command prompt.
CASINO was written for UNIX/Linux systems and is not supported directly under Windows. Nevertheless, it runs well enough in Windows using Cygwin, an emulation layer that provides a Linux-like compilation and execution environment for applications. Under Cygwin, CASINO works as if it’s running on a Linux workstation using the GCC compiler suite. Although Cygwin is considered an emulation layer, this only concerns system calls. The calculation speed of CASINO is native, and its performance is quite acceptable.
These instructions do not cover the installation and general use of Cygwin; please refer to Cygwin’s documentation for that. The instructions were tested under Windows 7 Ultimate 64-bit and Windows 8.1 Enterprise 64-bit, using Cygwin 1.7.28, with all Cygwin packages up-to-date as of March 2014.
Installation and compilation
1. Download and install Cygwin (www.cygwin.com), either 32-bit or 64-bit version.
2. Install at least the following packages and all their dependencies using Cygwin’s setup program (which is also its package manager) in addition to those packages that are installed by default:
gcc, g++, gfortran, make, bc, bzip2
3. If you want to run in parallel (which believe me, you do, if your machine has more than 1 CPU core) then you you should also install the following to get MPI support:
libopenmpi, libopenmpi-devel, libopenmpicxx1, libopenmpifh2, libopenmpiuse1, openmpi.
It’s not obvious if all these are necessary, but that’s what we installed and everything subsequently worked. Essentially, MPI should just work on Cygwin straight out of the box.
4. Set up your Cygwin environment so that you are using the bash shell (should be the default anyway).
5. Follow the instructions in the CASINO manual for downloading, unpacking, installing and configuring CASINO using its install script. The option [p] of the install script allows you to specify the value of CASINO_ARCH manually. The correct value is “cygwin-gcc” under Cygwin, or “cygwin-gcc-parallel” if, as is likely, you want to run CASINO in parallel. The auto-detect option of the install script should also be able to figure out the existence of these two CASINO_ARCHs all by itself (though because of a Cygwin bug preventing asynchronous operations, the auto-detection will run much more slowly than usual).
6. Use the install script’s compilation option to compile CASINO.
7. Run CASINO using the runqmc script from the Cygwin command line. Use of e.g. the -p flag to runqmc will allow you to run on more than one processor.
Technical Note about targeting Cygwin vs native environment under Windows
There are two ways to use the Cygwin environment to compile an application originally written for Linux and run it under Windows.
The first (recommended) is to compile it targeting the Cygwin environment. In this case, the application will require Cygwin to run, but will see an almost complete Linux-like environment, so usually little or no special care needs to be taken. The application does not need to be aware that it’s running on Windows. Specifying “cygwin-gcc” or “cygwin-gcc-parallel” as the value for CASINO_ARCH is intended to do this.
The second way is to compile the application to natively target Windows. You still need Cygwin for compilation (as it includes the necessary tools), but in principle the application then does not require Cygwin to run; it will run as a native Windows application. From our point of view this obviously involves extra headache because of issues such as the difference in path separator (/ vs \) and the fact that Windows doesn’t know what a symlink is. Specifying the CASINO make tag “NATIVE_WINDOWS = yes” in your arch file indicates that Unix filenames are to be converted to Windows filenames, and symbolic links are to be substituted with their targets. Specifying the “windowspc-gcc“, “windowspc-g95“, or “windowspc-ifort” value for CASINO_ARCH attempts to do this. However, note that doing so is not recommended, as it adds a needless layer of additional complication. Running outside of the Cygwin layer will also prevent the use of all of CASINO’s useful utilities, in particular the runqmc script. The windowspc-xx arches should therefore be considered to be obsolete and unsupported.
Visual cues
We at CASINO HQ understand that, as a Windows user, you are only able to contemplate performing actions for which there is a corresponding small picture of it on your computer that you are able to click. You will probably also find it helpful if an animation of a dog or something pops up after you type go, in order to say something like “Hi there. You appear to be running a quantum Monte Carlo calculation. Would you like me to help you with that..?“. We’re sorry, but CASINO doesn’t do either of these things, and we appreciate that your granny probably isn’t going to like that. If you wish to continue in your old Windows traditions, nonetheless, you might contemplate sending us a very large amount of money on a regular basis so that we can buy a yacht.
This warning message is generated at link time when the ‘--whole-archive‘ option and static linking of the Suse Linux-provided libpthread.a library is used. The warning message will not occur if the program is linked dynamically, and since that is what the general Linux community does, it is unlikely that SuSE will address this anytime soon. In all compilers except PathScale, Cray was able to remove these options and replace them with others.
Cray has added the following to the Cray Application Developer’s Environment User’s Guide (S-2396)
4.7.2 Known Warnings
Code compiled using the options --whole-archive,-lpthread get the
following warning message issued by libpthread.a(sem_open.o): warning:
the use of 'mktemp' is dangerous, better use 'mkstemp'. The
--whole-archive option is necessary to avoid a runtime segmentation
fault when using OpenMP libraries. This warning can be safely ignored.
In short, there is nothing you/we/Cray can do about this until SuSE addresses it.
File = /home/billy/CASINO/src/nl2sol.f90,
Line = 58, Column = 12
This numeric constant is out of range.
This appears to be a harmless pathscale/glibc bug – ignore.