CASINO was designed to run on machines running a Linux/Unix operating system and related operating systems (this in principle includes Macs, since Mac OSX is based on BSD Unix and has a functioning bash command line). If you want to use Windows, then you need to install a Linux-like Cygwin environment and jump through some hoops – some advice is given in the FAQ – or better still, install a Linux virtual machine using software such as Virtualbox.
CASINO determines what kind of computer you are running on by looking at the value of the Unix environment variable CASINO_ARCH, which must be defined in your shell session. This tells CASINO to look into a particular file in a set of ‘arch’ files — permanently stored in the arch/data directory of the CASINO distribution — which contain instructions that the make shell command and various CASINO utilities can follow both for compiling the code and running calculations on the machine in question.
The various ways of setting this up — both for currently supported architectures/machines and for new unsupported ones — are described below.
The most convenient way to do this is to use the automatic utility — called ‘install‘ — which can help you to setup and compile CASINO on any given machine. It may be run by typing ‘./install‘ then pressing Enter in the base directory of the CASINO distribution, after which you should follow the prompts. You can re-run this installer any time you like to amend your configuration.
To give you an idea of what it does, note that the install script will present you with the following options:
[c] Compile CASINO for already-configured CASINO_ARCHs
[s] Sort/remove configured CASINO_ARCHs
[a] Auto-detect valid CASINO_ARCHs for this machine
[p] Pick a specific CASINO_ARCHs for this machine
[n] Create a new CASINO_ARCH for this machine interactively
[y] Install CASINO syntax highlighting for various text editors
[i] Install required software using package manager (requires root access)
[r] Restore the CASINO distribution directory to its original state
[q] Save configuration and quit the installer
[x] Quit the installer without saving
These options will be discussed in what follows.
Note for sysadmins: CASINO is not currently designed to be installed system-wide by the root user; rather, a separate copy should be installed by the user under his or her home directory. Amongst other reasons, this is because the CASINO distribution contains a huge number of utilities (with large numbers of executable files and scripts which most users of a multi-user machine will not require) along with examples and documentation which the user will wish to access.
DETAILED INSTRUCTIONS
We will assume you have obtained a CASINO tar archive following these instructions. For standard users this will be called something like ‘CASINO_v2.13.134_standard.tar.gz‘; for developers ‘CASINO_v2.13.134.tar.gz‘.
- Change to the directory where you want the distribution to live (this is usually assumed to be your home directory, but it need not be).
- Remove/rename any existing CASINO directory.
- Unpack the
CASINO_vxxx.tar.gzdistribution (‘tar xvfz CASINO_vxxx.tar.gz‘). This will result in a new directory calledCASINOcontaining the CASINO distribution. - If you want to maintain different versions of the code, it may be useful to rename the
CASINOdirectory to be something likeCASINO_v2.13.378and set up a symbolic link calledCASINOwhich points to it (‘ln -s CASINO_v2.13.378 CASINO‘). There is a CASINO utility ‘update_src‘ which will do this for you – just type e.g. ‘update_src 2.13.378‘ in a directory containing aCASINO_v2.13.378.tar.gzarchive.
Preliminary configuration of the machine
You need to ensure that the machine has all the relevant software installed (Fortran, C, and possibly C++ compilers, an MPI library etc.) For machines administered by other people, this should probably have been done for you.
Refer to the FAQ file for notes on the preliminary configuration of machines that you administer yourself.
The install script can help you with installing required software using the package manager (choose the [i] option).
Finding or creating the arch file
AUTOMATIC PROCEDURE
The install script can largely do this for you. The options which concern finding or creating the arch file are [a], [p], [n] and [s].
First of all, try the 'Auto detect' [a] option. The install script may detect an exact match for the particular machine you’re running on (i.e. someone else has already set CASINO up on it) — in which case, after you accept its recommendation, CASINO will simply work. The script can also suggest generic similar machines on which you can base your installation.
Alternatively, if you know which CASINO_ARCH you want, you can simply type in its name after selection of the [p] option.
Choosing the [n] option will take you through a guided procedure for creating your own personalized arch file (make sure you have the machine’s documentation handy so you can answer the questions the script will ask you). The result of this will be the arch file describing your machine that will be placed in the CASINO/arch/data directory (see below); new arch files can be emailed to the CASINO developers for permanent inclusion in the distribution. On extremely complicated machines the arch file produced by install may need to be tweaked by hand.
Multiple alternative configurations are supported. This includes setting up CASINO for use with multiple compilers – each of which will have its own arch file. One may also create set-ups where multiple machines share the installation directory, such as different-architecture queues on the same cluster, or workstations sharing their home directories over a networked file system. For the latter case, run the installer on one machine of each relevant type to set it up.
Once you have all your CASINO_ARCHs defined, you may sort them into a preferred order (perhaps to indicate a preferred compiler) or remove them using the [s] option.
Your defined setup can then be permanently saved using the [q] option. If you’re running in a bash shell, the list of defined CASINO_ARCHs will be stored in a hidden .bashrc.casino file in your home directory, which will be sourced from your .bashrc every time you log in. The install script also works if you use the tcsh or csh shells — though without the facility to change CASINO_ARCHs described below. Support for other shells is not provided.
The .bashrc.casino file defines a ‘casinoarch‘ function, which will allow you to switch between defined CASINO_ARCHs. For example, on the current (Jan 2013) fastest machine in the world — the Cray XK7 ‘Titan’ at Oak Ridge National Laboratory — there are four defined CASINO_ARCHs:
linuxpc-gcc-pbs-parallel.titan.arch (397.20 sec.)
linuxpc-cray-pbs-parallel.titan.arch (456.63 sec.)
linuxpc-ifort-pbs-parallel.titan.arch (477.75 sec.)
linuxpc-pgf-pbs-parallel.titan.arch (490.10 sec.)
referring to the four different available compilers (Gnu, Cray, Intel, Portland), and where the time quoted is the time required to run a test DMC run (you can see it’s worth paying attention to doing tests to see which compiler produces the fastest executable!). You can switch between these four arch files by typing ‘casinoarch gcc‘, ‘casinoarch cray‘, ‘casinoarch ifort‘, or ‘casinoarch pgf‘.
As an example of what using CASINO’s automated utilities saves you, note that Titan requires ‘modules’ to be loaded and unloaded to set up particular programming environments with different compilers. Both ‘make‘ and the CASINO run script ‘runqmc‘ will do this automatically on detecting the current ‘CASINO_ARCH‘, so there is no need for the user to understand or interact with the module system at all.
More details of all the above procedures are given in the section `The CASINO_ARCH system’ below.
MANUAL PROCEDURE
If you don’t wish to use a magic script (though we strongly suggest you do), then you may attempt the following manual installation.
For most machines, a suitable CASINO_ARCH data file will already exist in the /arch/data directory. If you have installed CASINO before, you likely know which one it is. You may also look through the data files manually to see which one is likely to correspond to your machine. If your machine is similar to, but not identical, to one already set up, then you may borrow one of the data files and suitably modify it (see the instructions below and in Appendix 5 of the manual).
Assuming, for example, that your CASINO_ARCH is ‘linuxpc-ifort‘, then you need to set this as a permanent environment variable in your shell. If you use, for example, the bash shell then you need to add something like
export CASINO_ARCH='linuxpc-ifort'
to your .bashrc file, and then source ~/.bashrc.
Compiling the code
Once you have sorted out the arch file, you may compile the code and the utilities in two different ways.
AUTOMATIC PROCEDURE
Run the install script as before and this time select the [c][/c] option (‘Compile CASINO for already-configured CASINO_ARCHs‘). The script will respond with a list of defined CASINO_ARCHs and the following text (using the Titan machine as an example):
The following CASINO_ARCHs are configured (it is possible that not all of them can be compiled from this machine depending on your set-up):
[1] linuxpc-gcc-pbs-parallel.titan
[2] linuxpc-cray-pbs-parallel.titan
[3] linuxpc-ifort-pbs-parallel.titan
[4] linuxpc-pgf-pbs-parallel.titan
At the prompt below enter the numbers corresponding to the CASINO_ARCHs you would like to compile, separated by spaces.
You can specify which optional compile-time features to enable appending ':<feature>' to each number. Available <features> include:
- 'Openmp' for building OpenMP support
- 'Shm' for building the SMP shared-memory blips facility
- 'OpenmpShm' for building a version with both features enabled
One may thus type ‘1‘ to compile with the gcc compiler, ‘2‘ to compile with the Cray compiler etc.. One may compile special versions such as the shared-memory version of CASINO with the gcc compiler by typing ‘1:Shm‘.
We recommend that this automatic procedure is used, essentially for three reasons:
- One can build multiple CASINO executables with a single typed command such as ‘
1 1:Shm 1:Openmp 1:OpenmpShm 2 3:debug 4:Shm‘. - The
installscript will runmakein parallel as far as it can over multiple cores (this can of course be done on the command line, but the user may not know how). - On some obscure Unix machines, the default version of ‘
make‘ will work with a sufficiently different syntax that the CASINOMakefileis not interpreted correctly. Theinstallscript will know to run an alternative version ofmakewhich we know works, such as GNU’s ‘gmake‘ whereas just typing ‘make‘ on the command line will not work.
MANUAL PROCEDURE
The alternative manual procedure is to sit in the base CASINO directory and type ‘make‘.
Different versions of the code may be compiled, e.g., by:
make : standard version with optimizing compiler flags
make debug : standarc version with debug compiler flags
make Shm : shared memory version with optimizing compiler flags
make Shm/debug : shared memory version with debug compiler flags
make Openmp : OpenMP version with optimizing compiler flags
make Openmp/debug : OpenMP version with debug compiler flags
make OpenmpShm : OpenMP and Shm version with optimizing compilerflags
etc..
Note that the Makefile also supports all-lower-case versions of the above (e.g. make shm, make openmpshm etc.) even though these are less legible.
The make procedure is fully documented in the preamble of the file src/Makefile. More generally, src/Makefile works as follows:
Generic form of target:
<features>/<version>.<action>
where <version> can be:
- opt : use full optimization (default).
- debug: use debugging compiler flags.
- dev : use full optimization, but keep object files and binary
separate from 'opt' version.
- prof : use profiling compiler flags (if a profiler is available).
<features> can be:
- NoFeatures: no special features (default).
- Openmp : build OpenMP support.
- Shm : build SHM support.
- OpenmpShm : build OpenMP and SHM support (currently broken).
and <action> can be:
- <empty>: perform compilation (default).
- clean : remove object files.
- vclean : remove object files and binaries.
E.g., one can type:
make
make opt
make NoFeatures
make NoFeatures/opt
make nofeatures/opt
to the same effect.
For the 'clean' and 'vclean' targets, 'all' can be used to refer
to all features or versions, e.g.:
make all.clean # same as 'make NoFeatures/all.vclean'
make all/dev.vclean
make Openmp/all.vclean
make openmp/all.vclean
make all/all.vclean
NOTE: the above clean targets only work if sitting in the CASINO/src
directory. Typing 'make clean' in the CASINO base directory is equivalent to
'make all/all.clean' (remove all object files for all features and versions)
and 'make all/all.vclean' (remove all object files and all binaries for all
features and all versions). Or at least it as after version 2.13.279. Hope
that's clear.
You can also type ‘make info‘ in the base directory to get a print out of all the compilers and compiler flags that CASINO will use, without invoking any actual compilation.
To run the code you need to add the $HOME/CASINO/bin_qmc directory to your shell path, e.g., through (bash):
export PATH="$PATH:/YOUR_HOME_DIRECTORY/CASINO/bin_qmc"
so that your shell can pick up all the executable CASINO utilities. (The install script will do this automatically).
Running the code
You run the code using the ‘runqmc‘ script (see the manual or type ‘runqmc --help‘). This script is able to access all the run time information in the arch file to determine how to run jobs on your machine – even to the extent of loading modules, writing batch scripts, and submitting them for you.
You can run the various utilities just by typing their name.
The CASINO_ARCH system
In this section, we give further details of how some of these things work.
As already stated, CASINO determines what kind of computer you are running on by looking at the value of the Unix environment variable CASINO_ARCH, which must be defined in your shell session. The ‘arch/data‘ subdirectory of the CASINO distribution contains architecture data files which define system-specific parameters for compiling and running the code. Include files in this directory are named <CASINO_ARCH>.arch, and contain information about which set of parameters to use, including compiler name, compiler flags, library locations, how to submit jobs, etc. CASINO_ARCH should be set permanently on your machine, and will be re-used when installing future versions.
While there is no fundamental difference between CASINO_ARCHs, we define two conceptual types for convenience, which simply differ in purpose and naming convention:
(1) ‘Generic’ CASINO_ARCHs are intended to represent a class of systems. Their name is typically of one of these forms:
- Single-processor workstations:
<system>-<compiler> e.g. linuxpc-ifort
- Multi-processor workstations:
<system>-<compiler>-parallel e.g. linuxpc-ifort-parallel
- Clusters with queueing systems:
<system>-<compiler>-<queueing-system>-parallel e.g.linuxpc-ifort-pbs-parallel
(2) ‘Extended’ CASINO_ARCHs are intended to represent specific systems, and are usually modifications to existing generic CASINO_ARCHs. Their name is of the form:
<generic-name>.<specific-system-name> e.g. linuxpc-ifort.berts-weird-computer
The corresponding .arch file is typically intended to ‘include’ its generic counterpart, if it exists, but again this is just a guideline.
See the files in the CASINO/arch/data directory for the range of both generic and extended CASINO_ARCH names. If you end up generating your own unique arch file then you may send it to Mike Towler (mdt26 at cam.ac.uk) who can incorporate it permanently into the distribution.
Very large machines in national computer facilities often have specialized setups and requires Extended CASINO_ARCHs. Some examples of current top-of-the-range hardware:
* Titan (Oak Ridge, USA): linuxpc-<compiler>-pbs-parallel.titan
where <compiler> is pgf, cray, ifort or gcc.
* Hector (UK national facility): linuxpc-<compiler>-pbs-parallel.hector3
where <compiler> is pgf, path, cray or gcc.
* Darwin (Cambridge HPCF facility, UK):
linuxpc-ifort-pbs-parallel.darwin2 (Westmere partition), or
linuxpc-ifort-pbs-parallel.darwin3 (Sandy Bridge partition)
* Intrepid (IBM Blue Gene/P, Argonne, USA):
bluegene-xlf-cobalt-parallel.intrepid
* Blue Joule (IBM Blue Gene/Q, Hartree Centre, U.K.)
bluegene-xlf-ll-parallel.bluejoule
The full syntax of arch files is explained in the file CASINO/arch/README. This information is duplicated in Appendix 5 of the manual.
Further installation notes
- The
[r]option of the install script will restore the CASINO distribution directory to its original state for compilation purposes. This involves removing all binary executables, links to scripts, and compiler object files, which is achieved practically by removing the following directories:bin_qmc,lib/zlib,src/zlibandutils/zlib. - If you use the
vim,emacs, orgedittext editors then it is possible to configure them to highlight the syntax of CASINO’s arch files and the various input files. See theREADMEfile inCASINO/data/syntaxor just select the[y]option of the install script to automatically do the configuration. - For versions of CASINO prior to 2.10, a different setup system was used, involving environment variables
QMC_ARCHandQMC_ID(the latter for customization on specialized machines). These may be retained if you wish to continue to use older versions of the code, but the setup will need to be redone for version 2.10 and later.
Discussion forum
This site includes a discussion forum, where users may discuss issues relating to quantum Monte Carlo and the CASINO code. Expert users of the code are known to haunt the forum, and thus it may be useful for soliciting advice if problems are encountered in installing or using CASINO.
MDT 1.2013