Page 1 of 2

Changing decorr length in vmc for varmin_linjas and emin?!

Posted: Thu Sep 07, 2017 9:40 am
by Katharina Doblhoff
I made a strange observation in CASINO:
I do a varmin_linjas calculation and then use the pre-optimized result in an emin calculation. In the varmin_linjas calculation, for the vmc calculation of the parameters.casl that will be used in the emin calculation, I get:

Code: Select all

 In block : 1
...
 Correlation time              (steps)  = 1.9928E+00 +- 2.9654E-02
 No. of VMC steps per process           = 1303
...
 Time taken in block    : : :      618.6699
...
FINAL RESULT:
...
 -349.571859332862 +/- 0.001841494559      No correction
 -349.571859332862 +/- 0.002581542982      Correlation time method
But in the subsequent emin calculation (same number of vmc_nstep, vmc_decorr_period, vmc_nconfig_write) I get:

Code: Select all

 In block : 1
...
 Correlation time              (steps)  = 1.0789E+00 +- 1.1840E-02
 No. of VMC steps per process           = 1303
...
 Time taken in block    : : :     1303.8969
...
 FINAL RESULT:
...
 -349.575324654207 +/- 0.001901392864      No correction
 -349.575324654207 +/- 0.001979491390      Correlation time method
Clearly, the decorrelation period between the calculations seems to change. Why?
Thanks in advance for any suggestions that will help to solve this mystery!

Re: Changing decorr length in vmc for varmin_linjas and emin

Posted: Thu Sep 07, 2017 10:29 am
by Ryan_Hunt
Katharina,

Just to clarify, which VMC runs are leading to different correlation times?

Is it the correlation times in the emin config-generation VMC run and the correlation time in the varmin_linjas final VMC run that you are comparing? In this case, there's surely a bug (you are using identical wave functions here)

Is it instead the correlation times in the final emin VMC run and the final varmin_linjas VMC run? I.e. are you comparing two fully optimised wave functions, which have been optimised with different optimisation methods? In this case, the wave functions are different and it is perhaps unsurprising that the correlation time would be different.

Regards,
Ryan.

Re: Changing decorr length in vmc for varmin_linjas and emin

Posted: Thu Sep 07, 2017 11:12 am
by Katharina Doblhoff
Dear Ryan!
I am comparing the vmc of the last optimization in varmin_linjas, the parameters.casl file of which I use as initial guess in emin with the first vmc run in emin. So the parameters.casl should be the same (other than varmin, emin actually uses the parameters.cals file already in the first iteration -as I have recently learnt...)
Sounds a bit buggy to me too. But I do not see an obvious place where that error would be:, the run takes twice as long, while the correlation length is halved. This sounds suspiciously like a doubling of de decorr time. But actually, this would not explain the factor 2 either, because the intermediate vmc steps, for which the local energy is not computed should be significantly faster...
So, as I said, it looks like a mystery to me...

Re: Changing decorr length in vmc for varmin_linjas and emin

Posted: Thu Sep 07, 2017 11:32 am
by Ryan_Hunt
Katharina,

I am unable to reproduce this bug in a simple test with the current version (v2.13.614) of CASINO. May I ask:

(a) which version of CASINO are you using in this case?

(b) could you post your input files / do you have a minimal working example which displays the bug?

Regards,
Ryan.

Re: Changing decorr length in vmc for varmin_linjas and emin

Posted: Thu Sep 07, 2017 12:27 pm
by Katharina Doblhoff
Dear Ryan!
Thank you for looking into this.
I am using a somewhat old version (v2.13.569 from 2016-04-28) - admittedly with some changes to the code, which is why I like to stick to that version.
I could reproduce the problem with a small example for a Cu2 dimer (using an Ar core PP), the wavefunction of which I had lying around. In this case the observation is even worse: The time per vmc block (for the last parameters.casl in varmin_linjas, which is the first one in emin) increases from 2.8s per block to 9.9s and the correlation time decreases from 19 to 3 steps. I send you the input files below (I am aware that this is pretty crappy as input. I was just trying to generate a minimal script and just input some numbers...)
For varmin

Code: Select all

# SYSTEM
neu               : 11            #*!number of up electrons
ned               : 11            #*!number of down electrons
periodic          : F              #*! Periodic boundary conditions
atom_basis_type   : blip           #*! Basis set type
blip_periodicity  : 0              #*! Periodicity in 0D, 1D (x), 2D (xy), 3D(xyz)
complex_wf        : F              #*! Allow complex wfs

# RUN
runtype           : vmc_opt        #*! Type of calc
newrun            : T              #*! New run or continue old
random_seed       : 418            #*! Seed for rand number generator
max_real_time     :           90 hr#*! Max run time
vmc_nblock        :               2#*! VMC blocks

# VMC
dtvmc             : 0.12689        #*! vmc time step
opt_dtvmc     : 0                  #*! Do not optimize vmc time step
vmc_equil_nstep   : 1500          #*! Number of equilibration steps
vmc_decorr_period : 24             #*! Automatic detection of optimal vmc decorr steps is set to 0
vmc_nstep         : 250000         #*! Number of vmc steps
vmc_nconfig_write : 100000         #*! Number of vmc configs written
vmc_ave_period    : 1              #*! Number of steps to average before writing hist
postfit_keep_cfg  : T              #*! Do a last creation of configs

# OPTIMIZATION
opt_method        : varmin_linjas  #*! Opt method
opt_jastrow       : T              #*! Optimize Jastrow factor
opt_det_coeff     : F              #*! Optimize determinant coeffs
opt_orbitals      : F              #*! Optimize orbital parameters
opt_strict        : T              #*! Stop when energy increases
opt_fixnl         : F              #*! Fix NL potential
opt_cycles        : 4             #*! Number of optimization cycles
opt_backflow      : F              #*! Optimize backflow parameters

# GENERAL PARAMETERS
use_jastrow       : T              #*! Use a Jastrow function
for emin:

Code: Select all

# SYSTEM
neu               : 11            #*!number of up electrons
ned               : 11            #*!number of down electrons
periodic          : F              #*! Periodic boundary conditions
atom_basis_type   : blip           #*! Basis set type
blip_periodicity  : 0              #*! Periodicity in 0D, 1D (x), 2D (xy), 3D(xyz)
complex_wf        : F              #*! Allow complex wfs

# RUN
runtype           : vmc_opt        #*! Type of calc
newrun            : T              #*! New run or continue old
random_seed       : 419            #*! Seed for rand number generator
max_real_time     :           90 hr#*! Max run time
vmc_nblock        :               2#*! VMC blocks

# VMC
dtvmc             : 0.12689        #*! vmc time step
opt_dtvmc     : 0                  #*! Do not optimize vmc time step
vmc_equil_nstep   : 1500          #*! Number of equilibration steps
vmc_decorr_period : 24             #*! Automatic detection of optimal vmc decorr steps is set to 0
vmc_nstep         : 250000         #*! Number of vmc steps
vmc_nconfig_write : 100000         #*! Number of vmc configs written
vmc_ave_period    : 1              #*! Number of steps to average before writing hist

# OPTIMIZATION
opt_method        : emin           #*! Opt method
opt_jastrow       : T              #*! Optimize Jastrow factor
opt_det_coeff     : F              #*! Optimize determinant coeffs
opt_orbitals      : F              #*! Optimize orbital parameters
opt_strict        : T              #*! Stop when energy increases
opt_fixnl         : F              #*! Fix NL potential
opt_cycles        : 1             #*! Number of optimization cycles
opt_backflow      : F              #*! Optimize backflow parameters

# GENERAL PARAMETERS
use_jastrow       : T              #*! Use a Jastrow function
and my casl file:

Code: Select all

JASTROW:
  I am a random title
  TERM 1:
    Rank: [ 2, 0 ]
    Rules: [ 1-1=2-2 ]
    e-e basis: [ Type: natural power, Order: 4 ]
    e-e cutoff:
      Type: alt polynomial
      Constants: [ C: 3 ]
      Parameters:
        Channel 1-1:
          L: [ 4.0633, fixed ]
        Channel 1-2:
          L: [ 4.0633, fixed ]
  TERM 2:
    Rank: [ 1, 1 ]
    Rules: [ 1=2 ]
    e-n basis: [ Type: natural power, Order: 4 ]
    e-n cutoff:
      Type: alt polynomial
      Constants: [ C: 3 ]
      Parameters:
        Channel 1-n1:
          L: [ 4.0, fixed ]

Re: Changing decorr length in vmc for varmin_linjas and emin

Posted: Thu Sep 07, 2017 1:18 pm
by Ryan_Hunt
Issue also affects AE calculations...

Re: Changing decorr length in vmc for varmin_linjas and emin

Posted: Thu Sep 07, 2017 2:09 pm
by Pablo_Lopez_Rios
Hi Katharina,

Do the cut-off lengths change significantly in the first emin cycle? Are you using the same number of processes for your varmin_linjas and emin runs?

Correlation lengths are calculated directly from the local energy sample; I doubt a bug could account for a jump in correlation times without also giving rubbish for total energies.

Best,
Pablo

Re: Changing decorr length in vmc for varmin_linjas and emin

Posted: Thu Sep 07, 2017 2:39 pm
by Katharina Doblhoff
Dear Pablo, dear Ryan,
@Ryan: So you could reproduce the problem? What did you do differently beforehand, when you couldnot reproduce it? That might give a hint on what is happening.
@ Pablo:
I fixed the cutoff-lengths - so they do not change at all.
I also use identical submit scripts to start the calculations and I get "Running in parallel using 24 MPI processes." in both cases. So if nothing strange is going on, I am indeed running the same number of processes.

To me, it looks a bit as if the following were happening ('-' denotes a vmc step without calculating the energy, 'x' denotes a vmc step during which the energy is calculated and 'o' denotes a step during which the energy is calculated but not stored):
varmin_linjas: - - - - x - - - - x - - - - x
emin : - - - - o - - - - o - - - - x - - - - o - - - - o - - - - x - - - - o - - - - o - - - - x
In this case, the emin calculation would take 3 times as long, while the correlation time is reduced by a factor 1/3. (The 'o' are necessary, because the '-' would take much less time than the 'x'). Then, also the energy and the non-reblocked errorbar would be similar (as is indeed observed).
Why this might happen, is, however, totally unclear to me.

Re: Changing decorr length in vmc for varmin_linjas and emin

Posted: Thu Sep 07, 2017 2:49 pm
by Ryan_Hunt
Katharina,

Yes, I can reproduce the issue. Moreover, it doesn't seem specifically dependant on using emin in the second stage. Identical (disagreeing with the output of the varmin_linjas final VMC run) correlation times are found regardless of the new opt_method. Even if the new opt_method is varmin_linjas, there is a discrepancy. There are also discrepancies for the other mid-stage correlation.data/parameters.casl files (i.e. the bug seems to apply not only to the last varmin_linjas wave function, but also for each stage on the way).

Line 3145 of vmc.f90 may be to relevant...? (grep "Reduce VMC correlation period if"). Unsure this would affect either calc. differently, though...

Regards,
Ryan.

Re: Changing decorr length in vmc for varmin_linjas and emin

Posted: Thu Sep 07, 2017 3:04 pm
by Katharina Doblhoff
Dear Ryan,
Without looking into the code, I am pretty sure I know what you are talking about. But shouldn't that if statement only change anything if the vmc_nconfig_wrote is different in the two runs (which it is not - at least not in my example input files and the ones I used when I noticed the behavior)?
I will look into the bit of code... (after I bought some bicycle oil...)
Thanks,
Katharina