varmin gives NaN

General discussion of the Cambridge quantum Monte Carlo code CASINO; how to install and setup; how to use it; what it does; applications.
Katharina Doblhoff
Posts: 84
Joined: Tue Jun 17, 2014 6:50 am

varmin gives NaN

Post by Katharina Doblhoff »

Dear all,
I have a system that kind of likes to diverge in varmin, suddenly giving NaN for all coefficients (after doing a few successful optimization steps), giving me the following warning (which, I believe is just a consequence of all Jastrow parameters being NaN - or is it the other way round???):

Code: Select all

Warning: [CHECK_KE] This may indicate a problem. Continuing run, however, 
because the KE_FORGIVE flag is set to T.
I can usually make it behave well by changing the input a bit (slightly more configs or even just a slightly longer equil run), but since it keeps doing this and since I have never had the problem before, I started wondering whether there is some real issue with the system that I should care about. I would be astonished about the kinetic energy going nuts since I am using PPs.

Thanks for comments,
Katharina
Pablo_Lopez_Rios
Posts: 53
Joined: Thu Jan 30, 2014 1:25 am

Re: varmin gives NaN

Post by Pablo_Lopez_Rios »

Hi Katharina,

Does fixing the cut-off parameters help? You can fix them without modifying the parameters files by setting OPT_NOCTF_CYCLES to the same value as OPT_CYCLES in your input file.

I understand you are using OPT_METHOD=varmin. Do other optimization methods (madmin/emin) run into the same problem?

Best,
Pablo
Hey there! I am using CASINO.
Katharina Doblhoff
Posts: 84
Joined: Tue Jun 17, 2014 6:50 am

Re: varmin gives NaN

Post by Katharina Doblhoff »

Dear Pablo,

I haven't tried to fix the cutoffs, because the entire reason why I am running varmin is to optimize those. In emin, I have not seen this problem, but what I have seen is that I needed to move to a newer lapack version that has an updated SVD that allows for more cycles. Madmin, I have never tried - I will.

In any case, from what I understand, you would look for the issue in the optimization procedure rather than in an issue with the system (problems with the wf file, problems with the PP, ...), right?

Best,
Katharina
Pablo_Lopez_Rios
Posts: 53
Joined: Thu Jan 30, 2014 1:25 am

Re: varmin gives NaN

Post by Pablo_Lopez_Rios »

Hi Katharina,

In the first instance, yes. I assume your initial VMC cycle runs fine without a Jastrow factor, which would indicate that there is no obvious problem with the orbitals and pseudopotentials.

I would recommend not using varmin at all, and using madmin in its stead.

Are the cutoffs set to very large or very small values during your failed optimizations? If you are using the "gjastrow" Jastrow factor in parameters.casl you can set custom hard limits on their values to prevent this -- the "gjastrow" implementation is also much faster during optimization, which should be handy for this.

If you use enough configurations (~ 200k - 1M) it should be possible to get emin to optimize your cutoffs.

Hope this helps,
Pablo
Hey there! I am using CASINO.
Neil Drummond
Posts: 113
Joined: Fri May 31, 2013 10:42 am
Location: Lancaster
Contact:

Re: varmin gives NaN

Post by Neil Drummond »

To be honest I don't usually bother optimising cutoff lengths these days. It's usually more important to be consistent with one's choice of cutoff lengths.

When an optimisation goes wrong try the following: reducing the number of parameters (can increase it again later); increasing the number of configurations; reducing the (initial) cutoff lengths; switching optimisation method; ...

Best wishes,

Neil.
Katharina Doblhoff
Posts: 84
Joined: Tue Jun 17, 2014 6:50 am

Re: varmin gives NaN

Post by Katharina Doblhoff »

Dear Neil, dear Pablo,
thank you for your replies.
You got me confused in several points:
1.) When you say: " to be consistent with one's choice of cutoff lengths", you mean: "Use the same cutoffs for all calculations when taking energy differences", right?
2.) Whenever I tried to reduce the number of parameter first and to increase the number of parameters later using those that I had and setting the rest to zero, I observed my optimization to go nuts.... From what you say, I gather, that this should not happen and that I most likely always did something wrong there, right?
Best,
Katharina
Neil Drummond
Posts: 113
Joined: Fri May 31, 2013 10:42 am
Location: Lancaster
Contact:

Re: varmin gives NaN

Post by Neil Drummond »

Dear Katharina,

Yes, sorry, I meant that it is usually best to aim for consistency in the cutoff lengths between different calculations when calculating energy differences or calculating results to be combined in some way. For example, when studying condensed matter I usually set the cutoff length for u to the Wigner-Seitz cell radius, but fix the cutoff lengths of chi and f as I increase the size of the simulation cell.

When increasing the number of parameters, you need to be careful about where the new parameters go. E.g., if you increase N_u from 4 to 6 with a spin-dependence 1, you need to add zeroes. E.g.,

Code: Select all

START U TERM
 Number of sets
   1
 START SET 1
 Spherical harmonic l,m
   0 0
 Expansion order N_u
   4
 Spin dep (0->uu=dd=ud; 1->uu=dd/=ud; 2->uu/=dd/=ud)
   1
 Cutoff (a.u.)     ;  Optimizable (0=NO; 1=YES)
   6.0000000000000000                0
 Parameter values  ;  Optimizable (0=NO; 1=YES)
   2.4089364603250654E-003           1       ! alpha_0,1
   3.9451302584501399E-004           1       ! alpha_2,1
  -4.4157132792821229E-004           1       ! alpha_3,1
   2.3105407201268982E-004           1       ! alpha_4,1
   4.3644837189526784E-003           1       ! alpha_0,2
   1.6966410602089975E-004           1       ! alpha_2,2
  -4.2750833764570401E-005           1       ! alpha_3,2
   1.1699982334274643E-004           1       ! alpha_4,2
 END SET 1
 END U TERM
should become

Code: Select all

START U TERM
 Number of sets
   1
 START SET 1
 Spherical harmonic l,m
   0 0
 Expansion order N_u
   6
 Spin dep (0->uu=dd=ud; 1->uu=dd/=ud; 2->uu/=dd/=ud)
   1
 Cutoff (a.u.)     ;  Optimizable (0=NO; 1=YES)
   6.0000000000000000                0
 Parameter values  ;  Optimizable (0=NO; 1=YES)
   2.4089364603250654E-003           1       ! alpha_0,1
   3.9451302584501399E-004           1       ! alpha_2,1
  -4.4157132792821229E-004           1       ! alpha_3,1
   2.3105407201268982E-004           1       ! alpha_4,1
   0.0 1
   0.0 1
   4.3644837189526784E-003           1       ! alpha_0,2
   1.6966410602089975E-004           1       ! alpha_2,2
  -4.2750833764570401E-005           1       ! alpha_3,2
   1.1699982334274643E-004           1       ! alpha_4,2
   0.0 1
   0.0 1
 END SET 1
 END U TERM
... But I'm sure you know that already!

Best wishes,

Neil.
Katharina Doblhoff
Posts: 84
Joined: Tue Jun 17, 2014 6:50 am

Re: varmin gives NaN

Post by Katharina Doblhoff »

Dear Neil,
Thanks for elaborating!
I will play a bit with my system...
Best,
Katharina
Katharina Doblhoff
Posts: 84
Joined: Tue Jun 17, 2014 6:50 am

Re: varmin gives NaN

Post by Katharina Doblhoff »

Dear Pablo!
One last question (actually not related to NaNs any more, but to your suggestion of using the casl):
If I specify the rule 1-1=2-2 (and not 1=2) in the e-e-n term, I always get a problem with the parameters:

Code: Select all

[REDO_GAUSSIAN_ELIMINATION] non-linear parameters produce 2 zero(es)
 in the coefficient matrix for the constraint equations.  We recommend that
 you stop this calculation, change the values of any basis parameters involved
 to non-integer values, all slightly different from each other, and re-run to
 avoid possible problems.  This run will continue in the hope that the problem
 will go away during optimization.  The affected equations and parameters are
 listed below.

 Zero #1 encountered at equation #8, parameter #17.
 Zero #2 encountered at equation #9, parameter #18.
This sounds strange to me, since I can easily do that in the correlation.data file (using spin rule 1). I guess the distinction may not be necessary (at least not for my system), but I was wondering...

Best regards,
Katharina
Pablo_Lopez_Rios
Posts: 53
Joined: Thu Jan 30, 2014 1:25 am

Re: varmin gives NaN

Post by Pablo_Lopez_Rios »

Hi Katharina,

This warning message is triggered by a check of the initial constraint matrix which imposes symmetries on the linear parameters in the e-e-n term. The check attempts to catch cases where the initial non-linear parameter values (i.e., cut-off lengths) may accidentally eliminate a parameter from the equations, potentially altering the code's perception of which parameters depend on which others. This is a potential issue in the correlation.data implementation too, only that no check is performed in that case. In practice this often causes no problems during the calculation (and in fact the gjastrow automatically does a second constraining round with shifted values of L to resolve parameter dependencies, which should avoid the problem altogether), but to be safe it is better to set the cut-off lengths to non-integer values, and make L for different channels slightly different (i.e., not related by simple fractions), as hinted in the warning message.

Does the rest of the calculation misbehave? Have you tried explicitly setting L to non-integers?

Best,
Pablo
Hey there! I am using CASINO.
Post Reply