We have always been told that DMC energy only depends on the node surface of the employed trial wavefunction in all electron calculations, except that there may be some timestep bais, population control bias. The following are DMC results using CASINO on oxygen atom at different time steps:
(1) results using unoptimized Jastrow factor, or a correlation.data file with only cutoff length supplied.
timestep Total energy error bar
0.0006 75.049837261566 +/ 0.000207336708
0.002 75.050101265011 +/ 0.000167252471
0.006 75.058132020692 +/ 0.000129294675
0.010 75.071699038171 +/ 0.000148781425
(2) results using an optimized Jastrow factor
timestep Total energy error bar
0.0006 75.051337320261 +/ 0.000123950090
0.002 75.051419838597 +/ 0.000124191531
0.006 75.053246246030 +/ 0.000076934348
0.010 75.055898135753 +/ 0.000066688757
We can indeed see that timestep error with unoptimized Jastrow factor is larger than that with optimized Jastrow factor. However, when time step is sufficiently small (0.0006) in both cases, there is still some noticeable difference between these two results:~0.0015 a.u. with error bar of 0.00024a.u. This may indicate that different calculations may produce total energy difference of ~1kcal/mol. I would not say this is a small difference, especially when DMC is to pursuit results with high accuracy.
In fact, I also doubled target_weight in DMC to check population control bias, but the result do not change.
I am wondering whether this is normal with DMC?
Fan
DMC energy

 Posts: 84
 Joined: Tue Jun 17, 2014 6:50 am
Re: DMC energy
Are these all electron calculations? (Looks like it, since it seems that the time step has to be very small)
For me it looks as if the time step error were still considerable without a Jastrow (not surprisingly since the timestep error will be smaller for a better trial wavefunction). I would not be surprised if the two curves would not meet for even smaller time steps.
If it is a pseudopotential calculation the locality error will have an influence too.
For me it looks as if the time step error were still considerable without a Jastrow (not surprisingly since the timestep error will be smaller for a better trial wavefunction). I would not be surprised if the two curves would not meet for even smaller time steps.
If it is a pseudopotential calculation the locality error will have an influence too.
Re: DMC energy
There are all electron calculations. You are right, when PP is used, the results will also depend on the Jastrow factor. This means one should always try best on optimization of Jastrow factor when PP is used. It also seems to me that the two calculations with unoptimizated and optimized Jastrow factor will not give the same results for smaller time step. However, time step error should be negligible when very small time step is employed. I do not know what are the other factors that affect DMC results besides node surface of the trial wavefunction, timestep error and population control bias.
The point is one also needs to optimize Jastrow factor carefully even in allelectron calculations if such difference always exists.
The point is one also needs to optimize Jastrow factor carefully even in allelectron calculations if such difference always exists.
Katharina Doblhoff wrote:Are these all electron calculations? (Looks like it, since it seems that the time step has to be very small)
For me it looks as if the time step error were still considerable without a Jastrow (not surprisingly since the timestep error will be smaller for a better trial wavefunction). I would not be surprised if the two curves would not meet for even smaller time steps.
If it is a pseudopotential calculation the locality error will have an influence too.

 Posts: 84
 Joined: Tue Jun 17, 2014 6:50 am
Re: DMC energy
Dear FanWang,
As I said in my last post, I think, however, that your data does not support the statement of differing DMC energies yet! Have you looked at the plot of the energies over timestep? You are most definetly not in the linear regime! There is no way you can reliably extrapolate to tau=0: any extrapolation error would be in the order of magnitude of the energy difference between the calculation with and without Jastrow function.
So in summary, your statement that the DMC energy with and without Jastrow differ is not supported by your data yet in my eyes. (Which does not mean that there is no difference, but if there is, I think that something is going wrong  such a difference should not exist, if I am not mistaken!)
no, if the DMC energy of an allelectron calculation depends on the Jastrow, then something is going seriously wrong and you should investigate what it is.The point is one also needs to optimize Jastrow factor carefully even in allelectron calculations if such difference always exists.
As I said in my last post, I think, however, that your data does not support the statement of differing DMC energies yet! Have you looked at the plot of the energies over timestep? You are most definetly not in the linear regime! There is no way you can reliably extrapolate to tau=0: any extrapolation error would be in the order of magnitude of the energy difference between the calculation with and without Jastrow function.
So in summary, your statement that the DMC energy with and without Jastrow differ is not supported by your data yet in my eyes. (Which does not mean that there is no difference, but if there is, I think that something is going wrong  such a difference should not exist, if I am not mistaken!)
Re: DMC energy
What do you mean by linear regime? Do you mean calculated total energy is linear with respect to the corresponding time step? According to my experience, you will not possibly see such regime if very small time steps are used. In addition, extrapolation gives additional error. Extrapolation is something we need when we can not afford DMC calculations with very small time step. For very small systems, such as a single oxygen atom, it is always possible to use very small time step. In my opinion, when time step is small enough and total energy does not change significantly with respect to the time step, the total energy should be reliable and timestep bias will be negligible. Such results should be more reliable than extrapolation. You can see from my results that total energies at time step of 0.002 and 0.0006 are almost the same. This indicates that the total energy at time step of 0.0006 should be reliable. But, total energy with optimized Jastrow factot is different from that with unoptimized Jastrow factor.
As in my first post, we are taught that DMC energy of all electron calculations only depends on node surface except for timestep bias, population control bias. Here I demonstrate an example that this seems not to be true. Can you point out what is possibly wrong in my calculations? I can also post the input files if you want to try yourself. Or the total energy does somehow depend on the Jastrow factor in PRACTICAL calculations, although this dependence is not very strong, 1kcal/mol in this case.
Fan
As in my first post, we are taught that DMC energy of all electron calculations only depends on node surface except for timestep bias, population control bias. Here I demonstrate an example that this seems not to be true. Can you point out what is possibly wrong in my calculations? I can also post the input files if you want to try yourself. Or the total energy does somehow depend on the Jastrow factor in PRACTICAL calculations, although this dependence is not very strong, 1kcal/mol in this case.
Fan
Katharina Doblhoff wrote:Dear FanWang,
no, if the DMC energy of an allelectron calculation depends on the Jastrow, then something is going seriously wrong and you should investigate what it is.
As I said in my last post, I think, however, that your data does not support the statement of differing DMC energies yet! Have you looked at the plot of the energies over timestep? You are most definetly not in the linear regime! There is no way you can reliably extrapolate to tau=0: any extrapolation error would be in the order of magnitude of the energy difference between the calculation with and without Jastrow function.
So in summary, your statement that the DMC energy with and without Jastrow differ is not supported by your data yet in my eyes. (Which does not mean that there is no difference, but if there is, I think that something is going wrong  such a difference should not exist, if I am not mistaken!)

 Posts: 237
 Joined: Thu May 30, 2013 11:03 pm
 Location: Florence
 Contact:
Re: DMC energy
Dear Fan,
The DMC will take a lot longer to equilibrate in the case with the crap Jastrow factor. Are you sure you're not including higherthanaverage energies from the unequilibrated phase of the random walk in your statistics (e.g. by setting dmc_equil_nstep = 1000 or whatever in both cases without actually verifying that this is sufficient)? Check this with the reblock / plot_reblock utilities..
If that isn't the problem, then why don't you attach the full set of input files so we can check (note the forum won't allow you to attach text files for security reasons, but it will if you gzip them first).
Best wishes,
Mike
The DMC will take a lot longer to equilibrate in the case with the crap Jastrow factor. Are you sure you're not including higherthanaverage energies from the unequilibrated phase of the random walk in your statistics (e.g. by setting dmc_equil_nstep = 1000 or whatever in both cases without actually verifying that this is sufficient)? Check this with the reblock / plot_reblock utilities..
If that isn't the problem, then why don't you attach the full set of input files so we can check (note the forum won't allow you to attach text files for security reasons, but it will if you gzip them first).
Best wishes,
Mike
Re: DMC energy
Thanks for the suggestion. I use dmc_equil_nstep=5000 in my calculations and just tried reblock to delete the first 105000 lines, the result does not change. I attached the input files with this post for you to check.
with best regards,
Fan
with best regards,
Fan
Mike Towler wrote:Dear Fan,
The DMC will take a lot longer to equilibrate in the case with the crap Jastrow factor. Are you sure you're not including higherthanaverage energies from the unequilibrated phase of the random walk in your statistics (e.g. by setting dmc_equil_nstep = 1000 or whatever in both cases without actually verifying that this is sufficient)? Check this with the reblock / plot_reblock utilities..
If that isn't the problem, then why don't you attach the full set of input files so we can check (note the forum won't allow you to attach text files for security reasons, but it will if you gzip them first).
Best wishes,
Mike
 Attachments

 o.ae.tar.gz
 (24.88 KiB) Downloaded 684 times

 Posts: 237
 Joined: Thu May 30, 2013 11:03 pm
 Location: Florence
 Contact:
Re: DMC energy
OK  nothing obviously wrong with the input.
Can you post the output of the 'graphdmc' utility for the two dmc.hist files.
Mike
Can you post the output of the 'graphdmc' utility for the two dmc.hist files.
Mike
Re: DMC energy
graphdmc for DMC results without and with Jastrow factor:
 Attachments

 DMC results with optimized Jastrow factor
 dmcopt.jpg (237.18 KiB) Viewed 15981 times

 DMC results without Jastrow factor
 dmcnoopt.jpg (221.14 KiB) Viewed 15981 times

 Posts: 95
 Joined: Fri May 31, 2013 10:42 am
 Location: Lancaster
 Contact:
Re: DMC energy
Dear Fan et al.,
A few quick comments:
* The allelectron DMC energies obtained with different Jastrow factors ought to agree so long as time step bias is controlled, populationcontrol bias is controlled and the equilibration period is long enough.
* The Bohr radius of oxygen is 1/8 a.u., suggesting that time steps should be << (1/8)^2/3=0.005. Perhaps use time steps of 0.002 and 0.0005 a.u.
* For a time step of 0.0005 a.u. the number of equilibration steps should be >> 1/(3 x 0.0005)=666. I would use at least 10,000 steps.
* I would vary the population in inverse proportion to the time step. This means that when you extrapolate to zero time step you simultaneously extrapolate to infinite population. E.g., I would use a target weight of 768 with the time step of 0.002 and a target weight of 3072 when you use a time step of 0.0005.
* For your Jastrow factor you can probably set the spindependence of f to 0 (rather than 2). Although this gives slightly less variational freedom, it should aid optimisation.
* If you suspect a problem with the Gaussian orbitals, you could check them by performing a VMC calculation without a Jastrow factor and without the cusp correction. The resulting kinetic energy should agree with the kinetic energy reported by the DFT code.
* You could also check whether there is a problem with the (cuspcorrected) orbitals, you could try using the numerical atomic orbitals.
* A quick evaluation of the DMC energy of CASINO/examples/atom/oxygen using time steps of 0.001 and 0.004 gives 75.0508(2) a.u.
Best wishes,
Neil.
A few quick comments:
* The allelectron DMC energies obtained with different Jastrow factors ought to agree so long as time step bias is controlled, populationcontrol bias is controlled and the equilibration period is long enough.
* The Bohr radius of oxygen is 1/8 a.u., suggesting that time steps should be << (1/8)^2/3=0.005. Perhaps use time steps of 0.002 and 0.0005 a.u.
* For a time step of 0.0005 a.u. the number of equilibration steps should be >> 1/(3 x 0.0005)=666. I would use at least 10,000 steps.
* I would vary the population in inverse proportion to the time step. This means that when you extrapolate to zero time step you simultaneously extrapolate to infinite population. E.g., I would use a target weight of 768 with the time step of 0.002 and a target weight of 3072 when you use a time step of 0.0005.
* For your Jastrow factor you can probably set the spindependence of f to 0 (rather than 2). Although this gives slightly less variational freedom, it should aid optimisation.
* If you suspect a problem with the Gaussian orbitals, you could check them by performing a VMC calculation without a Jastrow factor and without the cusp correction. The resulting kinetic energy should agree with the kinetic energy reported by the DFT code.
* You could also check whether there is a problem with the (cuspcorrected) orbitals, you could try using the numerical atomic orbitals.
* A quick evaluation of the DMC energy of CASINO/examples/atom/oxygen using time steps of 0.001 and 0.004 gives 75.0508(2) a.u.
Best wishes,
Neil.