Goodhart's law and VMC optimization.
Posted: Mon Dec 18, 2023 6:11 am
Hello QMC people.
As Goodhart's law says "When a measure becomes a target, it ceases to be a good measure".
You may think that this is just a adage, but in some situations it can lead to catastrophic consequences.
https://www.lesswrong.com/posts/fuSaKr6 ... tastrophic.
Let's say we want to optimize WFN to obtain the minimum DMC energy, as a measure of DMC energy, we choose VMC energy, which is easier to optimize and it seems that these two energies correlate with each other. At this point Goodhart's law comes into play.
How to determine whether Goodhart is catastrophic in our situation?
As stated in the article Journal of Statistical Physics, VoL 43, Nos. 5/6, 1986 "The Statistical Error of Green's Function Monte Carlo" D. M. Ceperley
optimization of VMC energy leads to a decrease of DMC variance, this may not be entirely true, but gives a hint that the VMC energy as measure is bad.
One possible check would be to look at the joint distribution of DMC energy and VMC energy as functions of the random WFN parameters sets
distributed around minimal VMC energy one. The higher the correlation coefficient, the better Goodhard.
Best Vladimir.
As Goodhart's law says "When a measure becomes a target, it ceases to be a good measure".
You may think that this is just a adage, but in some situations it can lead to catastrophic consequences.
https://www.lesswrong.com/posts/fuSaKr6 ... tastrophic.
Let's say we want to optimize WFN to obtain the minimum DMC energy, as a measure of DMC energy, we choose VMC energy, which is easier to optimize and it seems that these two energies correlate with each other. At this point Goodhart's law comes into play.
How to determine whether Goodhart is catastrophic in our situation?
As stated in the article Journal of Statistical Physics, VoL 43, Nos. 5/6, 1986 "The Statistical Error of Green's Function Monte Carlo" D. M. Ceperley
optimization of VMC energy leads to a decrease of DMC variance, this may not be entirely true, but gives a hint that the VMC energy as measure is bad.
One possible check would be to look at the joint distribution of DMC energy and VMC energy as functions of the random WFN parameters sets
distributed around minimal VMC energy one. The higher the correlation coefficient, the better Goodhard.
Best Vladimir.