“Hypoglycemic headache” returns a ton of hits and although honestly I’d never heard of it until your post, apparently it’s not uncommon.
According to the first hit on google, the treatment is to A) see if you have diabetes and if so get on top of that, or B) keep your blood sugar up, which it sounds like just a small amount of chocolate helps you achieve?
Note: if you did want physician follow-up, since this isn’t actually diagnosed as a hypoglycemic headache, neurologist might be the place to start, since they do a ton of headache work and all the serious stuff has got to be ruled out (or in) first pertaining to your headaches. They might then suggest B (above), or testing, or another specialist follow-up.