I think youre getting good advice here (that is, lower it as your doc suggested).
I think 80g carb per hour is pretty intense for an amateur athlete / hobbyist. I am pretty passionate about cycling, but I dont follow a lot of the advice you see in the popular cycling media / sources. I think much of it is trickled down from pro level athletes programs, but not applicable for hobbyists. It might work for amateurs too, but these actions have consequences that probably do not balance out.
I don’t doubt there is some science study showing a statistically significant effect on more carbs relative to less carbs on some performance metrics, but it is simply not worth the while to get that benefit at the cost of your general health. However, if you were getting 6 figures + to ride your bike (pro level), then sure you could go do it for a decade or so.