I understand your point that you feel you can cope with high traffic when you expect it, but do note that Google has not phrased anything in the graphs in a way that overreaches its level of knowledge. It doesn't publish any unfounded assumptions about the level of service
at the venues it has this for— instead it opts towards the usage of very positive language, calling high traffic times "popular" times (referring only explicitly to your volume of attendance). Your patrons may make assumptions, but that would be your patrons—not Google—and presumably, if you are still highly responsive during high volume times, people who frequent your restaurant already know that.
Instead, since this feature neutrally labels high-volume times as "popular" times, it merely allows customers that may simply want the different ambiance
of a nearly empty restaurant to go at those times that fit that preference of theirs. Your traffic trends will change slightly and you can adjust staffing to suit the new norm. (No?)
To address the justified possibility that people may [mostly intelligently] assume that service might be slower at the peak times, maybe you would instead like to opt-in to different
graph of how long the service times are throughout the day?