I don't think it's unfair. I am sure that Mike is a great guy IRL. He's just robotic and dry sounding. Like he's reading a script instead of putting inflection and emotion into it like Tom does.
In his defense, Mike probably, almost certainly, has less time behind the mic than Tom does, but then, that's kind of the point.
Without getting into the whys and wherefores, Leo dumped an experienced, well liked host, for someone that is almost unknown in those same circles with a fraction of the experience, and none of the fan base.
I don't see that as unfair, I see that as people stating their enjoyment level, or lack thereof, with the new show structure.