The highest I've ever reached was 98. All that was left was low priority stuff.
+Andy Wigglesworth: A good way to think about page speed is to break down into steps: the browser requests the page, the server gets the connection and the request, which leads the server to have a think about what to send, which is then sent, and that gets rendered in the browser.
Depending on what your site needs in terms of computing power, hosting may not be the biggest bottleneck for you. The rendering part is a huge part of page slowness, and so focusing on that can get you serious wins quickly.
Pierre, I have done some checks using the PageSpeed Insights web tool. I score 91 for internal pages, but the homepage is lower, at 81 - I think because of all the images. Should I try to improve this, or just be happy that most of the site is reasonably quick?
The real thing is that it's perfect to do what you can to remove all detectable flaws and follow all rules. But what counts then, is the time until the content above the fold will be fully visible for the user.