After the 21st April Google will be downgrading non mobile friendly website in mobile based searches.
Check your analytics and you may find that over 20% of your visitors are on mobiles, and maybe higher for local businesses.
So losing mobile ranking may have a big impact.
Google Webmaster Tools now has a report on mobile issues. It will tell you if you have problems that may affect your ranking as well as explain how to fix them.
So it's time to check if people can easily use your website on their mobiles.