As far as I know the saying has two origins.
1. It doesn't matter from which side you start eating a sausage, it's the same either way.
2. Meat that can't be used for anything else is used for making sausages. So if you've got stuff left you don't care what's happening with, then "it's sausage to you"...
something along these lines
trust me, I'm an expert