The short answer is yes, it should remove songs from your device if you used doubleTwist to sync these songs to your device and no other sync item references those tracks.
The slightly longer answer is it depends.
What version of doubleTwist are you using? Version 3 (XP/Vista) or Version 4 (Win 7/8)? And if version 4, have you updated to 126.96.36.19917 yet?
After you removed songs from iTunes, did doubleTwist detect the removal and update it's library accordingly?
If you used doubleTwist to transfer songs to your device but if after you unselect them in the sync settings, they aren't being removed, this may be caused by problems in the sync database, which keeps track of what's been transferred to your device.
Here is a step by step guide to syncing using doubleTwist 4.0 (available for Windows 7 or Windows 8 only) along with some troubleshoot tips.http://www.doubletwist.com/help/question/guide-to-syncing