I have two words for you: TAX FREE! No sales tax on yarn... but there is on accessories and errything else :P
Lion Brand Studio - It's really nice. Their premium yarns are decent
School Products Yarns (look for camel and silk lace -NOM! I think they may have yak too... too expensive for my budget :P)
La Casita (brooklyn) small, but friendly. In Cobble Hill/Carroll Gardens section of brooklyn. Lots of nice places to eat and gawk at NYers
Brooklyn General (bk) a bit out the eay, but they have a selection of yarns, fabrics, and other fiber crafty stuff
Argyle Yarn Shop - in Windsor Terrace Brooklyn. Near Park Slope, and yuppy madness....it's my fave shop in NYC.
Museum of Natural History
Union Square (14th street) Forbidden Planet is there, if you like comic books and the like. There's a Whole Foods, DSW, and other shopping in the vicinity. If it's a nice day, the steps are worth hanging out on for primo people watching. Also, there may be artists selling art, a green market... who knows....
Madison Square Park (there's a Shake Shack in the park!! And also a nice seafood restaurant across from the park)
St Marks Place: Lots of kitschy stores and head shops... but there is Stomp, Joe's Pub, and The Public Theatre not too far away
Brooklyn Bridge: The view is phenomenal on the bridge!
Chinatown: for cheap (and good) eating, as well as knockoff designer bags
34th st (Herald Square) The famous Macy's, but off on the sidestreets is the garment district of NYC. There are lots of bead and trimmings shops over there, as well as Little Korea (Korean BBQ!) and karaoke places
Let me know if you plan to go into Brooklyn; there are so many other places here... too lazy to list 'em :P