This summer, I’m interning at Spotify in New York City, where I’m working on content-based music recommendation using convolutional neural networks. I wrote a blog post to explain my approach and show some preliminary results.
56 plus ones
Shared publicly•View activity
View 5 previous comments
- Not at the moment. My colleague Aaron and I had some smaller networks trained on audio samples for the Million Song Dataset, but they were on a drive that was recently formatted accidentally... so they're gone. Those networks were probably a bit too small to use for transfer learning anyway though.Aug 5, 2014
- fantastic write up! So glad you shared your detailed analysis for audio processing. I also really enjoyed your NIPS paper and have referenced both the technique as well as those fun t-SNE plots to work colleagues many times in the past six months. Thanks again!Aug 5, 2014
- I just read you said Nesterov momentum, was there dropout in there? No, right?
Another thing, you said no convolution sliding in the frequency axis. Was that verified as bad in the experiments? Wouldn't it make the method invariant to the same music played in two different tones? Or invariant with the gender of the singer? ...Aug 5, 2014
- There's dropout in the dense layers :)
I used a 1D convolution mainly because it's faster. In some past experiments I have found that a little bit of frequency invariance is probably useful (although that wasn't in the context of convnets), but going from a 1D convolution to 2D is going to slow things down a lot, especially because of the way it is currently implemented.
It's something I'd like to try but I don't know if I'll get around to it before the end of my internship.Aug 5, 2014
- Interesting! In my experiments dropout didn't seem to go well with Nesterov momentum. Did you have success training dropout+RMSprop as well?Aug 5, 2014
- I tried adagrad/rmsprop/adadelta when I was working on the galaxy challenge, but I never got any of them to work better than SGD+Nesterov momentum with the occasional 10x learning rate decrease. That may be down to my impatience more than anything else though :)Aug 5, 2014