12 plus ones
Shared publicly•View activity
- Suggestions for improvements?Jun 21, 2013
- Yes, it does the same processing (thus being a quite "dumb" test) ...
... and yeah, I would also suspect communicating larger batches of data would be better.
I wanted to test the line-by-line case though, since I think this is how I would use Go a lot of times, just write some naive Go code instead of doing it with python.Jun 21, 2013
- I updated the speedup numbers ... I was erroneously mixing the speedup with the cut in execution time.Jun 21, 2013
- Jun 21, 2013
- Thx, biogo looks interesting, yes!
(Though, I always find it nice to get a feel for how my own little algorithms will perform :) )Jun 22, 2013
- I have some experience working on the GVF format http://www.sequenceontology.org/resources/10Gen.html in python. I cant seem to find the code now but I remember parsing a 1GB of textual data was painfully slow., I think about 18-20 minutes on an AMD quad core 4GB Ram. Since I wrote the code for a cousin, I didnt go any further with that. Though you have already improved the performance by a lot just by using Go and channels but I think this needs to improved even more to work on larger data-sets. Very interesting. Lets give it a shot now. Good Job and Thank You!Aug 5, 2013
- : Thanks for the comment! In fact, I just updated (5 min ago) the numbers for this, since I now actually get even slightly better results with 3 threads, on my macbook air , which resulted in speedup of up to 80% (45% cut of execution time), as compared to the single threaded version that does copying of the lines.
Blog post is now updated with the new numbers and figures!
 The Go team suggests to use n-1 threads, where n is the number of cores ... so while the macbook has 4 "virtual" cores if accounting for hyperthreading, the results are in accordance with that.Aug 5, 2013
Add a comment...