Can anyone explain this performance anomaly between main thread and a web worker?
4 plus ones
Shared publicly•View activity
- Interesting issue... Could the difference be that you're using FileReaderSync in the web worker, but the asynchronous FileReader in the main thread? The synchronous version could be piling everything into memory with no chance for garbage collection. This would explain why smaller chunk sizes have no effect.
You mention that the slowdown happens in the parser, but resources getting hogged elsewhere could cause the cache issues that dlannan is talking about.Jul 12, 2014
- Wow! Changing from FileReaderSync to FileReader in the worker thread did it!
I used FileReaderSync because it's supposed to be faster than the async reader. But I still don't understand why there was "no chance for garbage collection" -- sure, the contents of each chunk of the file are being loaded and used, but then they are discarded when it goes out of scope, right? Does garbage collection in a worker thread need script downtime in order to do its job? (I did notice some memory usage increase, but not tons...)Jul 12, 2014
My second theory would be that web workers could be built to prioritize themselves based on how long they're alive. 30 seconds sounds like a good mark to allow for powerful bursts in the background to get a job done. Anything beyond that is either a massive job that could be threaded off again or else is (or at least should be) efficient code designed to be constantly running.
I don't know, but I am happy you brought this to my attention so I can keep this mind in the architecting phases. :)Jul 13, 2014
- Glad it worked! I was thinking more about memory not freeing up within the FileReaderSync itself. But yeah, I think most garbage collectors wait for the processor to idle before sweeping. Reading large files synchronously does not seem like a good practice.Jul 13, 2014
- Well, I've learned something new. Thanks for putting that out there. As web workers mature, I'm sure we'll see this more and more.Jul 14, 2014