Profile

Cover photo
Kiril Vatev
Works at Accusoft
Attended The Ohio State University
Lives in New York, NY
134 followers|500,662 views
AboutPostsPhotos

Stream

Kiril Vatev

Shared publicly  - 
 
Dear people who write software,

Stdout if for output from your program. Stderr is for writing error information. Please stop mixing them up.

Thanks,
Kiril
1
Add a comment...

Kiril Vatev

Shared publicly  - 
 
Should I port all of my existing Chrome Apps to Electron, or write a tool that ports Chrome Apps to Electron?

#decisions
1
Add a comment...

Kiril Vatev

Shared publicly  - 
 
I'm supposed to be spending most of my Saturday reading the source code of node-gyp, right?
1
Add a comment...

Kiril Vatev

Shared publicly  - 
 
Has everyone these days forgotten that async exists and it can solve all of your problems?
async - Async utilities for node and the browser
1
Add a comment...

Kiril Vatev

Question  - 
 
Is there such a thing as a maximum number of functions you can put in the event loop?

I have a highly asynchronous app, which results in a ton of async callbacks getting pushed into the event loop. One "operation" could result in several tens of async functions. Everything runs fine at low concurrency (like 5000 concurrent operations or so), but when I crank it up (20,000), it runs fine for a little while (less than a second), and then gets stuck. It queues up a bunch of callbacks, and they never get resolved. The app it just at a standstill. Is there any technical reason this would be happening?
3
Guillaume Gautreau's profile photoStephen Parente's profile photoKiril Vatev's profile photo
13 comments
 
+Stephen Parente That's just the thing, it's not using a ton of memory and CPU. Under normal operation (with about 5000 concurrency), it is using around 50-70MB memory and at 7-10% CPU. No big deal. When I increase the concurrency, it jumps up to those numbers for just a second, then goes down to 22MB memory and 0% CPU. It's seemingly not doing anything.

As I mentioned, this is a task runner. It's not queuing up any one single function over and over again (that would be silly). It's queuing many different functions during its operation. To repro, I have the actual task just use a process.nextTick(taskCallback), so the task itself does absolutely nothing. All the functions surrounding the task are things like timing functions, wrappers to make sure everything is asynchronous, and whatever the async library does to run tasks.

Editing memory limits did not help... which makes sense when you look at the actual memory usage.

In case it matters, the app does use multiple threads -- one as the master, and a configurable amount of worker threads. I can reproduce this with one worker thread easily. There is/was some IPC happening between the threads, which was what I originally suspected. However, I did two things to rule that out.

1) When looking at what actually is executing, I can see that attempted IPC messages do actually get through, and tasks get started in the worker thread. The callbacks of those tasks are never called though.

2) I rewrote the IPC bits to use a named pipe instead of an IPC channel -- one named pipe per thread, to be more exact. This had the exact same behavior of getting stuck at the same point in execution.
Add a comment...

Kiril Vatev

Shared publicly  - 
 
Why do people say "my module has 87 tests" like that is a large number? I am pretty sure I have written more tests than that just today.
1
Add a comment...

Kiril Vatev

Shared publicly  - 
 
I feel like I am working in the stone age when writing C.
1
Rob Rimbold's profile photoKiril Vatev's profile photo
2 comments
 
Yeah, but I usually have a guy that does all my stonework. His name is Haskell.
Add a comment...
Have them in circles
134 people
Dave Clark's profile photo
Michelle Brzoska's profile photo
Ivan Domkiv's profile photo
CodeKen's profile photo
Elizabeth Cusma's profile photo
Arun Goyal's profile photo
DubaiIlove's profile photo
PSD to HTML - SeoSemanticXhtml's profile photo
Penny Albritton's profile photo

Kiril Vatev

Shared publicly  - 
 
This is probably the only time I'll get to say this, but I wish this memory leak that I wrote was faster. I have been sitting here forever waiting for it to get to 8GB.
1
Add a comment...

Kiril Vatev

Shared publicly  - 
 
Google, what are you doing? I thought my days of hacking phone software were over.
1
Add a comment...

Kiril Vatev

Shared publicly  - 
 
Just in case someone who follows me has an answer. I'm running into a strange scheduling issue in Node.JS.
 
Is there such a thing as a maximum number of functions you can put in the event loop?

I have a highly asynchronous app, which results in a ton of async callbacks getting pushed into the event loop. One "operation" could result in several tens of async functions. Everything runs fine at low concurrency (like 5000 concurrent operations or so), but when I crank it up (20,000), it runs fine for a little while (less than a second), and then gets stuck. It queues up a bunch of callbacks, and they never get resolved. The app it just at a standstill. Is there any technical reason this would be happening?
13 comments on original post
1
Add a comment...

Kiril Vatev

Shared publicly  - 
 
You know those days when you really don't feel like working on your GitHub projects... so you sit down and write all of that code you've been avoiding all week?
1
Add a comment...

Kiril Vatev

Shared publicly  - 
 
So apparently, I am writing C++ now and compiling it to JavaScript. I am still not quite sure what the point is.
1
Add a comment...
People
Have them in circles
134 people
Dave Clark's profile photo
Michelle Brzoska's profile photo
Ivan Domkiv's profile photo
CodeKen's profile photo
Elizabeth Cusma's profile photo
Arun Goyal's profile photo
DubaiIlove's profile photo
PSD to HTML - SeoSemanticXhtml's profile photo
Penny Albritton's profile photo
Work
Occupation
Web Developer, UI/UX Designer, Geographer, Android Developer
Employment
  • Accusoft
    Software Engineer, 2013 - present
  • Freelance Web Developer
    2013 - 2013
  • The Ohio State University
    Graduate Research Assistant, 2011 - 2013
    Front-end and back-end programmer on the GeoGame project.
  • ESRI
    APL Intern, 2012 - 2012
    Rapid web-app prototyper.
  • The Ohio State University
    Office Assistant, 2009 - 2011
Places
Map of the places this user has livedMap of the places this user has livedMap of the places this user has lived
Currently
New York, NY
Previously
Sarasota, FL - Columbus, OH - Riverside, CA - Akron, OH - Dimitrovgrad, Bulgaria
Apps with Google+ Sign-in
  • ∞ Loop
  • Fast like a Fox
Story
Introduction
My favorite flavor of ice cream is Oreo Cookies and Cream.
Education
  • The Ohio State University
    Geography MA, 2011 - 2013
  • The Ohio State University
    Geography BS, 2008 - 2011