Profile

Cover photo
Niraj Tolia
289 followers|8,877 views
AboutPostsPhotosVideos+1's

Stream

Niraj Tolia

Shared publicly  - 
 
This has been floating around the interwebs for a couple of days now and I think the Google folks on the Guava project are getting a lot of unnecessary flak.

Designing a very commonly used library like Guava takes a ton of effort to get right. For example, see how the JDK has screwed a number of things up (date/time anyone?). And then, there is the testing effort that they put things through. I believe this is probably also why no one has forked Guava yet.

Personally, I am very happy to be an extensive user of a hardened library that just seems to work. They have a very reasonable and well-thought through API that covers most common use cases. As a counter-example, see Twitter's common library. It serves a similar purpose as Guava but is broken in many API-design respects, has poor documentation, and is just downright buggy for some things.

This is not to say that the Guava folks shouldn't be more accepting and some of the reasons outlined here seem a little extreme but, then again, I can see where the legacy internal needs come from. On the flip side, they have accepted a patch from me :).
 
(If you don't use Guava, you can skip this...)

The story with #guava and your patches

Guava users,

Many of you, when you request a feature for Guava, have submitted a patch to us with the implementation (or even pasted code directly into bug reports).

And we have almost never accepted any of these patches, let alone even read them. And I know this makes us look all manner of self-absorbed, arrogant and unappreciative. That's what I'd think in your shoes. So it's time I tried to explain to you more fully why it's like this.

I realize that from your perspective, you're handing us a shiny new feature on a silver platter. It should be making our decision easy, since the work is already done. It's a gift of your time and effort and you've already solved the problem and all we need to do is just accept it! Looked at that way, we're either idiots or jerks for not being interested.

But here's the part that I don't think many of you understand: the work you've done to produce that patch is actually minuscule compared to the total amount of work we have to do to put it in Guava. I know that it feels to you like you've certainly gotten us more than halfway there, but trust me, it's only scratched the surface.

- We have to work out whether the problem it's trying to solve is truly the right problem
- We have to work out whether the solution presented is truly the best solution we can come up with
- We have to find evidence in the internal Google codebase that users will actually use the proposed feature if we create it. If we are adding methods to our libraries that don't get used, it hurts our case when we try to argue to management that we're doing important work and need more staff.
- We have to figure out how it relates to the piles of legacy code we have floating around our libraries (that you, lucky folks you are, don't even see!), and how we would deal with migrating those users if they exist.
- We have to decide the best name and location for the new API. This is hard! We spend a lot of time in our API review meetings just batting names around.
- We have to review the code deeply. Our code reviews are grueling and go on for many rounds. When you look at the code in Guava it tends to look "obvious", but we work very hard to achieve that quality. It's only obvious in hindsight.
- In almost every case we have to completely rewrite the javadoc that first gets submitted to us. And this is very hard. Writing good documentation is probably the biggest challenge we ever face.
- The tests that were first written are rarely sufficient; we're going to need to add more. When we do, some usually fail.
- If the change touches on any existing functionality, we have to submit it to Google's global submit queue and analyze test results from many thousands of projects to make sure we won't break any internal users with it.
- If the change goes in, we have to deal with the machinery that gets that change integrated out to you in Guava.
- We then become responsible for fixing any bugs with it that come up over time, and dealing with the related feature requests it will touch off.
- And the code never "stays finished' in general; we are constantly performing various maintenance tasks over our whole library (or even the whole codebase of Google), to make various cross-cutting improvements, and every bit of new code added increases that burden.

There's more I'm leaving out, but you get the idea. Guava did not get to the level of quality it has by accident, but by us being really obsessive about these things.

Now, when the patch comes from outside Google, we have additional mechanical overhead. One of us has to sponsor the patch as if it's their own, converting it into an internal patch that can merge correctly (which isn't always as trivial as it sounds), and sending it for review to another member of the team. And because we are the ones most familiar with our own style, conventions, practices and pitfalls to avoid, etc., sometimes just doing that plus "cleaning up" the code to get it ready for review is already more time-consuming than if we had written it ourselves from the start. That doesn't even mean that the code sent to us in the patch was bad. It can be very good by most standards but still need a lot of rework for our purposes.

Remember, if your feature is valuable, then we're going to want it in Guava whether you provided a patch or not. Providing the patch doesn't make it more likely that we'll decide it's a good fit for Guava -- if anything it just puts us more on guard against that seductive temptation to think "but it's already mostly done anyway, might as well!"

And here's the last thing. Be honest: if you were going to sign yourself up for doing all that work above... wouldn't you at least want to have the pleasure of writing the code for it yourself? I love writing code -- that's why I do this! -- but such a large majority of my time goes into activities like those described above. If my job were all about just applying other people's patches, I would inevitably start hating it after a while. Let me have some fun sometimes, okay? :-)

I really hope this helps to understand why your patches seem to go into a black hole. I know that no matter what I say it will probably continue to seem unappreciative and condescending, and I apologize. I do recognize that you are just trying to help. But, if you really want to help, then keep an eye out for the times when we will ask for help on a particular issue, because that's where your time and energy will really do the most good!

Rantingly yours,
KB
1
Add a comment...

Niraj Tolia

Shared publicly  - 
 
Diwaker Gupta originally shared:
 
Halloween decor at Maginatics

I must say I'm quite pleased to be one of the perpetrators (thanks to +Alex Loukissas!). Photo courtesy +Niraj Tolia.
1
Hyang-Ah Hana Kim's profile photo
 
Good to see Julio again. Hi!
Add a comment...

Niraj Tolia

Shared publicly  - 
 
This post could have easily been titled "What you should know to be successful at a startup."
Matthew Might originally shared:
 
What every computer science major should know:
http://matt.might.net/articles/what-cs-majors-should-know/
5
Add a comment...

Niraj Tolia

Shared publicly  - 
 
To quote +Alex Feinberg on Twitter,

"Every time I read a fawning node.js testimonial, I wonder whether entering the software engineering profession was the right choice."
1
mukesh agrawal's profile photoNiraj Tolia's profile photo
4 comments
 
Evidently, to disprove my points, it looks like people are putting node.js to good use - http://venturebeat.com/2011/08/16/linkedin-node/
Add a comment...

Niraj Tolia

Shared publicly  - 
 
While I really like some of the NoSQL stuff, I somewhat agree with this rant. For example, MongoDB didn't have the D (durability) feature till recently.
Daniel Abadi originally shared:
 
Gotta love the NoSQL guys (http://bradley-holt.com/2011/07/addressing-the-nosql-criticism/). They start off by saying "We make the full set of ACID guarantees". Then they start back-tracking. "We're only ACID compliant within a single data item --- transactions that span multiple data items are not ACID" ... "Oh yeah, and if you want the 'D' from ACID (durability) you to need to know enough to change a default parameter --- but we don't recommend it because things will go SLOW" ... "and even then, it's your responsibility to make sure your OS deals with fsync the way we want it to".

So, uh, then you don't really guarantee ACID, do you?
2
mukesh agrawal's profile photoNiraj Tolia's profile photo
5 comments
 
Thanks, Niraj! Will check those out.
Add a comment...
Have him in circles
289 people
Alex Vorbau's profile photo
Wojciech Golab's profile photo
Iulian Moraru's profile photo
Siddhartha Srinivasa's profile photo
Neha Chandra Singhal's profile photo

Niraj Tolia

Shared publicly  - 
 
A problem faced daily by most folks working on computer systems.
6
1
Add a comment...

Niraj Tolia

Shared publicly  - 
 
Diwaker Gupta originally shared:
 
Please reshare!!

Do you want to participate in an exciting public performance arts project? +Surabhi Saraf is looking for volunteers for FOLD {Live}: http://foldliveproject.net. If interested, fill out the participation form http://foldliveproject.net/participate-volunteer/
1
Add a comment...

Niraj Tolia

Shared publicly  - 
 
Couldn't agree more with this post. You always want to hire the smartest people you can find with a solid grasp of the fundamentals. For them, picking up a new framework, programming language, or yet-another-distributed-system will simply be a short matter of time.
Daniel Lemire originally shared:
 
2
Steven Wilson's profile photoXiaozhou Li's profile photoNiraj Tolia's profile photo
3 comments
 
+Steven Wilson Completely agree. The usual excuse for companies is "We are too large, we need some basic filter" where the filter equates to GPA, years-of-experience in some-very-specific-X, etc., etc. However, if Google and Facebook can make this work, so can everyone else.
Add a comment...

Niraj Tolia

Shared publicly  - 
 
This is a very cool interactive map by WSJ showing VC funding nationwide. If I am reading this correctly, in the first six months of 2011, ~$400 million was invested by VCs in Mountain View (we are based there).

(via Christopher Shepherd on Twitter)
1
Add a comment...

Niraj Tolia

Shared publicly  - 
 
Diwaker Gupta originally shared:
 
Applying to Maginatics is now easier than ever. Thanks to Apply with LinkedIn (developer.linkedin.com/plugins/apply), now live at http://www.maginatics.com/jobs.html!!
3
Add a comment...
People
Have him in circles
289 people
Alex Vorbau's profile photo
Wojciech Golab's profile photo
Iulian Moraru's profile photo
Siddhartha Srinivasa's profile photo
Neha Chandra Singhal's profile photo
Links
Contributor to
Basic Information
Gender
Male
Niraj Tolia's +1's are the things they like, agree with, or want to recommend.