I have been in the software field for decades. This is not really the result of open-source-is-too-under-resourced.
Throughout the 1990s, Microsoft execs/spokesmen loudly declared the commercial software giant loved to use very small teams to develop their products. While they raked in many billions of dollars per year, they spent not as much money or people on their major products' programming teams.
In 2001, their CEO was quoted in many articles as saying they would export their programming projects that make/update software products to the cheapest labor corners to continue to decrease their software funding expenditures. 0.0 And their products got hacked by criminals. Badly.
It's not just a circumstance of open source software by any means. Huge corporations are unwilling to shower large numbers of top-notch programmers with money. They do want large numbers of bodies on software teams. [Why? Two reasons; one, tons of software development is just renting out labor to a client at an hourly wage x markup rate, and managers are compensated based on how many people they are in charge of not how much money they are paying the people — assuming the managers are not working for free. Cough.]
Unlike open source teams which can most of the time move at the pace the work demands, big companies are wont to spend much calendar time not just labor time on big projects. [Unless someone else is picking up the tab.]
Of course, their moods swing this way and that. Microsoft happily announced around 2002-2003 it was not going to be updating/modernizing IE much anymore . And indeed it didn't.
Firefox came out. Safari came out. They soared ahead. Users switched their ride on the information highway. Suddenly, Microsoft was feeling by 2007 or so that it was not so cool to be the sole tortoise in a race against vivacious hares. By 2010, its IE team had gotten dramatically prolific.
So you really cannot even generalize about one company or even one particular software product of one company. Constants aren't [and variables don't, the saying goes]. Sigh.
Heartbeat is a security issue because that's where the impact is. But there are axioms across all software and each type of software, and a number of common aspects of most software.
Hiring the wrong QA staff can lead to QA problems. Those could be security problems, but in general that is just one concern among many; gotta tackle them all, not just the flavor of the month.http://www.washingtonpost.com/wp-dyn/content/article/2010/07/13/AR2010071302840.htmlhttp://en.wikipedia.org/wiki/Illegals_Program
If a crucial flaw in something exists that can lead to great harm, you don't want the already programmed fix for that languishing in your QA department for four to six months — or more — whether that happens by surreptitious intent or sluggish accidents.
That goes whether it is a security issue that someone can exploit, or can do great harm anyway without someone profiting from it and intentionally actualizing it into a manifest disaster.http://www.theregister.co.uk/2010/01/22/aurora_exploit_known_months/http://en.wikipedia.org/wiki/Therac-25
Even a one off, one time tactical decision can create a disaster, despite abundant good engineers/engineering/products.http://en.wikipedia.org/wiki/Challenger_explosion#Thiokol-NASA_conference_call
Sometimes the engineers get the flack instead of kudos they deserve.http://www.nasa.gov/centers/langley/news/researchernews/rn_Colloquium1012.html
Sometimes execs take flack for others' mistakes, and get little attention for their own brilliant coups which produce series of what appear as "very lucky" events but took brains, time and effort — even if they didn't build or design the thing everyone buys/uses.
I think you can't really say all open source product projects fly in one flock, and commercial closed source projects fly together in a separate flock.
It really comes down to the team of the moment, those calling the shots, the external pressures/drivers, how they are internally handled, and what resources are available to get put into it at the time that goes into it.
I don't know that in this case, what really disappoints people is that a mistake was made.
I think what brings a shudder and hopefully some thoughtful reflection to software engineers is that the Heartbleed flaw was not caught for two years after it was made, instead of "being caught" sometime before it came out in the initial+subsequent non-beta production releases. "Not caught" is a problem, something that can be fixed, and not a red-herring.
We kind of need to cultivate predators to hunt these overlooked technical faults ["faults" means technical flaws/impairments not social blame], instead of being gatherers that just collect a lot of the bad stuff but not being remotely sure how much there is or how much has been left unexamined.
Just having something that barked the problem was there — or hadn't been checked for yet, would have been a big help.
Look how many billion+ dollar companies and nations were relying on this software. If there was a list they were shown and told, "we haven't checked for these things yet against the latest changes" could they have put forth some eyes to look?
It's open source, after all — so outside volunteer help would have been allowed to do so too; formally open to look-sees.
To do that for a closed source, proprietary, commercial product would mean getting ensnarled [ironically] in NDAs [NON-disclosure agreements] and made it costly, bureaucratic, cumbersome, and complicated. Maybe a nation-state would give up and walk away.
If I can volunteer a little observation, the computer industry tends to typecast programmers.
Once they have a little experience and demonstrated aptitude in one area they get pigeonholed as "the guy" to do that.
But what goes to waste, and overlooked, is the guys who have the knowledge/aptitude to do a great job at that "specialty," but are left untried like a good but disused tool in the shed that people do not look around in very often, paying attention to only the few tools they are familiar with and ignoring the rest. [I'm not saying you only need raw talent and no knowledge to be productive in "specialized" areas but having the precursor knowledge and knowing how things look from examining/thinking about it gets you way closer than someone who has not tried or cared.]
Here's another SSL flaw that Apple copped to having in its products just a couple of months ago, back in February.https://www.grc.com/sn/sn-444.htmhttp://www.zdnet.com/major-apple-security-flaw-patch-issued-users-open-to-mitm-attacks-7000026624/
Oh, and not a lot of time separated that Apple SSL problem from an utter breakdown of Microsoft's Azure web service "cloud" caused by a third party
software's digital certificate expiring, bringing down Azure Storage which relied upon it for its implementation [so much for the "you don't have to worry how things are implemented in the cloud if you are an end user" marketing tag line], which in turn brought down other crucial Azure services that relied on Azure Storage.
Reminds one of the old Ben Franklin parable about how for want of a nail, a horse's shoe was lost, then it stumbled and broke its leg, and its rider fell and was lost too. And Ben Franklin never saw a computer or anything electronic in the precise "current" sense of the word today.
Pretty cool that Ben Franklin understood fault=>error=>failure chain of circumstances and could put that aspect of QA [or disaster analysis] in such clear terms.
That broad failure in Azure services brought down a lot of commercial end user services that relied directly on Azure, and perhaps some that relied indirectly upon it and didn't even know it. They were down for more than a day.
People were told to turn off certificate-checking in their web browsers as a "work around," thought that kind of creates other problems — like the reason we have digital certificates and private keys in the first place!
Starting to see a glimmer of a pattern emerging?
This boring, scary, dry protocol technology that supposedly only experts would want to work on — so supposedly only very few people really should/could read/check it — had glaring flaws that tearing into it with tests, a debugger session, instrumented code, unit tests, or maybe just reading through the source code carefully looking for that thing, probably would have caught.
Should this be only the province of malicious hackers and overworked, pigeonholed experts? I don't think so.
SSL is not boring this week, thanks to the widely used OpenSSL software. And SSL was not a dry topic in the press coverage of Apple a mere two months ago either.
Instead of pillorying open source, which basically boils down to a communal or individual charitable effort on behalf of society more often than not, we probably need to draw a bead on what we are missing with our current QA practices. We are leaving some things out, or not doing them enough.
Checking out big, complicated, behemoths of software packages/modules is a huge job. But a lot of good techniques out there are not used because so many decision-makers believe they need to find "the best" tool out there and use only it, or "the best" technique and mainly just that. In chemistry or medical arts, they don't just use one or two kinds of tests or instruments.
There are mountains of great tools, techniques, approaches, to seek out technical faults/errors at QA time in software. There are good techniques to enumerate what you have to, have, and haven't tested too.
Too much focus in recent years has been to only look for failures and only notice failures; gross, impossible-to-overlook, far-from-obscure, immediate, external consequences based on a spec; as opposed to subtle failures at that violate the fundamental principles. But it is the tiny, hard to see cracks that pests get into — as they are specialized to do so, and primed to thrive in that environment.
Things can go wrong internally or externally that don't get caught when comparing the user instruction manual to the actual behavior of the system. That's important. But that's not all that is important.
This is that sort of thing. Not leaking info is a fundamental rule of a security/privacy thing but it's not a step in the instruction manual, or a display in the end user interface. It's an intrinsic quality, not a readily visible blemish.
The problem is reductionism; reducing things to as simple as they can be regarded, and then going even further, getting the big picture actually pretty wrong.
Look around on the web right now. You can already see human intelligence swarming around the notion that you just need to check to see if the web server you connect to for the web page your portal web sites use — so all you need to do is find a magic bullet "tester" program out there that tests for the flawed version of OpenSSL vs. the good version of OpenSSL. And a notion [180 degrees wrong] that you should change your password right now [terrible idea, since that means you're giving away your password to anyone able to glimpse the data traffic as the broken SSL will be providing a lot less of a veil then after it is fixed; just don't login let alone change password till its fixed, to be safe].
Being afraid of SSL, or protocols, or open source is falling prey to the same principle Mark Twain warned about a century ago.http://thinkexist.com/quotation/the_cat-having_sat_upon_a_hot_stove_lid-will_not/262047.html
Rather than giving the OpenSSL team a hard time, it seems like a great time to give them some help, some tools, some fruits of efforts, etc.
Oh, and the closed source, commercial, proprietary companies doing SSL could use some help too. Even if their model limits how much of the help you can give the OpenSSL team can be given to these other guys. But at least they can learn from what OpenSSL gets and does.
Those are the real lessons from all this.
This is a messy problem, and its not just or maybe a mess in the terms of something you clean up. It's a mess in that you don't know how much dirt has spilled, where it's going to go, or if you're going to be swallowing whatever came about from the spill in the future.
Lay people aren't picking that up, and most of the press is not "getting" that implication either. There's probably not going to be a decisive, signature moment when we categorically know what did happen either. We know the 'fault'. But the 'errors' are out in the world, and we know not how many or how bad or what from. The failures that are most worrisome might come from there.