Software engineering isn't much of a science -- we don't really have good data on what practices we preach actually matter and how much they matter (see, eg, http://carmandrew.com/home/2013/9/3/can-we-please-put-the-science-back-into-computer-science
) -- but this tech report out of MSR at least does a nice job of listing some of the questions software engineers would like answered. Here are some of my favorites:
“We are often told that code comments are important to assist others understand code, but is this really true?”
“Are there measurable differences in productivity between experienced developers with CS degrees and developers with unrelated degrees or no degrees?”
“Are daily Scrum status meetings effective?”
“Does peer programming improve code quality and throughput?”
“How does productivity vary between open work spaces and traditional offices?”
I'd add that I'd love to see studies on effectiveness of various testing/debugging techniques over time, such as how much code reviews matter (I'd guess a lot), how much time should be spent by the developer walking through his own code in the debugger (I'd guess a lot), how much unit tests help (I'd guess they hit diminishing returns fast, only a bit of time spent on writing them matters much), how much traditional QA helps (same), how much live rollouts like A/B testing of code helps (I'd guess a lot), and how much user testing helps (I'd guess a lot, that the most important errors are about an overall crappy UX, and that much more user testing could help a lot).
But this might all be very hard to do. It's likely variability is going to be very high in this data depending on the people, product, culture, and environment. But we can hope that software engineering might be able to be more of a science, can't we?