Wednesday, 25 June 2014

The importance of bug reporting


There's a longstanding debate in the iOS and OS X developer community over the value of filing bugs through the Apple Bug Reporter system, commonly known as radar. Some believe it's invaluable, the only way to give Apple the feedback they need to ensure bugs get fixed. Others believe it's valueless, a black hole of frustration from which little satisfaction, if any, ever escapes. I'm not a developer, but based on the conversations I've had, both points of view, and the experiences that informed them, are certainly valid. So, what good is filing radars and what should developer expectations really be?


If you're a developer, bug reporting is no different than any other aspect of any other relationship with Apple — it exists to serve Apple's best interests. Bugs hurt the experience of Apple's customers — which are also your customers — and it's in Apple's best interests to have you find and report as many bugs as possible so that the most critical ones can be fixed.


That last part is important to keep in mind. Apple's engineering load has scaled significantly over the last few years. There are now hundreds of millions of customers, almost 10 million registered developers, and over 1 million apps in the App Store. Apple is also releasing new versions of iOS and OS X every year. That results in a lot of bugs and a lot of filings. Add to that the Bug Reporter app on every iOS 8 beta device, and the bug screeners are being flooded at truly unprecedented levels.


Despite their size, Apple, like any company, is time and resource constrained. They have to prioritize. They have to fix the showstoppers that prevent software from shipping before fixing the glitches that may be maddening but are still manageable. They have to fix the bugs that can't be worked around before fixing the bugs that can. And they have to fix the bugs that affect a lot of people before fixing the bugs that affect relatively few.


That's where radar comes in. If someone at Apple wants to get a bug fixed, they pretty much need a radar to point to. If they want to get a bug fixed as a matter of priority, they pretty much need a lot of radars to point to.


That's why it's never safe to assume someone else has already filed a bug. Maybe they assumed the same thing. Maybe they simply didn't file it well. That's why filing duplicate radars (dupes) is also important. They're like up-votes. They indicate magnitude. A bug that nobody's filed is dark matter. A bug that only one person has filed is a tiny speck of light. A bug that's duped by dozens of people is a glow. By hundreds or more, neon.


Radars and dupes can also provide additional information. Even for known bugs, it's entirely possible an engineer at Apple hasn't yet figured out a fix. Seeing something in a radar or a dupe's description or sample project can help make everything fall into place.


What radars and dupes can't really do is start a conversation. Radar was never designed to be personable in any way. It doesn't thank anyone for their troubleshooting. It doesn't acknowledge the time and effort people put into filing bugs and providing sample projects. It doesn't give scores or points to tally. It certainly doesn't guarantee any particular bug will be addressed even months or years later, and if addressed, it doesn't guarantee anyone outside Apple will know about it.


Sometimes bugs get fixed in code for beta software. Sometimes bugs get fixed in code that supports unannounced hardware or software features. Sometimes bugs don't get fixed at all, simply because time is being spent fixing bugs that are considered more critical. Sometimes, many times, it really is a black hole.


That can be infuriating to a degree that makes some developers want to rage quit the reporting system. After having talked to a number of people, and repeatedly getting similar answers, however, I feel it's safe to say that, to the engineers and managers at Apple, it remains incredibly valuable.


They are human beings, after all. Some of them come from indie dev backgrounds and know just exactly what filing a radar feels like from the outside. Others know just exactly what filing hundreds if not thousands of radars feels like from the inside. All of them have lists of bugs they want to fix, and people who want them fixed yesterday. Getting anything added to those lists is tough. Getting anything pushed up those lists is tougher still. Without radars and dupes, however, it can be very nearly impossible.


So if you are a developer working on iOS 8 or OS X Yosemite apps and you're encountering bugs, please consider filing radars early and filing them often. Even if you never hear back about them, chances are there are people working on those operating systems right now, people who want to make great software and provide for great experiences — people who will deeply appreciate the radars you file, and your having their backs.



No comments:

Post a Comment