Like many—if not most—iPhone owners, I recently upgraded to iOS 5. As part of the upgrade process, the phone asked me if I would let it send debugging information to Apple. I instinctively answered ‘no’. However, after a bit of fiddling I found that you can view the data that it wants to send, so I decided to do a bit more digging, and eventually changed my mind. Here’s why.
Before I get stuck in, if you’re not prepared to take Apple at its word, say ‘no’ and stop reading now. I am assuming, amongst other things, that the data available to view on the phone is the sum total of what is transferred to Apple. If you’re not comfortable with that assumption, then this isn’t for you (but this might be). In the end it’s a personal choice, and here I hope to give enough information for you to make an informed decision.
Firstly I’m going to look at Apple’s stated policies on data collection: what it will and won’t collect, how you can control what data is sent, and what happens to the data once it’s been collected. After that, I’ll be taking a closer look at the data itself.
Apple’s policy, and user settings
The starting point is Apple’s statement about diagnostics and privacy, which can be viewed in Settings -> General -> About -> Diagnostics & Usage, and tapping on About Diagnostics and Privacy at the bottom of the screen.
The text is only about two screens’ worth and is written in plain English. The salient points are:
- The user’s explicit consent is required to collect and send data.
- Data can include device spec, performance stats and app usage data.
- Personal data is anonymised or deleted before being sent.
- Location data is optionally sent (see below).
- App developers may receive anonymised or aggregated diagnostic data.
Location information can be removed from the diagnostics in Location Services -> System Services (you’ll have to scroll right to the bottom for the latter), and setting the ‘Diagnostics & Usage’ option as you wish. Interestingly, the wee icon is absent from the setting on my phone, meaning that location services haven’t been used for diagnostic purposes in the last 24 hours. Perhaps it’s not used very often, or only when certain types of failure occur.
The data
Right, let’s look at the interesting stuff: the data. Tapping ‘Diagnostic & Usage Data’ will display a long list of log files, all of which have timestamps. There are several types of file I’ve seen on my phone, so let’s look at each of them (there are undoubtedly other types of logs that could be generated, but these are the ones I’ve seen).
awdd
I have no idea what awdd is. I’m not an iOS developer and Google isn’t shedding any light either, so I’m going to assume it’s the Advanced Whisky Distillery Daemon. Sounds reasonable to me. These logs seem to get written between ten and over a hundred times a day.
deviceId: "ea320fc7cc8116123587436c0f7fd8b93e7c914c"
isAnonymous: true
deviceConfigId: 101
triggerTime: 1318523908584
triggerId: 589831
profileId: 10105
investigationId: 0
wifiMetricIPv4DHCPLatency {
dhcpLatencyMilliSecs: 3802
securityType: 4
timestamp: 1318523908584
}
Interestingly, the deviceId string is the same length as the UDID reported by iTunes, but it’s completely different. However, it is the same in every logfile, so presumably it is unique to my device. Maybe it’s hashed in some way.
The rest of this file looks fairly uninteresting. I suspect that the securityType relates to the type of wireless network I was using at the time (on the other hand, they could relate to the colour of whisky; who knows?). Every file I’ve looked at has the same value in this field. In any event, I don’t see anything here that concerns me.
log-aggregated
These files are in XML format and are generated once a day. They contain statistics on app usage that look like this (I’ve aggregated the entries for the Amazon UK app):
<key>appActivationCount.com.amazon.AmazonUK</key>
<integer>1</integer>
<key>appActiveTime.com.amazon.AmazonUK</key>
<integer>47</integer>
<key>appBackgroundActiveTime.com.amazon.AmazonUK</key>
<integer>0</integer>
<key>appLaunchCount.com.amazon.AmazonUK</key>
<integer>1</integer>
So I’d be telling Apple how often I launch apps and how long they’re open for. There’s also some more data about how many SMSs and iMessages were sent, and how many times the phone has woken up for various different reasons. Personally I don’t mind them knowing about any of that.
LowBatteryLog
Not much explanation required here. These logs are short, so I’ll reproduce the whole thing:
bug_type
120
description
Incident Identifier: BAD9E2E2-6F3E-4641-A82F-0BFE9ECEC0E2
CrashReporter Key: ea320fc7cc8116123587436c0f7fd8b93e7c914c
Date: 2011-10-16 11:00:31 +0100
OS Version: iPhone OS 5.0 (9A334)
SpringBoard: com.apple.springboard.enableidlesleep EnableIdleSleep == 255, held for 00:57:44
SpringBoard: com.apple.springboard.idle PreventUserIdleSystemSleep == 255, held for 00:34:54
SpringBoard: YouTube[218]-218[10587200] PreventUserIdleSystemSleep == 255, held for 00:01:50
mediaserverd: com.apple.audio.VAD Aggregate Device UID 1.isrunning PreventUserIdleSystemSleep == 255, held for 00:02:27
mediaserverd: com.apple.audio.MusicPlayer.isplaying PreventUserIdleSystemSleep == 255, held for 00:02:03
Hardware Model: N88AP
Awake Time: 05:36:59 (20218)
Standby Time: 13:17:23 (47842)
Partial Charge: 1
Capacity: 1
Voltage: 3612 mV
displayName
LowBatteryLog
name
LowBatteryLog
os_version
iPhone OS 5.0 (9A334)
version
102
It looks like the report is identified using a randomly generated UUID, and it gives the hardware and OS version of my phone. Also present is a summary of what was going on when the battery was low, and some very general usage statistics. I suppose this might be useful in cases where lots of people are reporting poor battery life, and particular apps or functions are suspected as culprits.
LowMemory
This report consists of a header, followed by a table of processes along with their corresponding number of pages resident in memory. Some of these are apps, others are internal iOS processes. I’ve removed most of the processes for the sake of brevity, but you get the general idea. It looks like I was using the web browser when my phone ran low on memory.
SysInfoCrashReporterKey
ea320fc7cc8116123587436c0f7fd8b93e7c914c
bug_type
198
description
Incident Identifier: 99F97C39-012C-4712-9FA5-758EAF3DFE91
CrashReporter Key: ea320fc7cc8116123587436c0f7fd8b93e7c914c
Hardware Model: iPhone2,1
OS Version: iPhone OS 5.0 (9A334)
Kernel Version: Darwin Kernel Version 11.0.0: Thu Sep 15 23:32:27 PDT 2011; root:xnu-1878.4.43~2/RELEASE_ARM_S5L8920X
Date: 2011-10-18 17:55:30 +0100
Time since snapshot: 44 ms
Free pages: 454
Wired pages: 13803
Purgeable pages: 0
Largest process: MobileSafari
Processes
Name UUID Count resident pages
WordPress <5c3cec175a9235339df3b3c693c531f0> 2640
networkd <3e79339746bd3c21a73c608972e983bd> 157
SCHelper
BTServer
eapolclient <4443e27cb1d430bea1ea213cbd936f61> 173
MobileSafari <569d7e26e8563409a69b64ad4874e96a> 11438 (active)
...
**End**
displayName
MobileSafari
name
MobileSafari
os_version
iPhone OS 5.0 (9A334)
system_ID
version
103
Interestingly the kernel version and build number is given at the top, along with another UUID. The processes can make interesting reading; for example, eapolclient is presumably the supplicant process used to talk to the RADIUS-backed wireless network I’d have been using at the time (one of the protocols used for authentication is called EAP). Perhaps BTServer is something to do with Bluetooth, although it would have been switched off at the time the report was generated.
App-specific logs
These are arguably the most interesting logs, and form the main reason why I decided to enable diagnostic reporting on my phone. They’re generated every time an app crashes or has some type of unexpected problem. According to this article by Apple,
If the application was distributed via the App Store and the user has chosen to submit crash logs to Apple, the crash log will be uploaded and the developer can download it via iTunes Connect.
Here’s part of an example that was generated when Safari fell over:
SysInfoCrashReporterKey
ea320fc7cc8116123587436c0f7fd8b93e7c914c
bug_type
109
description
Incident Identifier: 2EE27DF7-778D-4C5C-AFAE-2EE9A2CC9C1E
CrashReporter Key: ea320fc7cc8116123587436c0f7fd8b93e7c914c
Hardware Model: iPhone2,1
Process: MobileSafari [714]
Path: /Applications/MobileSafari.app/MobileSafari
Identifier: MobileSafari
Version: ??? (???)
Code Type: ARM (Native)
Parent Process: launchd [1]
Date/Time: 2011-10-18 17:59:22.960 +0100
OS Version: iPhone OS 5.0 (9A334)
Report Version: 104
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x000001bc
Crashed Thread: 9
There’s the ubiquitous UUID at the top, followed by some lines identifying the particular app that crashed; in this case, ‘MobileSafari’. The most interesting bit is at the bottom, where we see that it received the bane of every Unix programmer’s life: a segmentation fault (‘SIGSEGV’). Without going into too much detail, this means that Safari—specifically, thread 9 of the process—attempted to access unaddressable memory. The operating system sends a signal to the process, which is invariably fatal.
What follows is a goldmine for the developers:
Thread 9 name: Dispatch queue: com.apple.root.default-priority
Thread 9 Crashed:
0 libsystem_c.dylib 0x33aacae4 0x33aaa000 + 10980
1 CFNetwork 0x36136784 0x360d6000 + 395140
2 CFNetwork 0x36137b2e 0x360d6000 + 400174
3 CFNetwork 0x36137be6 0x360d6000 + 400358
4 CFNetwork 0x36137bf8 0x360d6000 + 400376
5 MobileSafari 0x00066d9c 0x1d000 + 302492
6 libdispatch.dylib 0x32388d4e 0x32388000 + 3406
7 libdispatch.dylib 0x3239479c 0x32388000 + 51100
8 libsystem_c.dylib 0x33ab41c8 0x33aaa000 + 41416
9 libsystem_c.dylib 0x33ab409c 0x33aaa000 + 41116
Thread 9 crashed with ARM Thread State:
r0: 0x000001bc r1: 0x00000001 r2: 0x00000000 r3: 0xffffffff
r4: 0x000001bc r5: 0x00005458 r6: 0x00000000 r7: 0x04661ee4
r8: 0x002abc80 r9: 0x00003060 r10: 0x000ecf14 r11: 0x00000000
ip: 0x3f3ecf28 sp: 0x04661eb8 lr: 0x3613678b pc: 0x33aacae4
cpsr: 0x80000030
This is a stack trace of the crashed thread, followed by a dump of the processor’s registers at the time of the crash (the other threads are also present in the file, but I’ve omitted them). There’s enough information here for the developers to see precisely what happened, where in their code it happened, and what the conditions were that led to the crash.
The next section of the file is a long list of images present in memory at the time of the crash, and looks like this (again edited for brevity):
Binary Images:
0x1d000 - 0xdbfff +MobileSafari armv7 <569d7e26e8563409a69b64ad4874e96a> /Applications/MobileSafari.app/MobileSafari
0x23b2000 - 0x23b2fff PhoneNumbers armv7 <77a7bb77fd1f361990fce9bb7cafabe1> /System/Library/PrivateFrameworks/DataDetectorsCore.framework/PlugIns/PhoneNumbers.plugin/PhoneNumbers
0x41a6000 - 0x41bdfff QuickTime Plugin armv7 <5eba7a5e9d673734bb5de3532703ca90> /System/Library/Internet Plug-Ins/QuickTime Plugin.webplugin/QuickTime Plugin
0x2fe1c000 - 0x2fe3dfff dyld armv7
0x30351000 - 0x30371fff libxslt.1.dylib armv7
...
There’s no suspiciously personal information here, just the kind of data a developer would want if they were debugging their app.
Summary, and a bit of reiteration
I can’t stress enough that the app crash data is invaluable to developers. I don’t know how much data Apple actually releases to developers, but they have the potential to know about every instance of their application crashing, anywhere in the world. More data = bugs fixed faster = happy users… and this is why I switched on diagnostic data collection on my iPhone. I want developers, including those at Apple, to have everything they need to fix unpleasant bugs in their code; when the bad App Store reviews start flooding in, they’ll have a parallel source of data with which they can smash the bugs and make everything wonderful again.
I hope this article has made you consider doing the same.
Nathanael Holt
Thanks for this. So calming after the squauking panic of people on Apple Discussions. I’m not a developer, but found solace in your cool breakdown of the Diagnostics page.
Melinda
Thanks for this detail, it’s been the only place I’ve found that seems to give such detail about these files. I have files that are being created in regards to Bluetooth connectivity at/about the same time I jump into my car and then when I turn off my car. There are around 4 to 8 files each time and I don’t know why it is creating these files. I’ve assumed that the car and phone are working together as my car shows the icon that they have paired. Should I be worried about these files? I don’t seem to get the countless number of the other types of files that others have seen. If these aren’t error reports why would anyone want the data?
deviceId: “….etc….”
isAnonymous: true
deviceConfigId: 101
triggerTime: 13208794614
triggerId: 720911
profileId: 10119
investigationId: 0
bluetootServiceConnectionAttempt {
timestamp: 13208794603
deviceOUI: “00\n0”
service: 16
}
flup
They are diagnostic information rather than error reports. It doesn’t look like there’s any identifying information in the report; in fact it seems to contain little of value to anyone. Perhaps they would be used in the event of a Bluetooth problem that occurred later on, so that someone analysing the logs can see the history of the connection. But whatever they’re for, I wouldn’t worry about them if everything else is working.
Chip
I enjoyed the breakdown of how and why you declined then decided to send your diagnostic data. I went through the same steps and chose to release the data without location info. I’m a beginner but I found the data fascinating and have learned enough about the characters and dialog in the data stream to follow some of the story it tells. Thank you for your thoughtful answer. Now I don’t feel so alone.
Lee
It has been a while since your post but I wonder – in light of the Snowden revelations and Apple’s part in it – do you still trust that Apple only collects the data that is available to view?
Have you ever tested what is *actually* collected (or are you aware of any efforts to do so)?
flup
A completely fair point. The data is encrypted on the wire so there’s no way of knowing what’s really sent. On the other hand, if the phone is sending more sensitive data for nefarious reasons, we can assume that it does so even with this option disabled.
Lee
Of course, covert surveillance implies it is likely irrelevant what setting one chooses… 😉
I wonder if you think there remains any reason to prefer a centralised, closed system such as Apple/Microsoft favour, over more open, decentralised (and open source!) systems as Linux/Android/FirefoxOS in a post-Snowden era. Assuming one cares about avoiding unwarranted dragnet surveillance.
flup
The only reason to prefer large, customer-focused ecosystems is that they are large and customer-focused 🙂 The main problem faced by the open-source community, and the main reason why Linux on the desktop will always be a niche market, is that open-source products are developer- and enthusiast-focused. That’s great, and the end results are technically excellent, but for non-technical people they are confusing and badly documented. The only people I know who use Linux on the desktop are techies and their relatives.
The ‘post-Snowden’ thing I find puzzling, because very little that’s been revealed surprised me. The most surprising thing is that people are surprised! In the early 90s I heard a GCHQ joke about where to fax something: ‘oh just send it anywhere, we’ll get it eventually’. Sadly, for any non-trivial use of the Internet (and the phone, and to a lesser extent the post), some of your data *will* be harvested, and every country it passes through will be having a good old nose. The only cure for that is legislative.
As for Android: how do you know what your ‘droid phone is sending and to whom (particularly when it’s doing it over the mobile network)?
Lee
I thank you for your replies. I should have mentioned that my question was limited to smartphone and tablet devices.
I agree that the Snowden leaks have few surprises in one sense for those who were well-informed about surveillance capabilities. It has however widened the debate to include many who were previously blissfully unaware.
I disagree however that the only cure for warrantless data harvesting is legislative. Full end-to-end encryption is the only cure I can think of.
With stock Android firmware, one can’t know who the phone/tablet is talking to anymore than with WP or iOS.
With custom, “built from source” firmware, running FDE (e.g. WhisperCore or Guardian), firewalls and PDroid/AppOps-type filters, one can have much better control I would imagine.
flup
The problem with end-to-end encryption is that, for it to be relied upon, you can only interact with a very limited number of endpoints. You also have to trust the encryption (a subject well beyond both the scope of this blog and my knowledge 🙂 )
Want to email a friend? Make sure they know how to use PGP/GnuPG, and they don’t have anything malicious on or connected to their computer. Non-technical friend? Out of luck. Even with GnuPG the metadata is in the clear. And, as I’m sure you know, in the UK your recipient can be compelled to reveal his private key and passwords.
Want to buy something online? Your data will get spaffed all over the place by the shop, card intermediaries and bank.
Want to search for something? Even DuckDuckGo wouldn’t be able to resist a court order to reveal its users’ data (or its private key).
Want to conceal your IP address? Tough. I strongly suspect that TOR has been comprehensively compromised (and there’s no way to know either way).
As for your Android solution — yes, you’re right in that a highly technically adept person can do all they can to stop unauthorised code running, provided that they can use open-source drivers for all the hardware, they know that the toolchain is trustworthy (non-trivial in itself), and they can trust the hardware and its firmware not to do anything sneaky (consider the recent discussions about Intel’s random-number generator). Even then, without the cooperation of your mobile provider it’s pretty much impossible to be certain as to what’s being transmitted and to whom.
In summary: I don’t believe there’s very much the private citizen can do without severely limiting their online activities.
San d
You didn’t think of the important work some May do on their idevice and that they can access and steal the rights to whatever you have made in your work. Such as music if your a musician, or videos or ideas you may have saved. Then what you’ve made a developer rich off of your idea.
Ian Chard
You make a good point, but sending debugging information to Apple wouldn’t reveal this work. If it leaks from your device without your consent, it’s probably some third-party software leaching data, in which case debugging data is the least of your worries. If you use a device for proprietary development like this, security should be uppermost in your mind.
Alejo
Hello, I have the bug_type 120 in my iphone, is it a hardware or software problem ??? Thanks you, and I’m so sorry for my bad english
Michelle
Nice…thank you
catherine sealey
StIll didnt answer my question
Teresa
Hi, At first, I was suspicious of sending my Diagnostic Data to Apple, so I didn’t send it for a while. Later, after learning some interesting things about how difficult it can be sometimes to fix the bugs, I decided to start sending my data too. I agree, Apple and Android developers are not out to gather your personal data on you. I also learned that when your Apps crash for one reason or another, the coders, or programmers, or whoever it is that names these specific types of “Crashes” sometimes come up with pretty funny names for them like, 0X8badf00d, which means, “Ate bad food.” Anyway, some of the crashes are a real pain in the rear, but, you can bet that they are working very hard to fix the bug and make everyone happy! While at it, they might name a few more bugs along the way. I just hope they are as creative as the one mentioned above.