A data-driven evaluation of the current security state of Android devices

Android’s fast-paced development cycles and the large number of devices from different manufacturers do not allow for an easy comparison between different devices’ security and privacy postures. Manufacturers each adapt and update their respective firmware images. Furthermore, images published on OEM websites do not necessarily match those installed in the field. Relevant software aspects do not remain static after initial device release, but need to be measured on real devices that receive these updates. There are various potential sources for collecting such attributes, including webscraping, crowdsourcing, and dedicated device farms. However, raw data alone is not helpful in making meaningful decisions on device security and privacy. We make a website available to access collected data. Our implementation focuses on reproducible requests and supports filtering by OEMs, devices, device models, and attributes. To improve usability, we further propose a security score grounded on the list of attributes. Based on input from Android experts, including a focus group and eight individuals, we have created a method that derives attribute weights from the importance of attributes for mitigating threats on the Android platform. We derive weights for general use cases and suggest possible examples for more specialized weights for groups of confidentiality/privacy-sensitive users and integrity-sensitive users. Since there is no one-size-fits-all setting for Android devices, our website provides the possibility to adapt all parameters of the calculated security score to individual needs.