mobileOSuserexperienceindex

iOS 7 has taken top billing in Pfeiffer Consulting’s annual Mobile OS User Experience Benchmarks, scoring just over 73 percent against 57 percent for Android and 47 percent for Windows Phone.

The study attempts to calculate an objective rating for the usability of a mobile OS by a typical, non-technical user by measuring four elements:

  • Cognitive load
  • Efficiency
  • Customization
  • User experience friction … 

The methodology is not without its controversy (see the discussion in the comments on Pfeiffer’s site), in that the criteria used represent Pfeiffer’s own rating of key usability elements rather than the results of user testing, and the Android version tested includes’s Samsung’s own Touchwiz interface – chosen because Samsung is the market leader in Android.

Individual category scores are shown below.

Screen Shot 2013-10-11 at 15.17.35

Screen Shot 2013-10-11 at 15.17.47

Screen Shot 2013-10-11 at 15.18.00

Screen Shot 2013-10-11 at 15.18.13

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

19 Responses to “iOS 7 tops 2013 Mobile OS User Experience Benchmarks”

  1. benbehr says:

    I don’t know what to think about this comparison. The available pdf doesn’t tell much.
    What makes me questioning the value of the comparison is for example a comment in the “Customisation” part where it is noted that it is not possible to change the background image in WinPhone 8. This is however not the full truth. The truth is that the system doesn’t have a background image as the entire screen is filed with tiles. So it’s not a feature missing, it is a different design approach. Subtracting points for this is IMHO not right.

    • Ben Lovejoy says:

      Yes, it’s quite subjective. I suppose the argument there is you can’t customise the overall look of the screen in the same way you can on an OS that uses wallpaper.

      • benbehr says:

        Yes, I guess you got a point. But than this should give them a bonus in therms of the “Cognitive Load”. Its one less thing that can be changed! Though, personally speaking the absence of (or relocation) of expected features can cause the highest “Cognitive Load” as I keep on searching for something that I am convinced the device should be able to do. (No example comes to mind just now but I know I had this with both, iOS and Sony Android)

  2. Honestly, what can you ‘customize’ in iOS besides the wallpaper and pre-installed ringtones?

    • rettun1 says:

      Notification center layout, screen layout, and most importantly the privacy controls (saying which apps have access to what information)

    • A whole lot, meet the Settings app. Even unique things, like whether you only want to receive repeated calls, whether you want parallax, which accounts people can reach you at using iMessage and FaceTime, vibration or LED patterns as ringtones, privacy settings like no other OS, managing per-app iCloud data, cellular usage, multitasking capabilities…

      I’m sorry to hear you have some beta version of the original iPhone, but its quite a lot more advanced these days.

      • You couldn’t change wallpapers on the original iPhone ;)

        And claiming that iOS’s customization strengths are solely from within options in the Settings app is very underwhelming. The settings options on iOS are a mess, when some apps have settings accessed from within that app and others require going to the Settings app itself. Confusing.

  3. Interesting to note that of all the Operating Systems studied, only one of the companies responsible happens to be a MAJOR client of Pfeiffer consulting, the publisher of the report. Guess which one?…

  4. Darwin Evolved says:

    1- Samsung is not the Android interface or representative of what Google ships.
    2- iOS7 is so awful/horrible/tedious/fugly that I traded my iPhone 5 for a Nexus 4 and I am an Apple shareholder.

    The Hello Kitty UI of iOS 7 may impress Buffy and the hipster crowd, may get a free pass from the Apple fanbois, but looks like a concept proposal by an 8th grade wannabe with cheezy graphics and a blizzard of white.

    • Buy few share and yes you will become a shareholder so it is worth nothing to mention.

      For you iOS is awful blah blah blah but that is just your opinion.

      About Android, yes Samsung is not the android interface or whatever it is but Samsung is the only manufacture that contribute the highest usage of Android. So it could be considered as Android in comparison to iOS.

    • Apple fanboy here and I completely agree with you. It’s still mostly an eyesore for me. I HATE looking at photos with it – all the white around the image is terrible. Same with how the music player app looks – too much white for my album covers! I hate many of the icons designs and colors. I hate how many buttons are too simplified for me to think that it was done with a lot of thought into making something simple an elegant. No, it looks simple like they went with something basic and didn’t think any further.

      I have always gotten white iPhones because when I got my 3GS I didn’t see many white phones out there, so I wanted to be different. And white on white makes sense.. I don’t like how that translates on a black (or Space Gray) iPhone.

      I wish I could change the default system colors, I wish the home screen could rotate in portrait mode, I wish I could reply to texts within notification center, I wish the podcast app wouldn’t keep bringing me back to the main list of podcasts instead of taking me to the player controls, I wish it didn’t continually ask me if it should trust my computer when I’ve said yes dozens of times, I wish Siri would respond more often and more quickly, I wish I could copy+paste calendar events, I wish I could have the “block caller” button in my address book app, not just the address book inside the phone app (or just get rid of the address book all-together, why have two instances of the same thing?), I wish I understood how weather worked – I’ll have a page for my current location and a page where I’ve typed in my current zip code and I get two sometimes wildly different weather listings, I wish the return to home screen animation was faster, I wish Passbook actually updated discounts but it only did one the first time and never again, and while the phone “just works” I still wish Apple had official instructions because there’s just so much users have to learn via YouTube/Reddit/etc.

      • benbehr says:

        “I HATE looking at photos with it – all the white around the image is terrible. ”
        I think at least this one I can partly solve for you. When you are looking at a single image click on the white. For me it turns black and I can truly enjoy the images.

        This however only works when you only look at single images. If you are in any overview or even want to write a comment or so you are back to white.

  5. I cannot believe these graphics.

    What is better in iOS7?
    - ?

    What’s worse in iOS7?
    - Bugs, bugs, bugs … and … some more bugs.
    
- Control Center is much overloaded, so you switch things on and off unintentionally.
    
- Control Center has very low contrast. Depending on the background it is sometimes impossible to recognize a symbol.
    
- Control Center is nearly not accessible if you have your device in a casing with a frame around the screen.
    
- You often open Control Center accidentally while scrolling upwards.
    
- Many Icons have strange colors which are either too blurry or too flashy.
    
- Many Icons have a strange design which has nothing to do with its function, e.g. Game-Center, Reminders, Photos, etc.

    - The Flat icon design looks old fashioned.
    
- Transparency or translucency kills contrast.

    - 3D- or moving-effects have no benefit, but they can cause dizziness.
    
- Folders can only hold 9 icons in one screen (16 in iOS6 on the iPhone, 20 on the iPad). So, if you do not need more than 16 (20) icons in a group – and that is most common – it’s easier with iOS6.

    - Missing buttons make it difficult to find the functions. Often you do not know if something hides a function or if it’s just a name, title or description.

    - Several functions are only usable if you KNOW exactly where to find them. There is no chance for an unfamiliar user.
    
- Calculator has no memory buttons in normal mode.

    - Calculator buttons are not enough separated.
    
- The Notes-app has a very dazzling background color and is not as eye-friendly as the “old paper”-like look of iOS6.
    
- The Calendar is confusing and not well-arranged.

    - Fonts are much too thin and therefore often difficult to read.
    
- The new “whole-screen-unlock-function” bears the risk of unlocking the phone unintentionally in ones pocket.

    - You have to swipe down to find the Spotlight-search. Often you open an App or scroll down a list or open the Notification Center unintentionally instead of opening the Spotlight-search.

    - The Spotlight-search is less powerful than before. In iOS6 you can search the Web.
    
- The preview in the task list is mostly useless. If you only want to change an App, and that is what you do most with the task list – you have less Apps in view and more to scroll between.

    - Media-controls are often more difficult to access.
    
- Figuring out the “New-App” indicator is much harder.
    
- It’s more difficult to switch to contact details from a text message.

    … and some more :(