Why Are Tech Sites Reviews So Disconnected From User Experience on Windows Phone Devices?
There always seems to be two worlds out there when it comes to Windows Phone ratings and experiences. There is the"major tech site" experience and actual user experience.The tech site ratings always tends to be a solid point to a point and half lower (on a 1 to 10 scale) than the actual day to day user experience. (See above screen shot from the Verge) On the flip side other devices running other OS's the experience/rating seems to be about the same for reviewer and user. Why is that?
Above is the Verge's picks for best phones for each carrier and the shot above is for AT&T with the iPhone5 taking the "phone to buy" trophy home from AT&T and The Nokia Lumia 920 getting the "Budget Pick" trophy nod. The stickler about that is when you ask the end user of the phone (the people who I actually write reviews for) to rate the phones the iPhone 5 gets user rating of an 8.9 while the Nokia Lumia 920 gets a user rating of a 9.6. Now if my math is still even halfway decent 9.6 is larger than 8.9. Not only that but the Lumia 920 is a full
hundred dollars cheaper (sorry it's $200 dollars cheaper if you buy the 32Gb iPhone to match 32Gb Lumia 920) than the iPhone 5. So shouldn't that just make the Lumia 920 the phone to buy at AT&T on all fronts?
Not according to the Verge's rating system where the iPhone 5 got an 8.8 (very close to actual user experience) and the Lumia 920 got a 7.9 (which is a full 1.7 points away from actual user experience) again the why question pop up. While it's near impossible to figure out what each individual user is thinking the Verge did put up a handy little chart for us to follow. (iPhone on the left Lumia on the right)
So if you look at the chart there are some points that can be argued back and forth all day such as design, than there are some in my personal opinion is way off, (I am looking at you camera ratings) but I am not here to argue those points. Look at the last rating for ecosystem the Lumia 920 got a 5 out of 1o. Yikes! This means that every Windows Phone that the Verge reviews they all lose almost half a point simply because it's a Windows Phone. (FYI the Android Eco System only gets an 8.
Something smells fishy here)
But even if you would add that half point back in there would still be a gap of over a point difference between the tech site's review of the phone and the users review of the phone. Where did that point go?
My guess is that the bigger tech sites have hired seasoned, experienced writers that have been in the game for years and years and will get a Windows Phone in to review don't use it as their daily driver and never really get the simplicity that Windows Phone experience brings because they have so much experience with whatever OS they are currently using as their daily driver. (it took me three weeks straight of using Windows Phone before I finally "got it")
Mind you not that it's bad to have personal preference it just that if you want to review tech for a living you have to put your preferences aside as you are doing said reviews. I know that Android is a great OS however, it's just not personally for me. But as I am reviewing the Samsung Galaxy Note II that I have I will not be reviewing the OS for you I will be review the phone itself. You already know what OS you like and don't like I don't need to tell you that.
I don't really know what the solution is this issue... perhaps everyone should just add a point (or ten based on the rating system) on top of every major tech sites Windows Phone review. I can you one thing though if ever review a phone and am this far off from real world experience on it I would have to apologize to my readers. After that, who knows? Perhaps go out and buy the phone I was so far off on and make it my personal phone till I at least understood why everyone loved it and I did not.