‘HD audio’ has been a buzzword for the last few years, as Apple and several record labels have been working on higher-resolution audio formats to repackage and resell older music. But the format took a body blow this weekend when former NY Times columnist David Pogue put musician Neil Young’s new $400 HD Audio PonoPlayer up against a regular old iPhone using a ‘blind trial,’ in which the HD PonoPlayer appears to have lost…
Describing himself as a former professional musician, Pogue said that “the Pono Player story is a modern retelling of ‘The Emperor’s New Clothes,’ ” making claims contrary to science. For the blind trial, Pogue assembled 15 people aged 17 to 55, asking them to flip between three songs on the iPhone and PonoPlayer, each song in the device’s best resolution. In separate tests using ‘standard Apple earbuds’ and Sony MDR-7506 headphones, more people preferred the iPhone to ‘Pono’ or ‘neither.’ Explaining the difference between his and Pono’s results, Pogue suggested that Pono’s claims of superior sound were effectively rigged, based on listening sessions conducted in a car using low-fidelity MP3s.
While neither of the headphones Pogue selected is a great choice for comparing high-definition audio (even the MDR-7506 is only a $100 model with mediocre reviews), the point is clear: many average listeners and even audio professionals cannot tell the difference between regular and ‘HD’ audio with regular headphones.
Noticeably absent from the video was any commentary from professional audio folks including the one guy that Pogue signed up earlier in the video. I would have enjoyed getting the perspective from a musician who wasn’t necessarily a big iPhone guy like Pogue.
Pogue’s advice, which I can get behind however, was to skip buying a new player and new music, instead spending $200 on nice headphones. Audio-Technica’s ATH-M50x is a sub-$200 model with stellar (4.7/5) Amazon reviews, and Sennheiser’s HD 598 is sold for under $150 with a 4.6/5 Amazon rating. If you want to go the Apple route and aren’t audio quality-obsessed, a black pair of new Beats Studio headphones can be had for under $220. Any one of these options will make your music sound a lot better than a pair of stock Apple earbuds.
Pogue’s review also indirectly calls into question Apple’s long-gestating plans to offer HD audio in iOS devices and iTunes. In early 2012, Apple began the process by rolling out a “Mastered for iTunes” section, featuring recently-optimized AAC files. Reports that the iTunes Store would start to sell HD audio files based on the new iTunes masters have percolated for some time, and Lightning-cabled headphones have been shown, though it’s unclear whether they will actually include HD audio support or just reduced audio interference.
FTC: We use income earning auto affiliate links. More.
While I agree with the conclusion (I have an audio engineering background and have thought the Pono to be snake oil since it was first announced) I must disagree with the veracity of the test, and your statement that the “result is clear” based on this particular test.
The headphones used should be unimpeachable, in order to test the player. What’s happened here, is that test subjects have been asked to pick which Kardashian sister they like best, from behind a frosted glass window which makes all of them look like the same basic dark-haired blob on legs.
A clear window, or in this case, a set of headphones actually capable of high-fidelity reproduction are required. This test, and everything associated with the Pono player, is just more bollocks.
Hook the the device up to an oscilloscope and let’s actually get some real frequency response figures. It’s not that hard to do and will rule out any perception issues. We can then say objectively whether one is all that much better than the other, and make note of how perceptible that difference actually is, at the conclusion of the test.
Mind you, even with all that said… squeezing 1% of extra detail out of a portable player is a joke in the first place. Traffic noise alone would make the point of carrying around a “banana-coloured Toblerone” (as Ars described it) moot in the first place.
A device like this would have been far better positioned if it had an SSD drive in it, and was made to replace your stereo, along with a decent amplifier. If it had a 6.5mm headphone socket, you could actually use a proper pair of bloody headphones with it too.
The one predictable thing in this comments thread, was that people would disagree, even if they ultimately agreed. My perception is better than your perception… Your objective measurements are inadequate, etc.
All that may be so, but not using ANY objective methods is silly. Relying on the perception of people with no training is silly. Using cheap headphones is perhaps the most silly.
re: “then anyone claiming a perceptible difference is obviously talking crap.”
There is a thing called Mean Opinion Score and many telecom’s spend billions of dollars to stay on the correct side of 50. It is well known that all people will give different scores to the same thing if they believe it is different. Call it whatever you want but it is real.
While I agree it would have been best to use better headphones, if it was purely a blind choice because of “frosted glass”, then their preferences would have likely been higher on “neither” or closer to a 50/50 split between iPhone and Pono. But, more people picked the iPhone, which is interesting. This is a small study, so we can’t put too much stock in it, but it does seem to indicate that the iPhone, even on mediocre headphones, sounded better to more people (especially since they could not see what device they were listening to).
I was with you up until the Oscilloscope suggestion. :-)
I agree about the headphones and your general assessment of the test, and I love the “Kardashians at the window” analogy. What’s at issue here though is sound, not sound waves.
What the oscilloscope would or wouldn’t show is irrelevant.
It’s not irrelevant from a technical standpoint. It’s valuable to know whether it’s ACTUALLY technically better, instead of just believing Neil Young that it is technically better, but dismissing it because of perception issues anyway.
There are two issues at play here. Perceptible difference, and actual difference. If you can objectively rule out any actual difference, then anyone claiming a perceptible difference is obviously talking crap.
@ Milorad: Sorry, I don’t buy it. When talking about sound, perception is all there is. It matters not a whit what the scientific instruments record about the wave fronts, that’s not “sound.”
“Sound” is something an individual experiences. Without the ear, there is no sound, (tree falling in a forest etc.), it’s all about what the person hears. It’s a subjective experience by definition. The oscilloscope would show what sound waves the device produces, but say nothing about the sound itself.
Better headphones, as you say are crucial to tests like this and the test is definitely inconclusive without them, but the sound waves are (mostly) irrelevant.
No, you missed the point of the story, which is that spending $400 on an “HD” music player (or replacing your entire music library with more expensive “HD” tracks) is a waste of money if you’re going to listen with mediocre or worse quality headphones. This is akin to choosing a smartphone based on a 4K display that’s impossible to appreciate with the naked eye. Pogue is providing a public service by preempting the fleecing of consumers by snake oil marketing tactics.
The subtext of this story is that Apple and spoiled recording artists like Young and Bono have a rude awakening in store for them if they think they’re going to revitalize music sales by focusing on the alleged shortcomings of digital music audio quality. They are ignoring the fact that jaded consumers have grown less inclined to pay top dollar for throwaway, commercial pop music and other content that is virtually free to reproduce. People are not going to return to the days of blowing $20 of music albums any more than they’re going to return to the days of paying $50 to own a movie.
The truth is the average person listening to a song can barely or can’t at all tell the difference in audio quality as long as both formats are good. The human ear is not a perfect machine. This is the same issue with TVs a few years ago running 720 vs 1080 from a distance over 10 feet the average consumer isn’t able to discern any differences.
Oh God, another AB test trying to disprove “audiophile” to “non-audiophile”. I have YET to find a comparison test that wasn’t without flaws.
I’ve even been around experienced recording engineers that couldn’t hear subtle differences because they were so used to bad recordings and lousy playback equipment that they just didn’t know how to tell the difference and I wouldn’t say they have actually proven their abilities at being able to produce a high quality recording in the first place. Some recording engineers I’ve run into actually do know what they are talking about since they just have more experience with higher grade recordings, equipment and they actually do know how to listen. So when someone tells you they are a recording engineer, that doesn’t mean they have good hearing. Just listen to the crap sound quality of the recordings on the market. There are probably more bad recordings on the market than there are good recordings.
The other aspect is that’s important is the quality of the recording in the first place, the listening volume levels, ambient noise, quality of the entire food chain of the system, etc. There are a LOT of variables that can interfere and have an impact on the AB test.
The biggest difference right off the bat is that the iPhone doesn’t support FLAC or 24 Bit files (at least at this time). So if you have some FLAC files you want to play, then the Pono will play those. If you have some 24 Bit files you want to play, the Pono will play those.
There are a lot of 24 Bit files that are converted from DSD (used in SACDs) and they are mastered differently as they aren’t using audio compression. But beware because there are some 24 Bit files that are on the market that aren’t PCM conversions from DSD, but just up sampled 16 Bit and they don’t sound any different, so please understand that some 24 Bit recordings are better than CD, MP3, etc. and some aren’t. So don’t make any generalizations about all 24 Bit, because I can show examples of where 24 Bit was better than 16 Bit.
Either way, there’s an app coming on the market by a company called Coppertino that’s called Vox for iPhone. On their website they show an iPhone 6/6+ with a track being played and at the top of the iPhone display it clearly reads “FLAC 96/24” which the iPhone normally DOES NOT support. So if you want a music player to play FLAC files, 24 Bit files, maybe this app for your iPhone will do the trick. :-) It’s not yet available, but it’s forthcoming according to their site.
Here’s the link to the site and sign up and get notified once they release the product.
http://coppertino.com/vox/iphone
Bottom line, don’t let someone else tell you what you can and cannot hear. Do your own comparison tests and the only thing you can say is what YOU heard and don’t draw any generalizations other than that because you are only one person and there actually is no definitive method for an AB test and we all hear things differently so it’s kind of impossible to draw any definitive conclusions. Too many variables that will impact the outcome.
So several things…first of all, these headphones are not capable of rendering the differences in the sources. Whoever is running tests using these transducers shouldn’t be running the tests. They don’t know what they’re doing. This is analogous to somebody making comments on jet engine architecture because they’re able to put air in the tires of their car. Secondly, most people do not have educated ears. By this I mean that when folks listen to music over mp3 players, YouTube video and TV, they aren’t discerning enough in their listening abilities. It’s not a knock but when you have zero experience hearing high quality source material how could you know? When you grow up on junk food how can you be expected to appreciate a Michelin 3-star meal? Most of it will go right over their heads. Spend a few years listening to high-quality source material over very high-resolution systems before offering ‘expertise.’ As far as the comment suggesting an oscilloscope…a scope will only offer so much insight. Reading a steady-state tone playback is a small part of the story, as is frequency response. (I don’t work for Pono and have no financial stake in high-bandwidth content delivery sites.)
The scope wasn’t intended to be a complete picture, which should be obvious to anyone who knows what an oscilloscope is. What it will do, however, is tell you with certainty whether one device has any particular dynamic range benefits over another.
There’s no point disparaging an objective test, especially since it can also be used to provide some rudimentary assessment of the DAC if attention is paid to the encoding method of the source.
Beyond that, there are so many variables that a ‘trained ear’ is indeed useful, but one (or two) of those, will still never be able to tell a consumer whether THEY will hear any difference or not, so even a trained ear will “only offer so much insight” if the question is “should I spend money on this or not?”
What if you could demonstrate, via an oscilloscope, the ability to faithfully reproduce frequencies far above the limits of human hearing? If the device being tested were meant for use by human beings don’t you think this would be pointless or downright misleading?
An oscilloscope…or any of the very limited insights it might offer (there are far more sophisticated measurement tools)…will not be the determiner of whether somebody should buy one or not. A prospective customer should listen with whatever headphones they plan to use and simply see if they can hear the difference. Many…or most…wont hear it. As I said above, they’re not trained to do so. Without devices offering requisite resolution and without the patience and training to actually listen, most folks will simply not get it. That’s fine. This is not for everybody. Spend time listening to real instruments playing music and avoid mp3 audio for a while. Use very high slew-rate amps and light/fast/big transducers. Anybody who really wants to will eventually hear what’s in there.
By and large, HD Audio is pointless. Further, unless the quality of the recording is the weakest link in your audio system, it is completely pointless. Having said that, let me point out a couple things.
Using older recordings for comparison is not the way to go. Even if they were recorded with superb equipment, that was still old technology and the tapes have been degrading ever since they were recorded. Take a song from the 80’s where the synthesizers might be 8-bit sound producers, moving that to 24 bit is not going to improve the original 8-bit original sound. Also, recordings from places like Motown have heavy-handed compression recorded along with the original so they cannot be separated and remastered with anything gained. Often the older recordings simply cannot be improved.
With newer better recordings, a lossless recording can be remarkably better. I have heard this using top-of-the-line Sennheiser headphones. It helps to know what to listen to since it is mainly the high end that is affected by compression. Listen to the audience applause and the reverb fading to nothing and if your hear a phasing effect you know the sound is affected in a very annoying way. Still, many will not care.
i don’t think you know what you are talking about. Analog recordings from the past are find. this has nothing to do with how something was mixed. that is true artistic right and approach. additional analog recordings have no define resolution as analog is based on voltage where digital is based on bits and SR.
Recording higher SR at high BD you reduce a lot of of the artifacting and aliasing.
I do know what I’m talking about. Badly done original recordings cannot be fixed, particularly when clean isolated tracks were never made to begin with. An analog recording of an 8 bit digital source — some synthesizers — is not going to benefit by improving the bitrate of the digital rip, lossless or not.
Neither can recordings would be good even in modern times. in fact people back in the day were arguably better musicians then they are today. Tape was pricy and inherently so were mistakes and bad performances. I’m still not convinced you have any idea you know what your talking about. a 8 bit sync recorded with analog media will sound analogy. a sound most synth players are chasing today because that same 8 bit synth may sound too digital with todays recording tech. your argument makes no sense because that same synth played badly yesteryear will still be played badly today. its not a argument that attracts the sonic differences in conversion and remaster vs modern recording and master. there will be sonic difference of course but to say it will not benefit from high SR / BR rerecording shows a lack of understanding.
Restored and remaster recordings sound great partly do to high SR / BR rerecording in the mastering process. this is usually done with high end AD converters on the front end. but if you choose to discuss a digital instrument in a analog argument i can see where a failed understanding of digital media and data compression started
Sony MDR-7506 do not have ” mediocre reviews “! these are industry standard for a reason. do a little research and you will discover that. They are probably the best selling headphones of all time!
That being said. if you understand digital audio and know the nyquist theorem then you will know anything about 88k is tough to hear any difference. certainly not saying that there is no merit in recording high SR. in fact there are a lot of reason to record higher the 48k. but to the average joe on the street…. probably not a lot of reasons since most people have trouble discerning what they are hearing in high frequency range.
This is complete nonsense. Testing an HD audio player with cheap headphones is like trying to measure 1/16″ increments using a ruler that only goes down to 1/2″.
People may have preferred the iphone because they are used to its eq curve.
His player is probably great with cans in a quiet room but most people don’t buy a portable player for that type of experience.
I have a musical background and I can 100% hear the difference between a CD and a streamed mp3, I can 100% hear the difference between wired audio vs bluetooth. But the thing is as long as I keep listening to crappy audio I don’t really feel like I’m missing out. I’m pretty sure younger people who have never owned a CD and only listen to streaming/mp3s have no idea what they are missing. I know what I’m missing but 99% of the time choose convenience over quality. Even when listening to Neil Young.
I’m certain the iphone can’t hold a candle to the Pono but it doesn’t matter because of convenience. David Pogue is either a moron or he designed a test to fail the Pono which is ridiculous because virtually no one cares about audio quality, if they did they wouldn’t be streaming music into earbuds non-stop.
Well they had people just like you who apparently failed to hear any difference.
I can agree here. Back in the 80’s I went to an Audio Store in London to audition a new turntable [a Rega]. The guy set up the listening experience using the same equipment I had at home [amplifier, cables, speakers, etc… everything but the turntable].
I brought along a couple of the vinyl albums I knew pretty well, sat back and listened to the Rega. I was blown away at how much more I could hear in the recording. Then the audio guy swapped out the Rega with a Linn Sondek [at the time, a monstrously expensive device for my budget].
End result: I bought the Linn Sondek. To my relatively experienced ears, a phenomenal sounding turntable, albeit at around three times the price of the Rega, the sound from it was revelatory.
It was so much better than the Rega, which in turn was so much better than the crappy turntable I had at home, despite the quality of my other components in the ‘food chain’.
Point of the story can be summarized as, ‘crap in, crap out’. If the original recording is bad, a bad listening experience; if the speakers/headphones are bad, ditto result; if the cables/interconnects, likewise.
And since Pogues player which is supposed to be HD audio quality failed to be any better than why bother adding HD audio to iTunes. If you can’t tell the difference why would I want to spend more money for the same music just because it says HD audio? Yet you can’t even hear any difference. It doesn’t make any sense at all.
I’m all for skipping the Pono and paying for some good headphones, but the advice in this article about headphones is unfortunate. The Sony MDR-7506 is described as “only a $100 model with mediocre reviews”. Read the reviews – they are anything but ‘mediocre’. (4.6/5) Instead, the author recommends the ATH-M50x for $166 (4.7/5), the HD 598 for $150 (4.6/5), or even the Beats Studio for $215 (3.9/5).
As an owner of MDR-7506 I can tell you they are amazing. They are The Wirecutter’s recommendation for best over-ear headphones less than $150. Personally, I cried the first time I listened to them as I heard things in my favorite music that I didn’t even know were part of the recording. They are studio monitors, so if you want headphones that add extra bass these are not for you, but if you like a flat sound profile, save your money and pick up some MDR-7506s for $85. You won’t wish you’d spent more money.
So, what I am getting is that if you are the type of person who will spend thousands of dollars on an acoustically designed room to put thousands of dollars of high quality audio equipment in, then HD audio might be the way to go.
However, if you are the other 99.9% of the population who MIGHT spend a couple hundred dollars on some nice headphones to wear as you jog or work in the yard HD audio is a waste of space (storage) and money.
Thanks, got it.
Actually, the Sony MDR-7506 are thewirecutter.com’s favorite $150 headphone, so saying it is “only a $100 model with mediocre reviews” I would say is not a true statement.
Using a splitter invalidates the test.
Don’t take Pogue’s word for it, then. A couple of other outfits have come to the same conclusion…mostly based on actual science. I believe Ars recently ran a piece on this as well.
Pogue is a shill for Apple
Give me any pair of shitty headphones and down sample it all…
I’ll still tell you the difference between a 256 Kbps MP3 and a 24-bit FLAC file all day long.
There difference is staggering. Lossless audio is worlds better and no you don’t need great headphones to hear the difference.
How about asking people that actually care about lossless audio, and who already go to great lengths to listen to it.
Average people have no taste in anything, or sense of quality. You’re not going to turn on new people to lossless audio. And you’re not going to find average people with the ability to tell the difference.
For all the reasons enumerated above, the headline and the reported test are mere reactions to change by the 100’s of millions of people already financially and emotionally connected to the iPod and it’s successors. The iPad and iPhone are fantastic devices that deliver music wherever you are,but it’s pretty bad sounding. I,like to listen on a Bose Sounddock or Bose color since they have limited range and definition and creset a punchy music experience.
On decent stereo speakers or my studio monitors the iTunes library still sounds punchy and dynamic but it’s obviously compressed and lacks realism.
Who cares? Probably not a hundred million people but more likely a niche market like the vinyl fans of which I am one.
Sound is funny. I have only heard a few recordings where the guitar sounds like it does live. I one case I tracked down the recording engineer and discovered he went to great length to create a pure recording path. If pure sound was the goal,there would be no distortion in the electric guitar. We like the distortion. It’s part of our musical experience.
Right now I’m trying to figure out how to get a Pono and then how to get some high definition music that matches it.
Thanks for the this article on the test of the Pono player.
What I often miss in these kind of tests is that the tester should consider what
inventors thought why the made a product exactly like it is.
Some people say the Pono player is there to sell high definition music, that has been sold already
in lower resolutions. Maybe, but mainly as far as I understood it,
it is there for just being able to listen to music as it comes from a studio recording.
Of course the difference will be for some very high, for others it won’t.
How is the sound processing in an iPhone set up like?
Really neutral? Or is it more like psychological sound processing that some/most/all radio stations do,
that we are used to so much, that we actually like the sound profile of the iphone,
because we just recognize it. And by the way, will Apple ever let us know?
Just some thoughts.
Actually, the Pono player has a balanced mode, separating left and right channel outputs across the two jacks.
And the audio switch is really a joke …
I’m an electronics engineer who has been passionate about audio for decades. I have built my own amplifiers (including tube) and a home recording studio. I don’t consider myself an audiophile but I am a stereophile. My house is overflowing with amplifiers, synthesisers, and miscellaneous audio equipment.
The reality is even low-end consumer-grade equipment these days is excellent. Even the humble iPod has audio quality that would shame a mid-range component system from the 80s or 90s. The revolution occurred when manufacturers replaced analog circuits – which are difficult to design and require expensive components – with digital audio processing. Critics of digital who long for the “warmth” and “smoothness” of analog simply don’t understand the mathematics or engineering behind the modern DSP, because a modern digital circuit is better in every conceivable way.
And modern compressed music is practically indistinguishable from original source. I completely appreciate Neil Young’s passion for having the best material possible, but for my money there’s no reason for 24-bit 192kHz in consumer-grade audio players. The vast majority of people simply can’t tell the difference. I can’t. Professionals do need 24-bit 192kHz for mastering and mixing, but that’s an engineering issue and doesn’t matter for end-users.
My advice is to spend your money on the best headphones you can afford. That’s the last analog component in the audio chain you still have complete control over. Spend $300 on a decent pair of over-ear headphones from AKG, Sennheiser, Grado, etc (not Beats!). You will keep those headphones for 10 years (minimum) and it will be the single best upgrade you can possibly make to your audio experience.
Dumb guy watches blue ray on CRT TV, asks, what’s the big deal?