Can you trust measurements alone to make audio purchase decisions or should you compare products making music? Video.

[Ar9Jim 6,164]

It seems like a simple question. In this video one of the measurement sites reveals the marketing games that are played in the "best measurement equals best audio performance" deception in the hobby. Now these 'noble' sites are becoming more common and have started discrediting each other. This video says "take them with a grain of salt." These sites will eventually discredit each other, while attempting to discredit others themselves for views.. 

"Designing an amplifier to measure the best and designing an amplifier to sound the best are not necessarily the same goal." Bob Carver

 

 

Edited November 21, 2023 by Ar9Jim

[Dadvw 2,807]

Numbers to me, are relative.  Its part of the buying process, but equipment looks and how it sounds/performs is more important to me than the numbers.  And, who can hear the difference in specs from one piece of gear to another except the Audiophile Priests.........😂

[wrf 3,842]

I for one, am weary of the constant droning about resistive loads, measurements, reactive loads, high voltages, etc.

 

Measurements play a role in verifying the proper operation of an amplifier.  Your ears are the final judge.  ignore either at your own peril.

[Ar9Jim 6,164]

 

[Ar9Jim 6,164]

I'll be the 1st to admit that the specs on the 275 were poorly quantified. The marketing at the time not getting this right was tragic, imho..

 

I had heard an opinion from someone I respect, that has been in the industry since digital audio began. This man is smart almost beyond belief. I can't tell you companies this man has designed for but you know them all. I heard a comment about one of the measurement sites being a fixer for Chinese audio companies. Lots of interesting reading on the topic to be found online. This was the 1st click on the list. I don't know if it's true but people can look deeper and decide for themselves. Measurement sites discrediting measurement sites. 

 https://www.superbestaudiofriends.org/index.php?threads/audio-science-review-review.9827/

Edited November 22, 2023 by Ar9Jim

[iamjohngalt 2,087]

Thanks for posting the very interesting video, @Ar9Jim.

He hits the mark when he says "SINAD alone doesn't really tell you much." 

What he never says but should have said is, many of the differences in the results measurements

between the so-called best hardware and so-called lesser hardware are not audible to any human ears

and making any purchasing decision based on those differences is foolish.

He does say that hardware with nearly identical measurements can sound completely different. 

That may be the most important point of the video and one that is rarely made to consumers.

Measurements should at the very least be supplemented by advising what is and isn't audible to human ears.

 

Do audio web sites need a warning label like on cigarettes and on pharmaceutical ads?

 

[oldwood 9]

That is a great video that a lot of people need to see. There are just too many things that affect how a piece of equipment will sound in your room to have just one measurement be the only consideration.

 

I think humans do not like complicated situations and want to simplify everything, and that is why something like SINAD is easy for people to jump on as a silver bullet and ignore everything else. The idea that manufactures would change their process to make the device measure better in spite of the fact it would result in a device that does not sound as good makes me think of the loudness wars that did more to destroy the sound of recordings in the pursuit of more sales than any other event I can think of.

 

I found the bit about a gaming PC being the noisiest source you can use interesting, since I feed a DAC from my computer. I would like to hear more about that topic, including what the better solution might be.

 

Thanks for the video.

Edited November 26, 2023 by oldwood
add more info

[Ar9Jim 6,164]

 https://www.stereophile.com/content/parasound-halo-jc-2-line-preamplifier-measurements

 

Above are the measurements of the Parasound JC2 (John Curl) by John Atkinson at Stereophile magazine. John has been testing audio gear for 40-50 years and ranks the APx555 measurements of the JC2 as excellent.

 

Below is the same Parasound JC2 review by ASR. The product does not measure state of the art and can not be recommended. The same type of reviews can be found regarding Carver, Curl, Pass, King and other pioneers.

 

https://www.audiosciencereview.com/forum/index.php?threads/parasound-jc2-preamplifier-review.14779/

 

Both reviews are using the AP555 for their opposite conclusions. ASR does a good service on calling BS on snake oil or salesman quantifying specs before my time at Bob Carver. I applaud ASR in many respects. Like Bob says "there are companies selling products today with no basis in science."  ASRs work is good in this regard.

 

The only issue I have is the premise that designers of audio equipment should priorities their attention on improving specs further beyond audibility, that often come with sacrifices in an audible aspect of the design. The premise that prioritizing measurements in a design, will provide the most musical sonic value for the customer is just wrong IMHO.. It's an easy premise to do away with, when people actually hear audio by the great designers, instead of reading about it.

 

You can buy great testing products highly recommended as state of the art measurements by ASR for not a lot of money. If you want state of the art sound quality, that is a different goal and it cost much more to build.

 

My concern is for customer value and the state of the industry. Those selling "products with no basis in science" as Bob says, on one side, and on the other side, measurement sites inferring that better numbers equate to better sound quality, without end, as if the designers first priority should be to qualify as 'state of the art' on a chart, over ultimate sound quality goals. The best designers are smarter than to go off on tangents, chasing numbers beyond their sonic priorities. 

 

Who is serving the customers best interest here? The companies selling products with no basis in science, or those designing products to test 'state of the art' as a sales tool priority above the real goal of ultimate sound quality?

 

We certainly have no issues with measurements. They are indispensable objective tools we use every day in quality control. Each RAM 285 has an inspection report included in the box.

 

Some asked if I have an issue with ASR. Not at all, for the most part they are good at calling BS on snake oil, and that is healthy for the hobby, IMHO.. The only issue I have is the premise that the designers who design for best measurements, are superior to the world famous, world class, American designers who prioritize audio reproduction in their definition of state of the art. In the 2 reviews of the same product linked above, I'll put my bet on John Atkinson. 

 

[Yamacarver 664]

It's almost like they were testing two different pieces of equipment lol

The channels don't even match.

I don't believe a magazine would ever give a bad review to someone they take advertising money from. That's why we have consumer reports.

I've tried to "not like" Amir but I just can't do it LOL

 

Maybe he should just put tube sound amplifiers in a separate category since 0 distortion is not the goal. Distortion just means the signal has changed, it doesn't indicate whether it's for better or for worse as that is objective, however I'm sure most would agree that second or third order harmonics sound better than the power supply.

I made an amplifier switch and did some AB testing of this Yamaha (200 wpc) against my TFM 22 (225 wpc) that has 250 times more distortion. The difference was too small for me or another listener to have a preference. I had it set up to where you could hear the same clip repeatedly through the same speakers and switch back and forth instantly.

[Ar9Jim 6,164]

• Member
• 
•  625
• 619 posts

 

Posted 20 hours ago

"It's almost like they were testing two different pieces of equipment lol"

 

 

Exactly!!!!   

My droning about measurements has nothing to do with measurements. We use measurements every day..  Computer simulations used in RAM 285 manufacturing engineering are all math based. We embrace tech and objective measurements as the indispensable tools that they are.

 

The conflict in the measurements and product recommendations are polar opposite between Stereophile and ASR while using the same model analyzer.

In this case, one of these sites has an issue measuring audio equipment, regardless of the device resolution.  What about the customer reading conflicting reviews while trying to spend their money as wisely as they can for long term entertainment, and trying to not be disappointed in the end?

 

Without customers the hobby is dead. Where is the customer focus is this madness between products with no basis in science, and the premise that forever improving measurements should be the goal of audio designers, as if the science is that simple. Not even the same model of analyzer can agree on the same product while discounting John Curls work on one side. All I have to say is, you need to listen to audio. It's audio. Your ears play a small part and have to be present, you can't read about audio or measurements with any clarity at all.

 

The Carver Amp Challenge is the only way we know to serve the customers best interest.. Read the 2 reviews linked earlier for the same product. From the customers point of view who are they supposed to believe? In the end the customer has to hear the product to be sure of their purchase. 

 

Edited December 7, 2023 by Ar9Jim

[Yamacarver 664]

Sorry for the sarcasm what I meant was they were testing different preamps which of course they probably were.

I wouldn't doubt amir's test equipment because not everything fails his test like it would if his equipment was bad.

To say it has to be good because so and so designed it is idol worship.

Also they have had three years to submit a properly functioning unit which they haven't.

And I will reiterate that I wouldn't trust a magazine reviewer who takes advertising money. I'm calling snake oil.

 

[iamjohngalt 2,087]

2 hours ago, Yamacarver said:

Sorry for the sarcasm what I meant was they were testing different preamps which of course they probably were.

I wouldn't doubt amir's test equipment because not everything fails his test like it would if his equipment was bad.

To say it has to be good because so and so designed it is idol worship.

Also they have had three years to submit a properly functioning unit which they haven't.

And I will reiterate that I wouldn't trust a magazine reviewer who takes advertising money. I'm calling snake oil.

 

I agree that any reviewer who receives advertising funding from the companies he

reviews has a conflict of interest.

From the point of view of the reviewer that isn't the only concern.

Any reviewer who gives a bad review on equipment will likely not be given another

chance to review products from that company. Other companies will also hesitate

to send their products as well, and that could be the end of the reviewer's career.

(How many bad reviews have you read from major audio reviewers?)

I know I wouldn't send my products for a 'science' review because Amir's testing

is actually counter-productive for the audio customer's purchase decision.

Making a judgement on any audio gear without listening to the gear is irrational

and foolish.  Amir is misleading anyone reading his testing conclusions.

Edited December 7, 2023 by iamjohngalt

[Yamacarver 664]

I think what the magazines do when they find a problem is they'll give them a chance to correct it and may or may not mention it in the review.

Here is a good (mostly) review on a vintage piece.

https://www.audiosciencereview.com/forum/index.php?threads/yamaha-a-1-vintage-amplifier-review.9576/

[Ar9Jim 6,164]

20 hours ago, Yamacarver said:

Sorry for the sarcasm what I meant was they were testing different preamps which of course they probably were.

I wouldn't doubt amir's test equipment because not everything fails his test like it would if his equipment was bad.

To say it has to be good because so and so designed it is idol worship.

Also they have had three years to submit a properly functioning unit which they haven't.

And I will reiterate that I wouldn't trust a magazine reviewer who takes advertising money. I'm calling snake oil.

 

No sarcasm noticed my friend. Just good conversation. Thank you!

I don't doubt Stereophile or ASRs equipment. That's the interesting part. I'm not so sure John Atkinson is snake oil. I'm not ready to discredit either guy. I'm just saying audio involves ears and to suggest the customer should make their purchases based on a narrow range of measured parameters is also misleading. Not the measurements themselves.

Edited December 8, 2023 by Ar9Jim

[3M_Audio 934]

I miss Julian Hirsch's reviews. He was never arrogant when reviewing products. I think he evidenced a humble and genuine concern that his impressions of a product and belief of the characteristics of the technology wouldn't necessarily match a readers', so he was always gentle with his words. He knew it ultimately came down to listening and personal taste.

[4krow 5,118]

I remember the first review that I read in Stereophile when it was still a small format Readers digest sort of looking thing. The review was for a set of Altec Lansing speakers that cost $13,000. IIRC, they were 3 way active speakers and very large. Back in that day, I wasn't aware of any company charging that amount for any given speaker, but I was still new to high end audio. J. Gorden Holt tore them a new hole with his review. I don't think that he appreciated any aspect of the speakers and I was taken back by the review in general. I get that you don't care for a design, but to nearly condemn the product from top to bottom almost seemed suspicious to me. 

 Of course I appreciate the honesty that can be given, but at least a bit of diplomacy alongside it would raise my opinion of the reviewer. Another important fact that I try to live by is to consider the source. The source in this case seemed to be a nasty old man who didn't care what other people thought. If you could read past that impression and still try to get some kind of feeling from the reviewer besides being a total ass, it still might have merit. Those like Julian Hirsch however ride the other side of the fence, and I get more from the review but not enough. One of the worst case scenarios were reviews from Consumer Reports, who really should have stuck to washing machine reviews.

 Lastly, I also do not put a huge amount of weight behind customer reviews because of a sort of expectation bias, lack of knowledge in some cases, and not knowing the total impact of a system (including room) for their listening environment.

 I completely agree with the in home listening trial if it is allowed by the seller. >>> I have always been a big fan of the Carver challenge. Let's test the listeners as well as the equipment! <<<

[Ar9Jim 6,164]

Carver Amp Challenge rules! Let the customer hear and feel what is the best value. You can't read about it. Eye's and ears don't work that way.😁

Hopeful more manufactures will feel the need to let the customers hear their designs with a trial period. We win.

 

Edited December 9, 2023 by Ar9Jim

[xavionics 1,420]

Reading about Bob’s initial introduction of his Sonic Holography system is what cemented his name in my mind.

Add to that the Carver Amp Challenges and I was further intrigued.

One day, a friend replaced his unimpressive receiver with a pair of PL components. 
I was quite impressed!

Eventually, I finally bought my c-1 and m1.0t. 
The hook was set.

Oh, and I stumbled upon this article this morning. Nelson Pass goes a long way into explaining feedback and it’s effects on the signal, measurable distortions and sound quality.

Pretty interesting read over coffee….

 

https://www.passlabs.com/technical_article/audio-distortion-and-feedback/

Edited December 10, 2023 by xavionics
Added Pass article info and link.

[radtraveller 221]

Sounding good at very high spl and being able to “adjust” the sound to my liking.. never cared much for what the “purists” say..

I like my music the way I want to hear it.

Speakers always the limiting factor for me with secondary being source material that starts sounding terrible at higher volumes..  I detest delta sigma dacs… too much added “brightness”. But they measure sooo good!!

Tons of power available with Carver and speaker protection circuits saved me more than a few times…

Only put amps without protection circuit on the 901s .. rated IIRC “unlimited power in non commercial use”.. 🙂. Eq’d and frequency limited to mids and highs, (multi-amp, xovers and never without the 901 eqs)  don’t blow or distort with good source material.
I have blown and/or heard egregious distortion from speakers costing 2-5x as much, usually the tweeters just get too hot and burn away.  Looking at you Martin Logan Motion 60xt..

in other words.. I only care about what I hear, that meets my desires,  at a price point I can afford for equipment that has proven record of lasting decades.  

[Ar9Jim 6,164]

 

Edited December 16, 2023 by Ar9Jim

[Ar9Jim 6,164]

Safest way to buy is to arrange for a trial period of an audio product in your system at home, without charging you a restocking fee if the product is not the best in your system. 

All new Bob Carver products come with a 30 day trial offer with no restocking fees. If a company won't waive their restocking fees for your home comparison with a Bob Carver product, you might ask them why? 

[Ar9Jim 6,164]

 

[Ar9Jim 6,164]

 

[Ar9Jim 6,164]