|
Dirty-South Blues Harp forum: wail on! >
SPAH Comb Test Results Online
SPAH Comb Test Results Online
Page:
1
Brendan Power
83 posts
Aug 25, 2010
7:31 PM
|
The playing clips and relevant score card data are now online.
Go to http://www.brendan-power.com and click the SPAH link at top left.
Photos of Vern's Quick-Change Masking Bracket and PDFs of the Score Card and Instructions for Test Players are also there. Spectrograph images of the Test Tones will be added shortly.
Brendan
|
Greg Heumann
736 posts
Aug 25, 2010
9:56 PM
|
Thanks, Brendan
I'm lazy. Can you give us an overview of the results?
----------
/Greg
|
apskarp
312 posts
Aug 26, 2010
2:30 AM
|
I read it through. If you look for consistencies, which are interesting it test like this. The only thing I noticed was that "pear" was clearly differentiated from the others. Two persons liked it the best, one person the worst one. Other scores seemed to have the same variation as the "placebo changes" with brass combs. (If I understood it right.)
----------
Youtube
Hoodoo Sauna
Blog
|
htownfess
161 posts
Aug 26, 2010
2:38 AM
|
As I read it, the consenting trio indicated that it's hard to beat pearwood, for the majority taste--but hardly unanimous.
It seems like a test like this takes a lot of effort to put together, if you're an organizer.
I'm looking forward the EQ graphs.
|
Buddha
2388 posts
Aug 26, 2010
5:16 AM
|
"Very disappointing to read that some of the players deliberately sabotaged this procedure."
Nobody I know did that.
I didn't hear any difference between the materials. What I did notice is the inconsistency between the poorly made combs. Between them, it took more or less pressure to sound the notes. I felt obvious airleaks and could tell Vern was inconsistent in the way he screwed the plates to the comb.
I would like to see results from combs that are precisely flat and with plates that are properly set up. None of these combs were flat and the plates were not well set up if at all.
We were told to compare them to our own harps and if I honestly did that, I would have scored everything a zero.
This is in no way a reflection on Brendan or anybody else involved but I thought the test was very poorly executed and with subpar test objects.
I do have one issue though, to say that a tester, me in particular, did not honestly take the test is rubbish. I work with harmonicas and combs to a level that most never will. I didn't sit there for two hours just to **** around.
----------
"All is bliss"
Last Edited by on Aug 26, 2010 5:31 AM
|
Buddha
2390 posts
Aug 26, 2010
7:03 AM
|
@MVL
I know. i was simply responding to the remark.
----------
"All is bliss"
|
walterharp
438 posts
Aug 26, 2010
8:55 AM
|
i suggest that some people that consider themselves discriminating listeners listen to the recorded materials and score them. you cannot do it from the web page because the reed materials are listed above the recordings.... so some would need to have them sent to them in a different format and evaluate them.
|
barbequebob
1185 posts
Aug 26, 2010
10:00 AM
|
The test this time wasn't designed for the listener, but for the musicians playing them and often times what one hears when playing them is different than what someone in the audience hears, especially when they're listening at least 10 feet or more away. Most listeners usally can't tell the difference and most of the differences they're gonna hear have far more with the player's individual tone than anything from the instrument.
----------
Sincerely,
Barbeque Bob Maglinte
Boston, MA
http://www.barbequebob.com
CD available at http://www.cdbaby.com/cd/bbmaglinte
|
walterharp
440 posts
Aug 26, 2010
10:25 AM
|
right bob, but there are recorded sound files of all the players on all but one of the comb materials, so a very experienced listener such as yourself (or several different listeners) could rate them blindly... blind bbq bob... hey that has a real ring to it! :-)
|
LIP RIPPER
281 posts
Aug 26, 2010
10:52 AM
|
I had this crazy idea about a test for the next SPAH. I think it would be cool if the customizers that attend would all put together a harp, in the same key for a blind test and judging by say 3 rank Pro's. Of course Chris would have to leave off some of his ergonomic mods in order to keep players from being able to ID his.
|
isaacullah
1140 posts
Aug 26, 2010
10:55 AM
|
Interestingly, the coefficient of variation within the brass "placebo" of most of the players is quite high. In fact, it is so high in Players number five and three as to make it impossible to determine if any of the scores noted by those players were from "real" or "randomly imagined or perceived" differences between the combs. Player 6 had a lower variance on the brass placebo. The only preference that looks statistically significant at all to me is player 6's preference for pear wood.
Granted, I have not actually crunched the numbers, but I am reasonably competent at statistics and looking for significance in a bunch of numbers, since that's mainly what archaeology is (sorry to burst your "Indian Jones" bubbles!)... :)
----------
------------------
View my videos on YouTube!"
Last Edited by on Aug 26, 2010 10:13 PM
|
Littoral
63 posts
Aug 26, 2010
4:27 PM
|
Isaac, thanks for the stats view, that's what I was looking for. My take then is that I can do fine with pearwood. There's nothing in the numbers that say significantly otherwise. This test was a serious challenge to generate anything really valid but major props to Power and crew. I'll keep on the stumbling path to modifying my own harps. After I refine it enough to do my taxes and wash my clothes maybe Buddha will bless it with something better than zero.
(just kidding dude, don't sic the dogs or the creed on me)
|
apskarp
314 posts
Aug 27, 2010
12:10 AM
|
Well, Chris' comment is in my opinion also valid for the test. He said he didn't hear any differences between the combs. It supports the view that the comb material isn't as important as other factors like the flatness of the combs, the airtightness of the screws, etc.
Only thing that was seen from the numbers is that pearwood was differentiated for the better and for the worst. Does it tell something about the resonance of the material - or perhaps the other characteristics like how soft/hard material it is helping to seal the plates..
After seeing this test I tend to believe the material has only few important factors: Durability (does it last long, swelling), does it look good, is it easy to get flat/airtight with the plates.
----------
Youtube
Hoodoo Sauna
Blog
|
isaacullah
1142 posts
Aug 27, 2010
1:41 PM
|
I tend to agree with apskarp. I've never played a metal combed harp, but I have played several types of plastic combs (hollow and solidbody), three different wooden combed harps (pear, maple, and paka), and dymondwood. I personally like the dymondwood the best, but mainly because of the things apskarp says rather than any percieved tonal differences. To clarify, I think my dymondwood combed harps sound better than the same harps did before (ie, with their stock plastic or wood combs), but only because the harps are significantly more airtight, I can hold them more naturally/comfortably, and the tines are smoother and easier to move my tongue on.
----------
------------------
View my videos on YouTube!"
|
Brendan Power
84 posts
Aug 28, 2010
10:27 AM
|
I have uploaded spectrograph snapshot images of some of the short Test Tones that were recorded on the recent SPAH Comb Test. Description of the procedure and the nature of spectrography is included, plus a few quick observations at the end.
You can see the images by going to the SPAH Comb Test page on my website (click link at upper left). There is also a download link to the free spectrograph software I used (Wavesurfer), for your own further analysis. Enjoy,
Brendan Power
WEBSITE: http://www.brendan-power.com
YOUTUBE: http://www.youtube.com/BrendanPowerMusic
|
Brendan Power
86 posts
Aug 31, 2010
11:00 PM
|
(This is a copy of a message I posted on Harp-L, here for anyone who is not a member of that list).
SPAH Comb Test: Retraction and Apology
--------------------------------------
Thanks to Vern for his comprehensive round up of our recent Comb Test at SPAH.
It was interesting and enjoyable working together to make it happen. As he says, it was an exercise in (trans-Atlantic) co-operation and compromise. In the months leading up to the test there were lots of detailed, at times impassioned emails flying back and forth, arguing for this procedure or that. I wanted lots of combs, he only wanted two etc etc... (The compromise there was that we had ten tests per player but only seven comb types, so we each got what we wanted: the variety I desired and the duplication of tests Vern insisted on). Similarly we eventually reached agreement on other initially contentious issues.
We chose Marine Band reedplates and covers for every test, with just the combs changing. In addition to the stock MB pear and bamboo combs and a suitably sized ABS plastic comb, respected maker Chris Reynolds supplied us with the others to the same specs, ones commonly used in custom harps: brass, aluminium, corian, dymondwood.
Vern made a quick-change mechanism, which allowed for the combs to be switched quickly, and had a large weight attached to mask the differing comb weights (he has described the other ways in which comb differences were masked in his report). On the day, Vern's task was to do the comb switches, and mine to do the recordings.
We recruited six excellent players to test the harp/comb assemblies: Dave Barrett, Chris Michalek, Alex Paclin, Cara Cooke, Brandon Bailey and Jimi Lee. We assured them that their test scores would remain confidential, and this has been honoured. We also had the help of Winslow Yerxa and Michelle LeFree, who were invaluable on the day in making the test go smoothly by moving equipment around and making everyone comfortable.
The test went ahead; to our pleased surprise we got through it all in the two hours allotted. Afterwards both Vern and I got busy with SPAH activities, and it was only mid last week that I turned to the task of getting the results and music clips online. That led to an emotional process in which I've come full circle on my assessment of the test.
When I saw the score sheets, I was really surprised and very disappointed to see that three testers had given uniform scores for all combs - two of them from beginning to end and a third shortly after the start. After all our hard work preparing the test, seeing a "3" in every box for every category for every comb felt like a slap in the face, especially as the scores stayed the same from start to finish. On the face of it, that looked like a strategy that had been decided before the first test play. Unfortunately I reacted impulsively, posting my angry feelings in my initial online Test Report.
However, a week of reflection and private discussion has totally changed my opinion. I now think that the uniform scores were the most significant and useful data to come out of this test.
The main reason received for giving the uniform scores is that the test harp assembly was insufficiently airtight to give a clear distinction between tests. A reason given by one tester who marked uniform scores is that “…The only perceived difference seemed to be in airtightness and responsivness of the reeds as a result of differences in screw tolerances and comb flatness between each comb switch… the tests in the end were very inconclusive due to large variances other than the comb materials”.
Another has written that "I didn't hear any difference between the materials. What I did notice is the inconsistency between the poorly made combs. Between them, it took more or less pressure to sound the notes. I felt obvious airleaks and could tell Vern was inconsistent in the way he screwed the plates to the comb... I thought the test was very poorly executed and with subpar test objects."
That's a serious point, and deserves answering. As most of the testers are used to playing high-level custom harps, possibly they were comparing the test harp to the ones they normally play? However, it wasn't our goal to replicate a custom harp, just to get average out-of-the-box playability similar to a traditional 1896 Marine Band, while still allowing for quick changes. I tried the test harp for the first time the day before the test, and thought Vern had managed to achieve acceptable playability for test purposes with his quick-change assembly mechanism.
It sounded fine to me on test day when these six fine players blew it, and their pieces sound good on the recorded clips. They are all great players, and could make any harp sound good! However, on close listening you can hear this was not a high-level harp, and the leakage referred to can be detected in some places through the evidence of extra unintended notes sounding occasionally.
Obviously some of the testers felt that this lack of perfect airtightness was great enough to mask any comb effects there might (or might not) have been. Essentially they thought the test harp was inadequate for the task, as they have stated. Hence their uniform scores.
In our defence, we gave everyone the harp assembly to try in the half hour before the test. It might have been preferable for those who felt the assembly was inadequate to have simply withdrawn from the test at that stage, rather than sat through two hours of their and everyone else's time giving the same score for every test. We would have recorded their opinion, but we could have given their seat to someone else who was happy to note down their contrasting impressions of the different comb/harp assemblies, as the three remaining testers were.
But the three uniform testers decided to stay and sit it out, and I fully accept their scoring was sincere. After a week of reflection I've gone from being highly pissed off with them to thanking them, as I can now see how useful and valid their contribution was. It makes me realise we set our sights too low in accepting a stock nailed Marine Band as the benchmark for our test harp, and should have aimed far higher.
The whole aim of our test was to record player perceptions of the possible effects of differing comb materials. If the mating surfaces of the reedplates and comb were not perfect, whatever effects there could be would be minimised or masked entirely. This is what the uniform testers said, and I entirely agree with them. While I don't accept the assertion that the combs were not flat (they were checked against the light with a steel ruler), it's undeniable that the overall assembly was not perfectly airtight, as can be heard in the sound clips.
Though Vern may disagree, personally I have to admit that our test (while well-intentioned and the product of lots of hard work from both of us), missed the mark in its most critical element: the test harmonica. It did not provide the required level of perfection in the seal between comb and reedplates that was required to test the issue that we were aiming for. If a comb is not seating to the reedplates perfectly then any possible effect from the comb (if there) will be significantly reduced.
We have to thank the uniform scorers for being honest about that, and I sincerely apologise for initially disparaging their scores. Instead of testing an assembly at the level of a stock harp, to really look at this question it should be done with a test harp that adheres to the level of a top custom diatonic, with its superb level of airtightness between comb and reedplates. As one of the scorers said "I would like to see results from combs that are precisely flat and with plates that are properly set up."
I'd like to see that test happen too. It would take the close involvement of a top custom harp maker to create the assembly and check it constantly throughout the test, so there could be no quibbles from the test players.
Though I now feel our test was fatally flawed and therefore inconclusive, I belive Vern and I have contributed a lot in working out useful procedures that can be used in such a future test. They are recorded in the Instructions for Testers on the SPAH Test webpage.
I want to warmly thank everyone who took part in this test: Vern, Chris, Brandon, Cara, Alex, Jimi, Dave, Winslow and Michelle. Despite the difficulties and stresses, we had quite a few belly laughs in those two hours, and invented a new scientific measuring guage: The WINSPAN! Who knows, it could be our most lasting legacy...
Brendan Power
WEBSITE: http://www.brendan-power.com
YOUTUBE: http://www.youtube.com/BrendanPowerMusic
Last Edited by on Aug 31, 2010 11:06 PM
|
MrVerylongusername
1214 posts
Sep 01, 2010
2:04 AM
|
Thank you for the clarification Brendan.
@Brendan & Chris
I am conscious one of my above posts contains a quote from the, now retracted statement. I am happy to remove that if either of you feels uncomfortable with it remaining.
Regarding the experimental design, I'd like to throw this out there for discussion. Just the grain of an idea, but it might be an alternative way forward:
2 sets of plates, tapped for screw fitting, flat sanded to customiser standards.
6 combs, 2x wood, 2x brass, 2x plastic flat sanded to customiser standards.
doing away with the quick change rig, harmonicas are assembled as normal using a torque wrench to minimise setup variability.
The harps are then played in turn by one player (forget the embouchure variable for now) who is wearing high sensitivity binaural in-ear microphones (surprisingly inexpensive SP-BMC-1 mics being less than $50) which will produce recordings from his/her perspective. Multiple test tones are recorded multiple in a non-compressed format for each comb/plate combination under quiet studio conditions.
Hiding the comb material from the player is no longer necessary as the player is not rating the tones.
The comb is then swapped and the process repeated until all combinations of comb and plate have been recorded.
Sounds are then played back to test subjects using pro-quality, flat response headphones (borrowed from someone's studio???) and scores for each sound obtained.
Advantages:
More airtight harps, which can be set up to much higher level
potentially far more data can be obtained as, once the recordings are made, the appraisal stage can be repeated without limit.
no need to mask the playing surface of the harp
Eliminates the additional embouchure variable.
Disadvantages:
Not a perfect substitute for playing as "head sound" cannot be included.
Equipment costs (although you might be able to borrow some)
|
walterharp
443 posts
Sep 01, 2010
11:55 AM
|
I think the test was quite useful. You had some custom combs, and the harmonicas were put together by a person who is very mechanically adept (though perhaps not a full time or professional harmonica customizer).
Three of the testers obviously thought that other variables common to out of the box harps, or those put together with custom combs without the detailed work on the reed plates, and perhaps not as absolutely flat as the custom combs some use. Those that continued on could discern no significant difference among the combs.
This means, for the average person without very strong customizing skills, there is no reason with respect to sound quality to buy combs of different materials or what material a comb is made out of because so many other variables are important, including the ability to switch reed plates onto a custom comb. An exception to this is a player who does not want to seal their own combs but likes a specific brand otherwise might want to get a comb of a material that is impervious to moisture or buy a sealed comb.
It also suggests, but does not prove, that the comb material is not the most important consideration of the part of the harmonica tone that is independent of the player, even on customized harmonicas.
So material of the comb is more a matter of taste and how the harmonica looks than of playability, volume or tone. The ability to have a good seal between comb and reed plate is much more important than what the comb in made of, and most people do not have the time, training, or materials to create this perfect seal.
|
Post a Message
|
