Phil's Superpower of Enthusiasm

When I decide I want to review something, here’s my methodology:

The actual methodology:

1. Pick the product, and avoid reviews, videos, and data about the headphones as much as possible.¹
2. Acquisition: figure out how to acquire the product and actually get it in my hands.²
3. Brain burn-in: listen to the product for 10-15 hours as background, with albums that I like and know well.³ By the end of this, I’ve usually figured out what amp/DAC combination I want to use for the review (based on both my philosophy of resource allocation in headfi AND what sounds reasonably good). I mostly avoid EQ at this stage unless something is glaringly bad, and if it’s that bad, I’ll probably just stop the review.⁴
4. Critical listening: listen to the product for 4-5 hours in the foreground, paying attention to performance and making notes along the way.⁵
5. Writing: draft the bulk of the review (usually 4-10 hours of active listening, testing, and writing).
6. Comparisons: pick the comparables, and do extensive A/B testing with them.
7. Editing: edit the review (reading it aloud helps me pick up on word errors).
8. Find frequency response graphs for the headphones and read reviews from reviewers whose opinions I trust, to see if anything I heard stands out as divergent. If I have wildly different perspectives from other people (or the graphs have something like a giant treble spike I didn’t hear), go back and to targeted re-listening to confirm. Update appropriately.
9. Photography: draft the list of photographs that I think would enhance the review, and do the product photography!
10. Finalize: format, insert pictures, double-check spelling, and upload to WordPress.
11. Finishing touches: pick the overall and cost-adjusted scores (in X out of 10 Denalis), write the TL;DR, and publish.
12. Post to Reddit/elsewhere.

All told, this is somewhere between a 30-40 hour commitment per review, which is why I tend to focus on the things that I like. If I get a few hours into listening to something and I really don’t like it, I *might* decide to try applying a little EQ if the issue is just something like bloated bass or spicy treble. I also might just decide to add it to the “I won’t review this” list.⁶ Life’s too short.

I come to the review space with some priors:

1. This is for fun, not a job. If I’m not enjoying something, I’m free to walk away. This includes, especially, debates with #DACTruthers⁷ or #SquigSniffers⁸ who think they know more about my subjective listening experience than I do.
2. Graphs are a great resource, but not the be-all, end-all.⁹ There are a lot of caveats to using squigs, which I’ll lay out elsewhere at some point. The point of listening to music is, weirdly, actually listening to music. I *like* music. That’s why I spend money on audio gear. Crazy, I know.
3. Equalization (aka “EQ”) is great. It can really enhance the listening experience, particularly in the budget space where most of the headphones have some meaningful tradeoffs in the name of pricing. I often use the built in OPRA functionality in Roon and Roon ARC¹⁰ for my casual listening. That said, not everyone is able to or willing to use EQ so I mostly try to do reviews on base tunings. Where I am applying EQ, I will try to make a note of it (particularly in the ANC space, where a lot of consumer offerings have what I consider pretty bad base tunes), and where possible I will include notes on the changes I made.
4. Though I have satisifed myself that I can *hear* a repeatable difference between some DACs and amps, I’m not convinced that I can *meaningfully talk or write* about most of those differences in a helpful way, so unless something really blows me away, I’m probably not going to review them. Happy to answer questions though; I have quite the collection of both at this point and I certainly do not lack opinions.¹¹
5. Because of 4, I try to test setups that are reasonable in context. Would most of my headphones sound best from the $650 Chord Mojo 2? Probably. But no one is going to (and probably shouldn’t!) buy a Mojo 2 to listen to a $150 pair of Beyerdynamic DT 990s; most people will use a dongle DAC or at most something like a Schiit Modi/Magni stack, so that’s mostly how I’m goign to review things.
6. I think I’m the last person on this Earth who believes this, but I vastly prefer written reviews to videos. I read MUCH faster than people talk, and I like the ability to easily jump around or do keyword searches in a way that even the best-organized and bookmarked YouTube video cannot accommodate.¹²
7. I do this mostly for fun, but I also want to be helpful. I remember what it was like to walk into this hobby with no connections or sense of how things worked, and I remember some of the dumb questions I asked early on.¹³ It’s part of why I spend so much time lurking in r/headphoneadvice and r/stereoadvice. I think we would all do a lot better by each other if we remembered where we were when we were newbies too. If there’s anything I can do to be helpful, I’m open to suggestions and will give it a shot.
8. In its current state, AI is nonsense, including in the audio space. Hot take, right? There are a lot of contexts in which AI can be helpful as a *starting point*, but you shouldn’t trust any AI ever at this point. Particularly not as a resource to pick fights with people about equipment you’ve never heard; the possibility of hallucination is way too high, and Chat GPT spits out things that *LOOK* like an answer, not that *ARE* an answer. That’s an important distinction.

So there you have it! Is this overkill? Probably. But it’s fun, and a nice break from my day-to-day, and I realized early on that I wasn’t going to remember my impressions about a headphone very well the week after I listened to them, let alone a year or two later. This is my way of recording them in a way that might be helpful the next time someone asks me about a particular pair. They’re certainly not set in stone; I’ve started going through and updating some of them as things come up, and in at least one case I’ve also softened my opinion on a particular pair over time. I will try to make it clear where I adjust scoring or add information, and if push comes to shove, at least for now my original reviews are all still at https://daemonxar.blogspot.com/.

---

1. If I hear about a headphone from a video or review I don’t sweat it; I know my brain won’t retain much in the way of detail few weeks later when I’m ready to review, but I do want to avoid coloring my impressions while doing the review as much as possible. ↩︎
2. I prefer open-boxes where possible, though I would certainly borrow and review a product where it works out. ↩︎
3. I often have a Word doc open on my computer or a note file on my phone for quick impressions, which mostly ends up being reminders to come back to specific tracks to listen. Here’s an example from testing the Sennheiser Momentum 4 this week: “Guitar line on Damn (nightmares on wax) missing?” ↩︎
4. Every once in a while I’m convinced I have a bad unit, so I might try finding another unit to listen to at this point. I once got a BAD pair of Bose Quietcomforts, so when I tried them out at a big box store again I decided to give them another shot. ↩︎
5. Listening mostly to my speaker/headphone torture list, with each track picked to test something specific, though occasionally other tracks (especially if there’s something burned into my brain that week). Apple, Tidal, Spotify. ↩︎
6. Though once upon a time, the Bose Quietcomfort SC was on that list and, lo and behold … ↩︎
7. “No human is able to hear the difference between two modern DACs!” See, e.g., this absolute champion. They are, generally, assholes. ↩︎
8. People who think that you can know how a headphone will sound from looking at a graph, enough to then tell you that you aren’t hearing what you’re hearing. They are also, generally, assholes. ↩︎
9. Sometimes audiophiles get a little too far up their own butts when it comes to measurements, and start to sound like car audio enthusiasts who care only about how loud the burp sounds in their cars. Like … good for you, bro, but we have fundamentally different goals with audio. ↩︎
10. Where it’s available, I mostly use oratory1990’s auto EQ settings as a starting place; I trust his measurements and interact with him regularly in a couple of Discord servers. Smart dude, knows his stuff. ↩︎
11. Seriously. No lack. ↩︎
12.  I hate seeing myself on camera too, but I would totally do it if I felt like it was a better user experience. I’m just not convinced it is, though it certainly appears to be potentially more lucrative in the current social media environment. Also I ain’t got time to shoot b-roll; even product photography is a lot and lighting is hard. ↩︎
13.  I e-mailed Schiit’s customer service before I bought my first amp to ask a bunch of (in retrospect) dumb questions about amps, pre-amps, DACs, and headphones. Their response was … very kind and helpful, and that’s part of why I own so much of their gear almost a decade later. ↩︎