Peter Seale's weblog

Wednesday, August 22, 2007 4:53:59 AM UTC  UPDATE 2007-08-21: I've removed my earlier misinformation. In its place, I'll leave you with the following:

The Golden Rule of Advanced Troubleshooting

It's always the invisible layer.

In the most recent case, I was trying to eliminate a long web page loading delay. I'll go into details of the specific problem another time—the point being, the root problem was an invisible proxy on the network layer. Apparently HTTP proxies have problems working with gzipped packets. Who knew.

The Golden Rule: Enumerated

What do I mean by "invisible proxy"? Well, here are some examples (AKA culprits!):

• Firewall
• Antivirus
• HTTP (web) proxy
• SMTP (mail) proxy

If you're pulling your hair out troubleshooting like I was, you'll want to review the Golden Rule. Oh, and another thing:

Rare Problems

I think it's important to note that in this context  I'm talking about troubleshooting rare problems. If this was a common problem, you'd have fixed it already; you wouldn't be searching for invisible proxies or discovering new layers, because the visible/tangible layers are so much more discoverable. Why check your firewall settings when your network cable is unplugged?

Trust No One: You

Rare problems require a different mindset from common problems.

Trust nothing; start looking at your own assumptions and (of course) anyone else's assumptions; look to the Golden Rule of Troubleshooting. Double-check your earlier work. Double-check your double-checking to be sure you covered everything. Get some rest, then come back and double-check your double-double-checking.

Trust No One: Them

Double-check everyone else's work (yeah, I know).

When doing advanced troubleshooting, it's important to keep in mind that a) with computers, there is always a logical answer (even if that answer is "the hardware is intermittently breaking), b) you've already isolated every single layer possible, and c) it's now time to stop believing people and now time to start checking their assumptions. An important corollary to part c is you must also turn a distrustful eye towards yourself. Did you check everything? Are you sure? Before you make that accusatory phone call, are you 100.0% sure?

And of course, Google

I don't know what we did before search engines. If nothing else, even if Google can't help me find the solution, at least it can point me to others experiencing the same problem.  If nothing else, search engines prove that we're not alone—we're not crazy!

And Vendor Support

(this section intentionally left blank)