Tag Archives: fail

Premature Flip-flop Failure

Flip-flops are pretty much de rigueur footwear for Florida. There are exceptions, of course: work boots for lawn work, riding boots for motorcycling, sneaks for walking/running  are good examples that come to mind. But I take my flip flops kinda seriously.

Back in July of 2014 I wrote about the de-lamination failure of a pair of favorite flip-flops. The gist of the article lamented how the Florida environment seems to destroy just about everything. The failed footwear had lasted about eight years before succumbing, and Pam ordered replacements which arrived that day. I expected ’em to last a similar amount of time.

Flipflop failure. Inset shows failure point, earlier duct tape hack for comfort.

They didn’t.

July 27 they fully died. I say fully because the between-the-toes part of the right foot stretched had some and became uncomfortable sometime last winter. Pam addressed that emergency with a commodity plastic pair – I think she spent a dollar. (Still, I wrapped the stretched between-the-toes part with a bit of duct tape, which helped the comfort just a little bit.  Then I dedicated them to poolside use – they were too new to throw away.)

So, let’s see… Retail cost was $59, on sale for $29.99. Sales tax, Land’s End has nexus in Florida, $2.28. Shipping was $8. Total cost was $40.27. They arrived July 16, 2014 and, setting aside my duck tape crutch, they totally failed July 27, 2017 – that’s 3 years and 11 days. That brings the cost-of-ownership for those suckers to a whopping twenty-five and a half cents per week!

Contrast that to the earlier pair, which were $30.95 (with tax and shipping) and lasted 7 years, 11 months and 10 days. Only 7 and a half cents per week.

Lets put that into perspective, you bought a new Harley for thirty large in 2006. Eight years later you bought the same bike and the cost had jumped to a hundred. And the motor blew up three years later.

Planned obsolescence? Degradation of quality of manufactured goods over time? Product abuse? Or just Florida killing stuff?

You decide.

Hydra

This is a story about Hydra. Hydra’s a box, a computer, that up and died the death that old machines sometimes do.

Hydra, dead, stripped of all innards save the CPU and motherboard, awaiting transport to the parts shelf. Click for full-size image in a new tab.

I’m not 100% certain why Hydra’s dead, but pulling everything except the CPU still won’t elicit so much as a measly POST beep from the aged motherboard. I meter-tested the power supply. (I had another box on the bench for a PSU replacement, so I briefly stuffed the new PSU into Hydra just to make sure.) There’s nothing left to die except the mobo or CPU!

“So what,” I hear you thinkin’, “who TF cares about yer old box?”

Well, I do.

See, Hydra’s served the house in various capacities for a long, long time before retiring to the un-insulated sun room by the pool deck – most definitely an unfriendly environment for computers. The moisture, for one: Florida’s humid. The there are the temperature swings; in winter it can drop to near freezing and closed up in the summer it might reach 115F – or more. Environmental extremes have been the story of Hydra’s life. Finally, Hydra’s kinda remarkable in that it’s one of the oldest processors that Windows 10 will run on: the AMD Athlon 64 3200+.

So yeah, it’s worth taking a few minutes to write about little Hydra’s uncomfortable life.

Dex (left) & Reptar, circa 2002, about four years before Hydra.

For that we have to go back to Monday, October 16, 2006. That’s the day I walked into a local Comp-USA (remember that name?) with the idea of upgrading the house servers. At that time there were two. A more-than-10-year-old Pentium Pro box named Dex running Win2K Server, and a slightly newer Pentium II box named Reptar doing file server duty. Dex and Reptar were simply running out of gas.

I wanted a 64-bit CPU, a couple of GB of RAM with room for some future expansion. Remember, memory was considerably more expensive than it is today. I wanted the ability to use my existing IDE drives plus some SATA ports for later. I wanted a PCI bus. Overall, just something a bit more modern, something that would run VMware so I could segment the family’s workload.

I walked out with basically this:

Plus assorted support stuff like a cheap case, power supply, optical drive, and so on. Came to about six hundred bucks. Sure, I could have done better online but WTF, that’s what retail’s all about; getting it now. I assembled and IPLed the box that very afternoon and Hydra took up residence in the dusty, dark basement. Right next to the furnace. So Hydra’s twenty-four seven life began.

Hydra survived much abuse. The second phase of the basement refinishing project comes to mind. The drywall work deposited a coating of dust on Hydra’s innards that called for a weekly blowout to keep it from burning up. The un-insulated NJ basement was a harsh home.

Over the years came more memory, a couple of hardware RAID cards, more drives, and still more drives. That little case became dense and heavy. And ugly, as I cut more holes for fans. Yeah, it got loud, too, but in the basement it didn’t matter.

Win2K Server gave way to a bare-metal hypervisor for a while. Fast like shit through a goose, but tricky to administer. Bare-metal gave way to Linux. Hardware RAID gave way to software. The years passed.

In December 2012 we moved to Florida. We unceremoniously tossed Hydra into a U-Haul trailer with the rest of the stuff we didn’t trust the movers to handle and pulled to its new home.

Environmentally the new network closet was an absolute step up. But Hydra screamed like a jet on full afterburner with all those drives and fans. In the old basement it didn’t matter but the closet’s just off the office, quite distracting…

By the end of the first quarter of 2013 Hydra entered a much-needed semi-retirement. The replacement, named dbox, was a quad-core box from the parts shelf, with way more memory and fewer, but higher capacity drives. By then all the server roles were running as virtual machine guests. The migration was super-fast and super-easy.

In the garage, Hydra rested on the parts shelf before being called upon to support a Facebook project Pam had launched. I don’t really remember exactly when that began. Hydra was much quieter, stripped to a single drive running Windows 7. We shoved the headless case under the healing bench near the door and Pam ran her project from her Windows desktop, logged in using the Remote Desktop Connection tool. It wasn’t the highest performance configuration in the world but it got the job done.

Without the benefit of a proper UPS poor Hydra suffered a new peril: power glitches. We got used to looking for the power light under the workbench as we passed. If it was dark anyone could thumb the power button and go about their business.

That arrangement lasted about a year. Pam’s project wound down and Hydra went back into retirement.

Meanwhile, in the real world Windows 10 was getting legs. I’d come to like the Tune In Radio app. One can only take so much country and classic rock from the local stations and I’d had my fill. I wondered… could a Windows 10 box and Tune In Radio bring superior tunes to the pool deck? Was there any spare hardware around that could run Win10? Microsoft took great pains to exclude older hardware, even while offering free upgrades. Would Win10 run on Hydra’s CPU, now approaching twelve years since its introduction?

It turns out the answer was yes! Well, there were issues to overcome along the way, but yes.

A Win10 license costs more than the budget for this venture, which was exactly zero. Microsoft was still offering free upgrades from Win7 so the plan was to follow that path. Hydra had a Win7 Pro 64-bit OS from Pam’s project so we got that upgrade started. The several-gigabyte download took forever over the crappy ADSL connection. Then the upgrade failed.

That’s how I learned that Hydra’s Athlon 64 CPU doesn’t support the CMPXCHG16B instruction. This instruction, commonly called CompareExchange128, performs an atomic
compare-and-exchange between 16-byte values. And 64-bit Win10 (and 64-bit Windows 8.1) requires this instruction.

CMPXCHG16B isn’t required by a 32-bit Win10. The path became clear. Install a 32-bit Windows 7. This meant giving up any installed memory over the 3.5 GB mark. Fine. Get Windows 7 activated. Install all the service packs and patches. Finally, upgrade it to Win10. Remember that crappy little error-prone ADSL connection? That, along with the lengthy downloads and general slowness of the ancient hardware… there went a couple of days. Thankfully it didn’t need much attention.

But it worked!

And that’s where Hydra lived out its days. Providing great radio out on the pool deck. Enduring temperatures from near-freezing to well over a hundred degrees.

The evening of May 15, 2017, I attempted to kick Hydra to life to collect the latest Win10 updates. I thumbed the power button, and heard it starting up as I walked away. Later I noticed it had gone down. Hydra never booted again.

A few interesting observations…

  • Hydra began and ended life on a Monday. (Watch out for Mondays.)
  • Hydra ran ten years and seven months. 10-7. If you remember the old 10-codes the cops and CBers used to use, 10-7 means “out of service”.
  • Hydra ran 24/7 for most of its life. If we assume about 9 years of total running life, that works out to about three-quarters of a cent per hour against its original installed cost. Absolutely worth every nickel.
  • Hydra died on its side, on the floor, in an overheated room, alone, behind the bar. A noble death.

And that’s where today’s story ends.

Maybe you’ve got an old AMD Athlon 64 3200+ floating around in your parts bin? Maybe you’d like to give it a new home? If it resurrects Hydra then it’s mine and I’ll give you a nice, fat mention in this story AND a link in the sidebar. If not, I’ll send the chip back to you with my thanks for a noble effort.

But wait! What about the tunes out on the deck? It just might be resolved. Well, at least some preliminary testing seems to show that it can be resolved with a little bit of creativity.

So that part of the story needs to wait. But I can promise you that if this scheme works it’ll be even weirder.

Shifty Business – Another F-150 Failure

Pam and I had gone to the local pizza place for some takeout. In typical “I’ll buy, you fly” mode, Pam drove the F-150 with me in the passenger seat. She parked head-on by the front door and waited while I ran in for the goods.

It was just past closing time. Staffers killed the lights and ran out the door behind me.

Pam selected reverse and backed out, then pulled the lever for drive, hit the gas – AND THE TRUCK SURGED BACKWARD!

There wasn’t much room in the lot so she maneuvered to the curb, occupying about three marked spaces, and killed the engine.

It’s dark, beginning to rain, maybe three or so miles from home, hot food in the back seat, and we’re hungry. I called the kid for a lift home. We’d eat and think, then come back. Maybe the rain would pass.

The console lever felt sloppy, disconnected, I suspected a mechanical issue like a linkage or cable had worked its way loose. The console lever has a history. I removed the plastic shroud and felt around as best I could in the darkened cab; nothing felt out-of-place and my earlier hack/repair felt intact.

The kid soon arrived. We locked up and left to eat.

Sure enough, a bit of Internet searching turned up lots of similar failures! The most common issue came down to the cable-end detaching from the transmission lever. We returned to the truck armed with a good flashlight and I crawled under for a look. Before you ask, yes, the rainwater had pooled underneath…

Now it was clear that the cable-end was no longer attached. By manipulating the console and transmission levers it was possible to reattach, but when the console lever moved the cable end fell right off. The press-fit retention was no longer retaining!

I figured we could get the truck home, though. That’d free up parking for the restaurant and make for more comfortable work. The safety interlock only allow the truck to start in park and I could manipulate the levers to do that from beneath. With the truck started I could move the transmission lever into drive while Pam applied the brake. Then I’d crawl out she could drive home.

Pam was less than thrilled with the idea. But she did it. She probably pressed that brake pedal almost through the floorboard while I was under the truck fiddling with the transmission lever, engine running!

Cable end at top, transmission lever at bottom. The white plastic bit captures that knob.

The Root Cause
The cable end is a roundish half-bowl of plastic molded onto the semi-flexible metal rod that extends from its sleeve. In the picture you can see a white plastic insert fitting in the black bowl. The transmission lever has a protruding machined knob that’s captured by that bit of white plastic.

Failure occurs when the white plastic insert no longer captures the knob.

The cable assembly is not adjustable. The length is exactly what it is. That’s important because…

When the console lever’s placed in park – that’s how the truck spends much of its time since being manufactured – there’s much linear force being applied against the transmission lever. By that I mean if you select park and slip the cable end free of the transmission lever (easy to do, now that the part’s failed), the cable end springs out extend a good 3/8″ past the transmission lever. You cannot put it back in place without manipulating the console lever positions, the end-to-knob alignment is that far off. So of course that 29-cent bit of plastic will fail eventually! It’s designed to fail!

The Repair
The cable assembly – part number 4L3Z-7E395-CA for my 2004 unit – is available on Amazon for about $48. I don’t have a shop manual – that’s another story for another time – but it looks like it could be replaced in a couple of hours. Beer optional.

Obviously, a dealership could handle the repair. I heard that runs around $300-$350, including parts.

Ascension Engineering’s current location, under construction when Google Earth grabbed this shot.

But let me introduce you  to Ascension Engineering. They produce a line of replacements for those little white pieces of plastic – apparently it’s a common failure mode across a wide variety vehicles, not merely Fords. The parts sell through their website, BushingFix.com. And business is apparently pretty damned good – Ascension Engineering’s principal relocated to some considerably nicer digs between May 2015 and March of 2016…

Shipping origin. Click for detail.

Y’know, $25 is a lot of money for a bitty bit of plastic. (Update: I learned, when sending a link to this article to the manufacturer for review, that the price is now reduced. My luck, right?) Okay, there’s design, tooling costs, manufacturing, but any number of Chinese outfits will do all that. Probably including the engineering design, too. It seems likely that they already manufacture those little bushings for the auto manufacturers. That Mr. Smith, he’s one smart cookie!

Screw it. I ordered a kit. There was sales tax, we’re both in Florida. And shipping was, I thought, a little high at just under $6 for USPS. The total cost was $32.29. I showed up at my door in a few days, shipped from Charlotte, North Carolina.

The cage-hack. See text for features. Click to enlarge for detail.

But First, An Interim Hack
A couple of weeks before the order and permanent repair, a simple hack was necessary to keep the truck on the road. A truck’s a useful tool here in rural Florida. There’s trash and recycling to haul, stuff like that.

I hacked up a little cage from (what else?) coat-hang wire – easy to work with, yet stiff enough to enclose the cable end.

It’s got some nifty features. First, it’s a cage. It’s solidly attached to the transmission’s shift lever and doesn’t contact the cable or its end except where it absolutely must, to prevent the thing from slipping off the knob. That contact is minimized by a custom thrust plate constructed of softer plastic. (Don’t be fooled, the thrust plate is from a plastic storage bin. We use the bins as high-walled litter boxes for our feline residents, and this is the material cut out to form a door.) Notice the bend in the thrust plate, and the cutouts so the cage retains it.

The hack would likely outlast the truck. But a replacement part was on the way.

The Replacement Bushing
It arrived in about a week. Here’s what $32.29 bought me:

The package, as received, followed by the individual parts and instructions. It’s actually well-thought out and about as foolproof as it gets. Click for a larger image.

Three bits of plastic: the bushing itself plus two more that served as press-blocks. Click to enlarge the image and see the instructions. Leaning way towards foolproof, I’ve gotta say. It took me longer to remove my hack than it did to install the replacement.

Debris in my cable-end cup. The rear of the truck is the right side of the image. The cable presses rearward when the transmission is in park. Click to enlarge.

Part of the install involves digging out any old bushing parts from the cable end cup. I used a carbide-tipped scribe and it cleaned up in a few seconds. Here’s a shot of what was left in my cup before that step. Pretty disgusting. But what’s clear as an unmuddy lake is how the linear force of sitting in Park had basically ruined one side while the other remained basically unworn. Ford’s non-adjustable setup is designed to fail. It’s only a matter of time.

Bushing installed. Click to enlarge.

I assembled the sandwich of plastic bits and used a pair of Channellock pliers to give it a squeeze. I chose the Channellocks because of the adjustable jaws but I think a pair of ordinary pliers like those found in the average person’s tool bag would have done the job just as well. You’d need pliers, though, it’s a bit much for fingers alone.

Repair complete. Click to enlarge.

The rejuvenated cable end mated to the transmission lever with a satisfying click. Then I exercised the console lever. It felt great.

Notice in the completed repair image that there’s a slight gap between the cable-end cup and the transmission shift lever. This tells me that the replacement bushing is the proper size for the job. If the knob sat too deep then the two parts would rub, wearing the cable-end cup.

So How’s It Holding Up?
It’s been a few months since all this went down and so far, so good.

No issues, no complaints, the repair feels as tight as ever. How long will it last? Hard to say. Ford’s designed-to-fail assembly of the subsystem remains unchanged. What’s a worse environment? New Jersey winters or the Florida heat? Time will tell.

Dodge Dart

When I was in my twenties I went through one of my car-less periods, only a motorcycle for basic transport. Rain or shine, winter or summer, I rode. Jerry, a guy I knew, felt bad for me one wintry day. He gave me a car.

I don't recall the year, but it may have looked something like this. Certainly not as clean.
I don’t recall the year, but it may have looked something like this. Certainly not as clean.

It was an old Dodge Dart. I don’t recall the year. If you’re anywhere near my age you’ve probably seen thousands upon thousands of these old Dodge Darts on the road. They were bulletproof: slant-six engine; three-speed on the column; torsion-bar suspension; bench seats complete with the saggy back rest. This one was blue. The interior was all musty from sitting in Jerry’s mom’s backyard for months and months. (She may have pressured him to get it out of there, helping lead to my good fortune.)

I remember when Jerry bought the Dodge. He wasn’t much of a mechanic and he had asked for my help with its assessment. “The clutch is slipping some,  you’ll need to replace it eventually. But otherwise it’s reasonable.” I think he paid a couple hundred for it.

When I got my hands on that old Dodge the clutch was still slipping some. The engine had two operating temperatures: hotter than hell, and hotter than hotter than hell. Coolant boiled out regularly; the water jugs in the backseat were a permanent fixture. But that ol’ engine never faltered, not once. In fact, it always delivered excellent heat. And judging by the sludge in the crankcase I don’t think Jerry ever got around to changing the oil in the couple of years he had it. I know I didn’t.

I used to have fun with that slippery clutch! I knew a new friction plate would be cheap and easy to install. It became something of a game to see how much abuse the poor little clutch could take. I’d wind that little engine for all it was worth and sidestep the pedal just to catch a whiff of the burning plate.

I was using that very technique to enter the highway, pulling out of a local titty bar one afternoon, when the clutch signalled it had finally had just about enough. The sound was odd and clunky – not good at all. A bit of friction remained, though, and the car lurched ahead. I didn’t dare touch the pedal during the short ride home.

The decision had been a sound one. The very next pedal depression was its last. Oh, the pedal would move alright, but it no longer mattered. Engine on or off, pedal or not, any gear could be selected at will while the car no longer moved on its own.

So the next weekend I picked up a friction plate, release bearing, and other assorted parts and set to work. The drivetrain and transmission came out easy enough. But what remained of that poor clutch was a sight to behold. Some dust and shredded friction material along with some broken metal fell out of the housing to the asphalt. It was one of those moments that fairly begged for a digital camera. But this was WAY before that technology became ubiquitous. In short order the new clutch was again transferring engine power to the transmission!

I was working in Bridgeport, Connecticut at the time and Monday morning I set out from my New Jersey home with confidence. With a week’s worth of clothing (and several water jugs) in the backseat, all was well with the world and the ride up Route 95 went without incident. But as I reached the job site there was a mighty clunk from the front end as the left front quarter sagged nearly to the ground. A quick look confirmed my suspicion: the torsion bar had broken free, its mount in the frame rusted out.

At the end of the day, before I checked into my hotel, I found a salvage yard and limped the old Dodge to its final resting place.

The yard operator paid me just about enough to cover my clutch parts.

Lizard – –

We have lots of lizards in Florida. Today, one less.

Pam had been out in the driveway hosing down a trashcan. We like to keep ’em clean – bears, y’know – and Pam was handling that chore while I was in the garage installing new foot controls on her motorcycle. She called me outside.

Pam was saying something about the hose and the water and how it wasn’t flowing, and my mind raced ahead. I was thinking the worst, of course: plumbing failure, well pump failure, and so on. I caught myself and tuned back in. Pam had checked the hose for kinks and, finding none, removed the high-pressure nozzle. And the root-cause of the flow issue had become painfully obvious.

A lizard apparently entered the garden hose. Then Pam came along, connected the hose to a spigot and added the high-pressure nozzle in order to work on the trashcan.

The lizard was undoubtedly quite surprised by the face full of water.

The exit hole of the high-pressure nozzle is no more than 3/16 of an inch when fully open. The considerably larger lizard made a valiant effort at getting through that tiny hole, tail end first. It failed. But along the way it managed to slow the flow of water to a near stop.

And that’s where I came in.

Lizzie was in pretty bad shape, as you might imagine. I tried, for a bit, to clear the nozzle. But it quickly became clear that tools would be needed and it wouldn’t be the most pleasant of jobs.

I suggested the medium-pressure nozzle and Pam resumed her work. I left the lizard-clogged nozzle in the grass and resumed my work.

With that move the lizard assumed its place in the food chain. By tomorrow or the day after, I figure, the nozzle will be clear.

No pictures because, well, y’know, it’s really kinda gross. I’ve got no qualms dispatching insects but dead lizards are sort of sad.

Jury Duty – NJ Fails Again

I returned from some travel the other day to find this in the mailbox. It’s an unwelcome notice, if what I hear from many people is any indication. But this one brought some different feelings.

Richard is my Dad. He passed almost two years ago.

One would think that the various state and public records of Richard’s passing would have prevented the generation of this notice, but no – it’s a fail. To underscore the failure, I recall that Richard was summoned to Jury Duty some years back. I handled the notice because he was unable to read it for himself – stroke damage had robbed him of that ability.

On his behalf I had requested – and and was subsequently granted – an excuse. I cited reasons including health and ability as well as age, which alone would have sufficed (see the NJ Judiciary FAQ). Age only goes backward in the movies, and so I figured Richard would no longer be troubled by Jury Duty.

I was wrong.

I’m thinking that I’m going to fill out the form and return it, requesting an excuse on account of, well, death. Perhaps they’ll get the message.