March 2022, 6:40 PM: The Dry Run That Wasn’t
I was alone in the integration shop on a Friday — never a good sign. A packaging client had a conveyor line that needed to be running by Monday morning, and I’d spent the better part of three days writing what I was sure was clean basic programming logic of Omron PLC on a CP1H controller. Servo positioning? Done. VFD-driven conveyor? Wired and tested. Safety interlocks? Checked. All that was left was the first full dry run.
I remember thinking: “This is gonna be smooth.”
It wasn’t. But the real problem started weeks earlier, and it had nothing to do with code.
The $600 I “Saved” on VFD Repairs
Three weeks before that Friday, our maintenance guy — Dave, who’d been fixing drives since before I graduated high school — flagged the VFD on the conveyor motor. His assessment: the cooling fan was squealing, the heat sink was caked with dust, and the thermal paste was probably due for replacement. His quote for the VFD repairs came to $600 in parts and an afternoon of labor.
I looked at the budget spreadsheet. We were already $1,800 over on this project. My gut said, “Pay the $600, deal with it now.” The spreadsheet said, “Push it to Q3 — VFDs are tanks, what are the odds?”
I went with the spreadsheet (ugh). That $600 “savings” becomes relevant in a minute.
The Basic Programming Logic Mistake
Meanwhile, I was writing the pulse output logic for the servo axis. I’d used Omron PLCs before — mostly CJ and NJ series — and I figured the CP1H pulse instructions worked the same way. I cracked open the Omron PLC SPED(885) instruction manual, skimmed the section on pulse output completion flags, and moved on.
I knew I should’ve read the whole page — especially the part about when the completion flag actually fires relative to the pulse output — but I thought: “I’ve used pulse instructions for years. What are the odds this one’s different?”
Well, the odds caught up with me.
Here’s the short version: I’d written the logic assuming the SPED(885) completion flag would fire after the last pulse was sent. In reality — because I’d misread the timing note in the manual — the flag fired when the pulse output started its deceleration phase. My downstream logic treated that as “position reached,” which meant the conveyor kept advancing for about 80 milliseconds longer than intended.
Eighty milliseconds. At that conveyor speed, that’s about 1.2 inches of extra travel. On a packaging line with mechanical end stops, 1.2 inches is all you need to slam into a hard stop and jam the whole system.
The Dry Run (and the Relief That Didn’t Last)
When I hit the start button, the servo moved through its sequence, the conveyor advanced, and the system stopped exactly where the software said it should. I watched the position readout hit the target, the completion flag trip, and everything come to a clean halt.
So glad I double-checked the wiring (thankfully, I had). I almost skipped that step to save time, which would’ve meant a much more embarrassing first run. But the run looked clean. I logged the test as passed and went home.
That was the mistake I didn’t catch until Monday.
The Crash (and the $2,400 Bill)
Here’s what actually happened when we ran the line at production speed for the first time:
- The servo over-traveled and hit the mechanical stop — because that 1.2 inches of extra travel only showed up under load, at speed, with the conveyor loaded.
- The conveyor motor — which was still running because my interlock logic didn’t account for the jam — stalled under load.
- The VFD, whose cooling fan I’d deferred, tripped on overheat. The braking resistor cooked itself trying to stop the motor.
- I stood there, alone again (this time on a Monday), watching the fault code blink on the drive display (unfortunately, I knew exactly what it meant).
The surprise wasn’t the servo crash — I’d half-expected to tune something after the first production run. The surprise was the VFD. I’d assumed the drive would just fault out and reset; I didn’t realize the deferred cooling issue meant the braking resistor was already running hot, and a stall condition was enough to push it over the edge.
Those VFD repairs I’d deferred for $600? Emergency after-hours repair with rush shipping came to $2,400. Plus three days of lost production on the client’s line. Plus a very awkward Monday call with the plant manager where I had to explain why the line that was supposed to be running by Monday morning was going to be down until Thursday.
What the Numbers Actually Show
Let me do the total cost of ownership math I should’ve done in the first place:
- Deferred VFD maintenance (the “savings”): $600
- Emergency VFD repairs: $2,400
- Lost production (3 days at the client’s rate): ~$3,000
- Rush freight on the replacement drive: $340
- My weekend of unpaid overtime: not billable, but real
Total: roughly $5,700 in direct and indirect costs to “save” $600. That’s a 9.5x multiplier on a decision I made in about 90 seconds.
“The cheapest line item on the spreadsheet was actually the most expensive decision on the project.”
The lesson wasn’t subtle. The cheapest line item on the spreadsheet — the one I deferred — was actually the most expensive decision on the project.
What I Changed (and the Checklist I Built)
I’ve now logged 14 significant mistakes across 7 years of systems integration work, totaling roughly $11,000 in wasted budget. Some cost more than others, but this one was the worst, because it was two separate “what are the odds?” decisions stacked on top of each other.
Here’s what I do differently now (and what’s in the checklist I keep on my desk):
- Read the whole page in the manual. Not the section I think I need — the whole page. The SPED(885) manual has a timing diagram that would’ve saved me if I’d actually looked at it. Now I print the page and highlight the timing section before I write a single line of logic.
- Schedule VFD preventive work as TCO, not as a line item to cut. A $600 fan-and-paste job is 4x cheaper than an emergency repair. That math isn’t hard, but somehow we never ran it that way before. I now include a “deferred maintenance cost multiplier” line in every project budget.
- Double-check pulse logic in test mode. Before ever hitting a real conveyor, I now run pulse instructions against a scope or a dummy output and verify the timing against the manual’s timing diagram. Costs 20 minutes. Saved me twice since — both times on CP1H projects, both times the timing was off by less than 100 milliseconds.
- Don’t ignore the gut. The spreadsheet said defer. My gut said don’t. The spreadsheet was wrong — and honestly, the spreadsheet is wrong more often than I used to admit. If my gut and the spreadsheet disagree on a maintenance call, I now go with the gut and find a way to justify it later.
And yeah, I’ve started applying the same TCO thinking to everything — even at home. Last month, I looked up the proper way to tighten an oil filter instead of just “hand-tighten until snug” like I used to (I once had a filter vibrate loose on the highway — could’ve been an $8,000 engine, cost me nothing because I caught it, but still). And when I swapped out the Chamberlain garage door control panel last fall, I actually read the wiring diagram first instead of assuming all control panels wire the same way. Old me would’ve learned the hard way — again.
Two years later, I still keep the SPED(885) manual on my desk. Not because I need it every day (I don’t — the CP1H is an older platform now), but because it reminds me that the most expensive mistakes I’ve made were the ones I was too confident to check.