
Most manufacturing platform evaluations run for six to nine months and produce an opinion. A structured ninety-day pilot on a single line produces evidence, and it is cheaper by roughly an order of magnitude.
Here is a pilot design that has worked repeatedly, with the parts that people usually skip marked clearly, because those are the parts that determine whether the result means anything.
Before week one: choose the line properly
The instinct is to pick the newest line with the best-instrumented equipment. Resist it. You will prove that a modern line with clean data can be connected, which nobody doubted.
Pick a line with a known, quantified, irritating problem. Unexplained downtime that everyone argues about. A scrap rate nobody can attribute. A changeover that takes longer than it should and varies more than it should.
Two criteria beyond that: the line must be representative enough that success transfers, and the supervisor must be willing. A pilot on a line whose supervisor resents it measures resentment.
Weeks 1 to 3: baseline and audit
The baseline is the part people skip, and skipping it makes the whole exercise unfalsifiable.
Measure the problem before you touch anything. Downtime by cause using whatever imperfect method exists now. Scrap by product and shift. Changeover duration and variance. Actual output against planned.
Record the numbers people distrust, and record why they distrust them. That distrust is data — it usually points at exactly the measurement the new system needs to fix first.
Then audit the line physically. Every asset, its protocol, its data rate, its owner, whether it can be modified without revalidation, and whether it has a maintenance window you can use. This spreadsheet predicts your timeline more reliably than any vendor estimate.
Expect surprises. There is always at least one machine whose controller vendor no longer exists, and one where the only available signal is a stack light.
Weeks 4 to 8: connect and make it useful
Deploy the edge gateway, establish connectivity, get data flowing into the platform.
Then do the thing that pilots frequently omit: put a genuinely useful view in front of the operators and the supervisor, not in front of the steering committee. If the only consumer of pilot data is a monthly slide, you are not testing adoption, and adoption is the variable most likely to kill the full deployment.
Two technical checks belong in this window and both are easy to defer past the point of usefulness.
Disconnection. Physically pull the uplink mid-shift. Confirm that operators keep working, that data buffers locally, and that on reconnect everything arrives exactly once and in order. Do this during the pilot, when the consequences are contained.
Buffer limits. Ask how much local storage exists and what happens when it fills. Telemetry can be decimated; production confirmations cannot be dropped. Verify the cap rather than trusting the datasheet.
Resist scope expansion to a second line. You are testing the architecture and the organisation, not the product's breadth.

Weeks 9 to 12: one closed loop
Run exactly one improvement end to end.
Good candidates, in rough order of how often they work:
-
A downtime reason taxonomy that operators actually use, replacing whatever is in place now.
-
A predictive alert on one asset class that triggers a defined maintenance intervention.
-
A changeover sequence made visible and measured, with a target.
-
An inspection step moved from paper to the terminal, with the results actually analysed.
Closed loop means something changed as a result. Not a dashboard that someone looks at — an action that happened differently because of the system. The difference between those two is the difference between a pilot and a demonstration.
Measure the delta against your week-one baseline. Then measure the thing that predicts the full rollout best: how many people reverted to their previous spreadsheet or notebook, and why.
The deliverable
One page. Baseline numbers, post-pilot numbers, cost to extend, connectivity effort observed versus estimated, adoption observations, and a recommendation.
The recommendation must be allowed to be no. A pilot that cannot conclude negatively is a procurement formality, and everyone in the room knows it.
If the answer is no, you have spent ninety days and a modest budget rather than two years and a large one. That is a successful pilot.
What usually goes wrong
No baseline. Everything afterwards becomes an argument about whether things improved, settled by whoever is most senior.
Vanity metrics. Data volume ingested, dashboards created, assets connected. None correlate with operational change.
Connectivity underestimated. In our experience it is routinely a third of total effort. Estimates that ignore it overrun and the overrun gets attributed to the platform.
Operators as data sources rather than users. If the people entering data get nothing back, data quality decays within weeks and the whole evaluation rests on decaying data.
The pilot never ends. Without a defined decision date it becomes a permanent side project that nobody can cancel and nobody will extend.
Scaling afterwards, briefly
If the answer is yes, do not go straight to the whole estate. Treat the pilot line's plant as a reference implementation and roll out with deliberate, documented local variation.
The scarce resource at that stage is an internal team who have done this once. Hub-and-spoke exists to create that team before you attempt it eleven more times. The integration boundary with ERP should be written down at the same point — ERP owns the order, the commitment and the cost; the operations platform owns execution state — with an explicit rule for what happens when both sides disagree. We go into those patterns in our cloud application development guide.
Frequently Asked Questions
Why ninety days rather than a longer evaluation? Because a longer evaluation produces opinions and committee alignment, while a short instrumented pilot produces measurable evidence on a real line.
Which line should the pilot use? One with a known, quantified problem, representative enough that success transfers, and with a willing supervisor. Not simply the newest or best-instrumented line.
What makes a pilot result meaningful? A baseline measured before anything changed, and one closed-loop improvement where an action actually happened differently because of the system.
What are the most misleading pilot metrics? Data volume ingested, dashboards built and assets connected. None of them correlate with operational change.
Should a pilot be allowed to fail? Yes. A pilot that cannot conclude negatively is a procurement formality. Concluding no after ninety days is a good outcome, not a wasted one.
Full version with architecture patterns and cost traps: Cloud Manufacturing Software in 2026.