Commercial construction runs on claims. A coating supplier says its product doubles the life of a roof membrane. A software vendor says its scheduling platform cuts delays by a third. A consultant says prefabrication trimmed 20 percent off a hospital build in another state. Every one of these claims comes with a case study, and every case study looks convincing.
The problem is that a case study is a sample of one. The project it describes had its own climate, labor market, subcontractors, and management culture, and there is no way to know from a brochure whether the result came from the product or from everything around it. Other industries that make expensive, irreversible decisions have developed a sharper way to read evidence, and the construction sector can borrow it wholesale.
What Evidence Synthesis Actually Is
In medicine, aviation, and public policy, the standard answer to “does this actually work” is not the most recent study or the most polished one. It is a structured process with a simple logic: define the question precisely, search for every study that has tested it, screen out the weak ones using explicit quality criteria, and then look at what the surviving studies say together. When the results are combined statistically, the numbers from many projects are pooled into a single estimate with honest error bars. The two approaches are related but distinct, and the practical difference between a systematic review and a meta-analysis matters when you are deciding how much weight a conclusion can bear.
Done properly, this process has a habit of deflating marketing claims. An intervention that shows a spectacular result in one study usually shows a smaller, more believable effect once every study is on the table. Sometimes the effect disappears entirely.
Why Single Results Mislead, in Construction Terms
Three mechanisms do most of the damage, and every one of them has a construction analogue.
The first is small samples. A safety program trialed on two job sites can show a 40 percent drop in recordable incidents through nothing but luck, because incident counts on any two sites bounce around from year to year anyway. Small pilots produce big, unstable numbers in both directions.
The second is selective publication. Vendors publish their wins. The three projects where the new formwork system saved money become case studies; the two where it fought with the local labor pool and blew the schedule are never written up. Researchers call this publication bias, and it means the visible evidence is systematically rosier than the full evidence. Any decision based only on what a supplier chooses to show you inherits that bias in full.

Ten outcomes of the same product across ten projects. Only the best three become case studies.
Image Source: original illustration created for this article, royalty-free.
The third is confounding. The flagship project that finished early with the new scheduling platform also had the company’s best superintendent, a mild winter, and an owner who made decisions quickly. Case studies bundle all of that into one number and credit the product.
Putting It to Work in Procurement
None of this means ignoring innovation, and it does not require a research department. It means changing what you ask for before a major commitment.
Ask vendors for the full record, not the highlight reel. How many installations or deployments exist? What is the average outcome across all of them, not the best three? A supplier who will not answer that question is answering it.
Ask whether independent evaluations exist. For many products and methods relevant to construction, from modular techniques to safety interventions to contract structures, peer-reviewed studies, standardized test data, and government evaluations do exist. Trade associations, university construction management departments, and bodies like the Construction Industry Institute publish syntheses that no brochure will show you.
Weight evidence by quality, explicitly. A measured before-and-after comparison across a dozen sites outranks a testimonial. Data collected by someone with nothing to sell outranks both. Writing these rules down before the pitch meeting keeps the shiny case study from doing your thinking for you.
Run your own pilots with the decision in mind. If you trial a method on one project, define success in numbers before you start, and treat the result as one data point to add to the pile, not as a verdict. Two pilots with pre-agreed metrics beat five impressions.
The Estimate Is a Forecast, and Forecasts Need a Base Rate
There is a deeper habit underneath all of this. Every bid, schedule, and budget is a forecast, and the reliable way to sharpen a forecast is to anchor it to the measured outcomes of many similar projects rather than to the details of the one in front of you. Cost overrun research across thousands of projects shows the same pattern again and again: the inside view, built from a single project’s particulars, is optimistic, while the outside view, built from the distribution of comparable projects, is accurate. Forecasting researchers call this reference class forecasting, and evidence synthesis is simply the same outside view applied with discipline.
Firms that internalize this make measurably different decisions. They adopt fewer innovations, but the ones they adopt tend to survive contact with reality, because the claim had to hold up across the whole body of evidence rather than in one flattering story.
None of this is exotic. Evidence-based practice built modern medicine, and the same structured reviews that decide which treatments hospitals use can be applied to any operational question with published research behind it, and specialist teams like the one at www.researchgold.org spend their working lives running exactly that process. The tools are open, the logic is public, and the question that starts it costs nothing to ask: not “do you have a case study,” but “what does all the evidence say?”
That one question, asked consistently at the point of purchase, is worth more than any single product on the market.





