Independent Review in AI Product Development

Poor decisions do not always result from missing information. A team can have enough information to feel certain without testing whether that certainty is justified.

As building gets cheaper, that becomes a more expensive mistake. When a team can go from idea to working prototype in days, the constraint moves. The question stops being whether something can be built quickly and becomes whether the reasoning behind it has been challenged by anything other than its author.

The persuasive force of a single analysis

A single author organizes the material into a coherent narrative. That can make the analysis appear more certain than the evidence warrants. Uncertainties omitted during drafting are no longer visible to the reader. This applies whether the author is a colleague, consultant or language model.

Tversky and Kahneman documented the mechanism underneath it in 1974: when people estimate, they anchor on an initial value and adjust too little, even when the anchor is arbitrary.¹ Once the first analysis in a discussion sets a frame, later thinking adjusts around it instead of starting again.

At group level the effect compounds. Janis studied a set of policy failures, Pearl Harbor and the Bay of Pigs among them, and named the pattern groupthink: cohesive teams under time pressure suppress dissent, converge early and stop testing assumptions.² The driver is social, not intellectual.

What research on group judgement shows

The findings are narrower than popular summaries suggest, but they still offer useful guidance.

Galton reported the well-known case in 1907. At a fatstock exhibition in Plymouth the year before, visitors paid to guess the weight of an ox. Of the cards, 787 were usable, and the middlemost estimate came to 1207 pounds against a true dressed weight of 1198, within roughly one percent.³ Galton's point was not that crowds beat experts, and he did not claim that. His point was about the shape of the distribution and the condition that produces it: the guesses were made independently, by people who could not see each other's cards.

That condition is the whole mechanism. Independent errors cancel. Correlated errors accumulate, and the group's apparent agreement becomes evidence that people read each other rather than evidence about the ox.

Woolley and colleagues reported in 2010 that group performance across varied tasks loads onto a general factor, correlated more with even participation and social perception than with the members' individual intelligence.⁴ Worth carrying the caveat that the finding has a contested replication record, with later work arguing that member intelligence predicts group performance more strongly than the original paper suggested. Treat it as suggestive rather than settled.

The Delphi method and early group influence

In the early 1950s RAND sought forecasts from expert groups without distortion from rank, dominance or premature consensus. Dalkey and Helmer published the resulting Delphi method in 1963. Experts first respond independently and anonymously, then see the distribution without names and may revise their judgement.⁵

Cooke later formalised the harder half of the problem. Reliable elicitation means calibrating experts against quantities whose answers are known, weighting by track record and stating explicitly where uncertainty remains instead of forcing a clean answer.⁶ The output is not an average of opinions. It is a summary of what the evidence supports, including where it stops.

Klein's premortem approaches the problem differently. Participants assume that a plan has failed and identify possible causes.⁷ Mitchell, Russo and Pennington described how treating an outcome as settled changes the explanations people generate.⁸ Percentages commonly attached to this effect are not adequately supported by the paper and are omitted here. The method remains useful because it makes criticism easier to express.

The evidence for structured disagreement

Modestly, and with caveats worth stating.

Schwenk's 1990 meta-analysis found advantages for devil's advocacy and dialectical inquiry over consensus approaches on decision quality, drawn largely from laboratory studies with student samples, and inconclusive between the two techniques.⁹ Nemeth and colleagues found in 2001 that authentic minority dissent stimulated more divergent thinking than one assigned devil's-advocate condition, with some evidence that role-played dissent hardened the majority instead.¹⁰ Neither result licenses a universal claim. Both point the same way.

Blinding does measurable work. Tomkins, Zhang and Heavlin ran a controlled experiment on the reviewing for a 2017 conference and found that reviewers who could see author identity favoured papers from prestigious institutions and famous authors, a preference absent from the double-blind arm.¹¹

More opinions do not automatically produce better decisions. The evidence supports independent first judgements, visible disagreement and summaries that preserve uncertainty instead of averaging it away.

How Forenta applies these principles

The implementation differs between the two product environments. Forge analyses text supplied by an individual user and returns a single structured assessment. It should not be read as an expert panel or an independent audit.

Forenta for Business supports a controlled organizational process. Relevant perspectives are kept separate before they are brought together, material disagreement remains visible and the resulting advice is attached to a decision point. The public explanation is intentionally limited to these design principles. The exact orchestration is part of the internal system.

Safeguards with operational consequences run outside the language model. An unresolved blocking risk cannot disappear because a synthesis sounds confident. A person may still override a recommendation, but the reason and the original advice remain in the record.

Where this falls short

A structured process can reduce anchoring and premature consensus, but it cannot guarantee an accurate decision. AI-generated perspectives can still share blind spots, especially when they rely on similar models and the same source material.

Independent human judgement on real decisions remains the relevant test. Forenta has not yet published enough outcome data to claim that its method improves project success rates.

Forge does not look up missing facts. It reasons about the text the user supplies. A Forenta for Business pilot may use explicitly connected read-only context, but the analysis remains limited to the evidence made available.

Judgement is a process, not a moment

The finding is narrow and it holds. Independent analysis, collected before group influence, structured so disagreement survives, and summarised without averaging, produces better calibrated judgement than consensus-first thinking.

The practical version for anyone building something: the moment before execution is the cheapest one to discover a weak assumption. Every week of work built on an unexamined premise raises the cost of the correction. That is not overhead. It is the only thing that stops a bad premise from compounding.