A reputation score loses value when almost everyone receives the same high rating. The system still functions, but the result no longer separates users effectively.
Filippas, Horton and Golden tracked one online labour market over its data window and found the share of workers receiving a perfect rating climbed from roughly 33 percent to roughly 85 percent in about six years.¹ Average feedback rose steadily. The work did not. What changed was the social cost of an honest low rating, which on most platforms is paid by the person giving it.
Source: Filippas, Horton and Golden (2022), Reputation Inflation, Marketing Science 41(4). Only the two endpoints are published; the path between them is not drawn here.
Why ratings increase over time
Review order is one important cause. On many platforms one party submits first and the other sees that rating before responding. The second assessment is therefore no longer fully independent.
Fradkin, Grewal and Holtz tested this on Airbnb by hiding both reviews until both parties had responded.² Retaliation and reciprocation declined and average ratings fell. The authors did not find a measurable effect on who was ultimately booked. The design made ratings more independent without demonstrably improving the market outcome.
Source: Fradkin, Grewal and Holtz (2021), Journal of Marketing Research.
Fraud sits on top of that. Luca and Zervas documented systematic review fraud on Yelp and showed it concentrates where competitive pressure is highest.³ Inflation and fraud are different problems with the same result: once nearly everyone is excellent, the score stops separating anyone.
Earlier research on online reputation systems described the basic purpose as creating enough information for strangers to transact, while also warning about manipulation and the difficulty of eliciting honest feedback.⁴ Experimental work by Bolton, Greiner and Ockenfels later showed that design choices can improve trust and efficiency, but that reputation information changes participant behaviour rather than merely reporting it.⁵
When five out of six users receive a top rating, the score makes little distinction between them. This follows from how ratings are collected, not from a technical failure.
The components of Signal Score
Signal Score contains seven components. Each is scored from 0 to 100 and assigned a published weight.
| Component | Weight | What it counts |
|---|---|---|
| Projects | 25% | Projects you own, with extra for launched ones |
| Ratings | 20% | Average score others gave you |
| Collaboration | 15% | Share of your collaborations that reached completion |
| Profile | 10% | Completeness: headline, bio, location, focus, skills |
| Engagement | 10% | Current activity streak |
| Referrals | 10% | People you referred who became active |
| Responsiveness | 10% | Share of your workspaces where you actually posted |
Limitations in the current formula
The current Signal Score also uses measures that do not directly establish the quality of collaboration.
The heaviest single input, at 25 percent, is a count of projects you created yourself. Nobody has to agree that they were worth creating. It is the same shape of proxy as a follower count, and it is the largest term in the formula.
Engagement, at 10 percent, is a login streak. Twenty-five consecutive days scores full marks. That measures showing up, which is not nothing and is not quality either.
Both are there for a defensible reason, which is that they are the only inputs available before anyone has collaborated. That reason expires the moment real collaboration data exists, and the weights should move when it does.
What a Signal Score is actually made of today
Forenta is in public beta, which determines which data is currently available.
There is no rating interface yet, so the 20 percent for ratings has nothing to average. No code path marks a collaboration as completed, so the 15 percent for collaboration is structurally zero for everyone, not just for new accounts. Between them that is 35 percent of the formula sitting at zero for reasons that have nothing to do with the person being scored.
A Signal Score today therefore consists mainly of profile completeness, an activity streak and referrals. Components for which no data exists should be read as not established. Such a score is not simply low. Its evidence base is also incomplete.
What a paid plan does not buy
The effect of a paid plan can be verified directly in the product logic.
A plan writes a multiplier next to your score, 1.5 for Pro and 2.0 for Max. That multiplier is stored and it is never applied. The weighted total is computed from the seven components and nothing else touches it. Your plan does not move your Signal Score.
Where a plan does have an effect is the separate match score used when Forge suggests people, and during the beta nobody can buy Pro or Max anyway. Both are announced for after the beta, not purchasable today.
What this does not cover
This describes the design and the current state, not a proven outcome. Nothing here shows that Signal Score predicts good collaboration, because there is not yet enough completed collaboration to test it against. The weights are a starting position, and a starting position chosen before the data exists is a hypothesis.
The disputed cases are also unsolved. What a collaboration that ends by mutual agreement should do to the score, or what happens when two people disagree about whether something was finished, has no answer in the code yet.
The full component breakdown lives on the Signal page. If you want to argue with the weights, that is the place to look, and arguing with them is reasonable.
Does paying for Pro or Max raise my Signal Score?
No. The plan multiplier is stored next to the score and never enters the calculation. It affects a separate match score used when Forge suggests people, and paid plans cannot be bought during the public beta in any case.
Why is my score low even though I have real experience elsewhere?
Because nothing is imported. A score that could be seeded with outside credentials would be gameable from the first day. The trade-off is real: experienced people start at the bottom alongside everyone else.
Which parts of the score work right now?
Profile completeness, activity streak, referrals and projects. Ratings and collaboration completion are not yet produced by any part of the product, so those two components are zero for everyone.
References
- 1.Filippas, A., Horton, J. J., & Golden, J. M. (2022). Reputation Inflation. Marketing Science, 41(4), 733–745.
- 2.Fradkin, A., Grewal, E., & Holtz, D. (2021). Reciprocity and Unveiling in Two-Sided Reputation Systems: Evidence from an Experiment on Airbnb. Marketing Science, 40(6), 1013–1029.
- 3.Luca, M., & Zervas, G. (2016). Fake it till you make it: Reputation, competition, and Yelp review fraud. Management Science, 62(12), 3412–3427.
- 4.Resnick, P., Kuwabara, K., Zeckhauser, R., & Friedman, E. (2000). Reputation systems. Communications of the ACM, 43(12), 45-48.
- 5.Bolton, G., Greiner, B., & Ockenfels, A. (2013). Engineering Trust: Reciprocity in the Production of Reputation Information. Management Science, 59(2), 265-285.