9.3. Tuning Your System

Sometime in the latter half of the testing phase, the reputation system begins to operate sufficiently well enough to gauge the general effect it will have on the application and early indications of its likely success against the original goals. This is when the ongoing process of reputation model tuning can begin.

9.3.1. Tuning for ROI: Metrics

The most important thing the reputation team can do when implementing and deploying a reputation system is to define the key metrics for success. What are the numerical measures that a reputation-enabled application is contributing to the goals set out for it? For each measure, what are the target values? Is there a predicted lift in user contributions? Should there be more page-views? Is there an expected direct effect on product revenues? Is the model expected to decrease customer care costs? By how much? All of these metrics allow both the application and reputation teams to identify areas for tuning. Sometimes just the application will need to be adjusted, other times just the model will, and (especially early on) sometimes it will all need tuning.