My Uncle Jack's response to my blog post on military rating systems:
I believe you captured perfectly the essence of the rating systems for enlisted and officer members of the military. The goals of the troops in the trenches are tacitly accepted but seldom stated. Unfortunately, all the parties involved have different opinions of what the goals really are. The ratings are therefore essentially based on "feelings," the supervisor's perceived needs, personal bias, etc, etc and isolated events, good or bad. I think this applies from the President- Joint Chiefs level on down.
Civilian organizations, at least the ones I've been in, don't usually have such clear-cut systems for rating performance but involve high-minded processes that require a development of "goals," which one commits to. This is followed by events and direction from above that ignore the agreed-upon goals and substitute instead the urgent problems at hand. This is also known as fighting fires. The flaw is that the agreed goals are usually crisply defined, while fire-fighting accomplishments are amorphous and hard to define or measure. The rated party is supposedly empowered to invoke his goals statement as a defense against fire fighting but this doesn't usually work and may even be dangerous to one's tenure. At the end of the rating period the system breaks down into the same personal bias as the military system.