There is no good way to set this up through the Admin UI, and no way to truly randomize the evaluations chosen for an evaluator.
About all all I can think of would be to create two policies for each evaluator and give them a different pool of agents for each policy, then set the metered number to one on each policy. That will ensure the evaluator gets two different agents to evaluate each period, but it does not guarantee they won't get the same two agents each period.
------------------------------
George Ganahl
Principal Program Manager
Genesys
------------------------------