If a challenge is refused, the declining player must retire one character nominated by his opponent among those that could have accepted the challenge, of course!). The retiring character is moved to a position in the unit where he is not in base contact with any enemy and replaced with a rank-and-file trooper. If this is not possible, as there are no positions in the unit where the character can avoid being in base contact with an enemy, the challenge cannot be refused and must be met.

A character that has been retired after refusing a challenge may not fight or do anything else that turn and loses all of his advantages (see the rules for characters that are not in the front rank of a unit). The (so-called) hero has chosen to hide away behind his fellows rather than face the challenger one-on-one. The retired character is automatically returned to a fighting rank at the end of that close combat phase, ready to fight in the following turn. He is positioned in the same place as before.

Sometimes a single character, possibly mounted on a huge monster or a chariot, will attack a unit of troops. If the single character finds himself challenged, he cannot refuse as he has no formation to hide behind.