Not all survivors are ready to revenge after you raid them, so it is possible that you do not get raided back. 
 
To summon the raiders, your character must be at level 150.