When you pray at each of the Altars a Shrine Protector will spawn on either end of the Altar. You do not have to fight them to update the quest but if you wish to avoid them you best move the moment you finish praying.