Every time you collect a skull there is a chance (about 25%) to spawn Nymox the Deceiver, who will appear and attack you so be ready for it. The skulls are found on altars along the way. There is at least one at the bottom and one at the top of Riverfell. The skulls respawn quickly.
NOTE: Additional completions of this quest reward 25 Plainarite instead of a Corrupted Sourceshard.