Generally speaking, those are only npcs central to main quests. Unfortunately, there are only two ways to find out without trying to kill them.
1. If on PC, use the isessential console command. A returned value of 1 means they are.
2. Apparently, if you have the master pickpocket perk "perfect touch," you still won't be able to steal their equipped gear (you'll get an alert to that affect, I think).
3. If you have completed a quest involving an npc, there's a decent chance you can kill them now. Essential status is usually removed when they are no longer needed to keep quest lines operational. At least, that's how it was in Oblivion--I imagine they would have kept it for Skyrim.
4. Look to see if they are on this
list. Link contains spoilers.
Otherwise, I'm pretty sure essential npcs always accept a yield.