Level depends on class, but try grinding the 38-40 elite ogres in Alterac once you can if you still need silk. As a 42 priest I *can,* but it's a bit too hard (usually end up at 25-45% health and 75% mana while in Shadowform). I've a 36 warrior friend who could take one down and still have a good 80% of his health. They usually drop 4-6 silver and 2-4 pieces of silk each, not to mention some decent exp, and there's enough quests (as Horde, at least) concerning them (The Crown of Will series) that you'll probably kill them off quite a few times and get a few stacks of silk even if grouped.
Prettymuch every humanoid in the upper 30's range seems to drop silk at least fairly often... up until level 42 I've probably looted no less than 800 pieces of it (I've used up a good 25 stacks just making silk packs).