Health is the amount of damage your character can sustain before dying. In other games, this might be called "Hit Points," or HP for short.
There are two important totals concerning your health -- your maximum health and your current health. Your maximum health represents just how much of a punishment you can take, and is often the number players are referring to when they talk about health. How your current health compares to that is usually more important to the solo adventurer or the healer of a party.
The amount of (maximum) health your character has is determined by a few factors, but the most important among them are class, level, and stamina. Some classes naturally have higher health totals, but all classes receive an additional 10 health per point of stamina. This makes stamina a very important attribute in PvP.
The green bar next to your character portrait shows how much of your maximum health you have currently. Mousing over the health bar or opening the character window will display exact numbers, and your interface can be configured to display this automatically. Most healers need to use custom interfaces to see the health of all the members of a raid.