There is a common misconception about the definition of atheism. Many perceive an atheist as someone who believes that God does not exist (i.e. there's no God.) -- which is not necessarily true.
Let's consider the term theist first. A theist is someone who believes in the existence of God. He believes that there's a God. If you think of this particular belief (there's a God) as a metaphysical entity, then this entity exists in the mind of a theist. While in an atheist's mind that belief simply does not exist. Note that this does not necessarily mean that an atheist believes that there's no God. A belief that there's no God is, another metaphysical entity, which might or might not exist in an atheist's mind.
There are two possible opposites of belief (1) disbelief, and (2) absence of belief. The first one is active denial. While the second one is a mere passive position. Normally when one hears the term atheist, they think about the 1st position (i.e. disbelief in God). Position (1) is called active (or strong) atheism, while (2) is referred as non-theism. I think it's fair to lump (1) and (2) together and tag both positions as atheism. But it's important not to forget that an atheist can belong to either (1) active atheism, or (2) non-theism. Active theist affirms the non-existence of God, while a non-theist rejects theism.
So while theism (and hence atheism) is about belief. Agnosticism, on the other hand, is about knowledge. A person who knows for sure that God exists is a gnostic. And a person who doesn't claim to know whether God exists or not is an agnostic.
Contrary to common understanding, a person can be both: a theist and an agnostic, a believer without claiming to know for sure if God exists or not. Similarly you can be an atheist as well as an agnostic. In fact, being an agnostic can be a reason why someone is also an atheist (i.e. he lacks the belief, because he's not sure.)
On religious subjects, the only world religion that's firmly agnostic - Buddhism - is of Indian origin. A particular school of thought in Buddhism, called Theravada, a predominant religion in Sri Lanka, is actually non-theist. In Hinduism too, the Carvaka philosophy of skepticism and materialism (also known as Lokayata), which originated in the 6th century, is classified as a nastika (i.e. atheist) system. Jainism also rejects the beliefs in a personal creator God.
Amartya Sen has explored the heterodoxy of Indian religious beliefs in his fascinating book The Argumentative Indian. I take the following passage from his book: The so-called 'song of creation' (or the 'creation hymn', as it is sometimes called) in the authoritative Vedas ends with the following radical doubts:
Who really knows?
Who will here proclaim it?
Whence was it produced?
Whence is this creation?
The gods came afterwards, with the creation of this universe.
Who then knows whence it has arisen?
Whence this creation has arisen -
perhaps it formed itself, or perhaps it did not -
the one who looks down to it, in the highest heaven,
only he knows -
or perhaps he does not know.
[From Rigveda. English translation by Wendy Doniger O'Flaherty, in Rigveda: An Anthology.]
