In the wake of September 11, 2001 the word Jihad became well known in the non-Muslim world, especially, in western societies. Some politicians and media have used it to conjure up terrifying images of rabid Islamic fundamentalists. The word Jihad come from a religious context, and has a long history and has been used for a range of complex meanings. However these days it tends to be translated “holy war” from its association with the medieval Crusades. Because of this, there is a current debate whether the usage of the term jihad without further explanation refers to military combat. Some apologists and some western scholars insist that jihad is primarily nonviolent. On the other hand, some Islamic fundamentalists turn the confusion over the definition of the term to their advantage. In this article the author examines how the concept of Jihad was developed in the Quran and the Hadiths and how each of the sub-groups of Islam defines its meaning in their historical contexts. The author begins with the hypothesis that the common mistake made in the study of Islam by outsiders, as in the study of other religions, is to treat Islam as a unified religious system. The reality is that all Muslims are not same in their manner of thinking, believing, and living. Historically proto-Sufism began to add nonviolent concepts into the meaning of Jihad in the context of the conversion of ascetics. This became the understanding of Sufism. For them the concept of greater jihad and lesser jihad were used to imply the meaning of inner spiritual discipline. On the other hand the main streams of Islam, Sunni and Shia, do not reject the use of violence in when they performing jihad. In fact they seem to have added additional kinds of jihad, as well as to develop some preconditions for waging holy wars. Though some pacifistic groups, particularly Ahmadiyya, reject the violent aspect of jihad, modern fundamentalists insist that every Muslim should engage in militant jihad and destroy indiscriminately everything which is in opposition to Islam. In conclusion, the author suggests that we should not expect monolithic answers when we pose questions of Muslims. Instead, we should begin to ask what kind of worldview each particular Muslim person and community have. We will see that each group in Islam, including secular Muslims, have quite different perspectives on the meaning and practices of jihad. To serve Muslims effectively, we need to begin with an informed understanding of each of them if we are to establish a respectful and trustful relationship with them.