After the battle of Yermuk, when the main Muslim army under Abu Ubaida and Khalid left for the north of Syria, some Muslim contingents under Amr bin Al Ass and Shurahbil remained stationed in the southern sector comprising Jordan and Palestine.
Finding that the bulk of the Muslim army had left, Artabun the Byzantine Governor assembled a large force at Ajnadin in another bid to drive away the Muslims from the soil of Syria. The battle at Ajnadin fought towards the close of 636 was very bloody and gruesome. Both sides fought bravely but ultimately the Byzantines were defeated, Artabun defeated with heavy loss fled to Jerusalem with the remnant of his army.
After the victory of Ajnadin the Muslim forces spread in all directions in Jordan and Palestine. The towns of Sabtah, Gaza, Nablus, Bait-Jibrin and many other towns were captured one after the other. That cleared the way to Jerusalem. The city of Jerusalem sacred to the Jews and the Christians was strongly fortified. It was protected on every side by deep valleys and steep ascents. Military engines were mounted on the walls which were intended for playing havoc with the advancing invader. It was the winter season, and the severity of the winter added to the difficulties of the besieging Muslim force. The siege dragged on and the Byzantines offered very stiff resistance.
Amr b Al Aas the Muslim Commander in the southern sector wrote to Abu Ubaida for reinforcement. By this time, northern Syria had fallen to the Muslims and Abu Ubaida was able to spare many contingents which rushed to the aid of the Muslims fighting in the southern sector. When the citizens of Jerusalem came to know that the besieging Muslim forces had been considerably strengthened they lost heart. Finding further resistance futile the Patriarch of Jerusalem sued for peace. He said that it was written in their holy books that the city would surrender to the man who was the best among the Muslims. He accordingly desired that the Caliph Umar should come to Jerusalem personally to receive the surrender of the city.
Abu Ubaida referred the matter to Umar at Madina. Umar called a meeting of his Consultative Council, and asked for their advice, Othman expressed the view that it was not necessary for the Caliph to go, and that the defeated Byzantines would themselves surrender. Ali said that Jerusalem was as much sacred to the Muslims as the Jews or the Christians, and that in view of the sanctity of the place it was desirable that its surrender should be received by the Caliph personally. Umar decided to accept the advice of Ali.
Leaving Ali as his deputy at Madina, Umar proceeded to Jerusalem. No retinue accompanied the Caliph. Umar was accompanied by one slave only, and between these two persons they had only one camel which they rode turn by turn. As they neared Jabia where the Muslim Commanders were to meet Umar, it was the turn of the slave to ride. The slave wanted Umar to ride the animal, but Umar refused. As they came to Jabia the people saw the strange spectacle of the slave riding the camel and the Caliph walking on foot.
At Jabia the Muslim Commanders met Umar. Abu Ubaida was dressed in coarse garments, and Umar was much pleased to meet him. Yazid b. Abu Sufiyan, Khalid bin Walid and some other commanders were dressed in fine clothes and Umar expressed his displeasure at their gaudy dress. Abu Ubaida explained in detail the situation in Syria. He elaborated how with the grace of God the Muslims had been able to overthrow the mighty Byzantine power in Syria. As Umar saw the green fields, orchards and lofty buildings of Syria he was greatly moved and he recited from the Holy Quran:
"They have left many a garden, fountain, park, arbour, and riches which they used to enjoy. Thus it is that We put another community in possession thereof."
A deputation from Jerusalem waited on Umar at Jabia and a treaty was drawn up. According to the treaty security of life and property were guaranteed to all citizens of Jerusalem. The safety of churches and other religious buildings and places was provided for. The citizens were required to pay Jizya. Any one not agreeable to owe allegiance to the Muslims was given the option to leave the city.
After the treaty had been drawn up, Umar decided to travel to Jerusalem. Again he travelled in a simple way as an ordinary traveller. No guard was suffered to accompany him. He rode on a poor horse, and refused to change it for a better charger.
At the gate of Jerusalem, Umar was greeted by the Patriarch of Jerusalem, the elite of the city and the Muslim Commanders. While those who had come to receive him wore costly dress, Umar was dressed in a garment of coarse cloth ordinarily worn by an average Arab. When some one advised him to wear a better dress befitting the state occasion, Umar turned down the suggestion saying that he derived his strength and status from his faith in Islam, and not from any dress. When the Patriareh of Jerusalem saw the ascetic simplicity of the Caliph of Islam, and then looked to his own costly dress, he said, "Verily Islam has excelled all other religions".
The Partriarch of Jerusalem handed over the keys of the city of Jerusalem to Umar. The Muslims were now the masters of Jerusalem. That was a special divine favour of God to the Muslims. As Umar entered the city he was greeted by the citizens with great enthusiasm. Umar said that he wanted to be led to some place where he could offer thanksgiving prayer to God. He was led to a Church but he refused to pray there, on the ground that that would set a precedent for the Muslims of the following generations to forcibly convert churches into mosques. He was thereafter led to a place where the prophet David used to pray. Here Umar offered special prayers of thanksgiving and all the Muslims joined him. As the Byzantines watched the Muslims at pray, they felt that such people so obedient to God were bound to command. The Patriarch said that he was not sorry for surrendering the city for he had surrendered it to a better people.
Umar stayed in Jerusalem for a few days. He reorganised the administration, and made the necessary arrangements to look after the needs of the citizens. He founded a mosque at an elevated place in the city. This mosque came to be known as Umar's Mosque. On the inaugural occasion Bilal was requested to give the call to prayer as he used to do in the time of the Holy Prophet. After the death of the Holy Prophet, Bilal had ceased to give the 'Adhan.' At the request of Umar he agreed to give the Adhan to mark the foundation of Umar's mosque. As Bilal gave the call to pray in his stentorian voice. Umar and the Muslims wept recalling the days when the Holy Prophet used to be in their midst. As the inspiring words of the Adhan resounded in the hills and dales, the people stood in awe realising that a new era had dawned in Syria.