Sahih Bukhari Volume 2, Book 25, Hadith 1585

حَدَّثَنَا مُسَدَّدٌ، حَدَّثَنَا أَبُو الأَحْوَصِ، حَدَّثَنَا أَشْعَثُ، عَنِ الأَسْوَدِ بْنِ يَزِيدَ، عَنْ عَائِشَةَ ـ رضى الله عنها ـ قَالَتْ سَأَلْتُ النَّبِيَّ صلى الله عليه وسلم عَنِ الْجَدْرِ أَمِنَ الْبَيْتِ هُوَ قَالَ ‏"‏ نَعَمْ ‏"‏‏.‏ قُلْتُ فَمَا لَهُمْ لَمْ يُدْخِلُوهُ فِي الْبَيْتِ قَالَ ‏"‏ إِنَّ قَوْمَكِ قَصَّرَتْ بِهِمُ النَّفَقَةُ ‏"‏‏.‏ قُلْتُ فَمَا شَأْنُ بَابِهِ مُرْتَفِعًا قَالَ ‏"‏ فَعَلَ ذَلِكِ قَوْمُكِ لِيُدْخِلُوا مَنْ شَاءُوا وَيَمْنَعُوا مَنْ شَاءُوا، وَلَوْلاَ أَنَّ قَوْمَكِ حَدِيثٌ عَهْدُهُمْ بِالْجَاهِلِيَّةِ فَأَخَافُ أَنْ تُنْكِرَ قُلُوبُهُمْ أَنْ أُدْخِلَ الْجَدْرَ فِي الْبَيْتِ وَأَنْ أُلْصِقَ بَابَهُ بِالأَرْضِ ‏"

Narrated By 'Aisha : I asked the Prophet whether the round wall (near Ka'ba) was part of the Ka'ba. The Prophet replied in the affirmative. I further said, "What is wrong with them, why have they not included it in the building of the Ka'ba?" He said, "Don't you see that your people (Quraish) ran short of money (so they could not include it inside the building of Ka'ba)?" I asked, "What about its gate? Why is it so high?" He replied, "Your people did this so as to admit into it whomever they liked and prevent whomever they liked. Were your people not close to the Pre-Islamic Period of ignorance (i.e. they have recently embraced Islam) and were I not afraid that they would dislike it, surely I would have included the (area of the) wall inside the building of the Ka'ba and I would have lowered its gate to the level of the ground."