When all the people present somewhere will offer Prayer in one direction by standing in one line step with step it will bring unity among them. in Prayer a Billionaire and a Poor man stands together step by step in one direction. another interesting fact is that Kaabah is in the middle of the world. in the history maps where made south facing upwards and north downwards and that Kaabah used to came in centre of the world. Later on when Westerners drawn the world map they taken north facing upwards and south downwards and even then Kaabah was in the centre of the World.
Another Important point that proves that Muslims don,t worship Kaabah is the saying of Second Caliph of Islam Hazrat Umar R.A about Hajre-Aswad the Stone present in Kaabah :I know that you are a stone and can neither benefit nor harm". The Last Point to prove that Muslims don,t worship the Kaabah is that Many People in history and present gives Adhan by standing on the the Kaabah. My Question is that any person who worships a Idol will ever stand on that Idol? So this point also proves that Muslims don,t worship the Kaabah but only one God.
Sahaba also used to give Adhan by standing on the roof of Kaaba. a person who worships a idol will he like to stand on that Idol? its another proof that Muslims not worships Kaaba.
Doing Idol Worship is one of the major sin in Islam that is also known as Shirk which means associating partners with Allah. May Allah swt save us all from doing the sin of Shirk. We worship none but one God.