"Seven hundred churches in Kigali? Are these boreholes that give people water?" asked President Paul Kagame when he heard that more than 700 churches had been closed down by authorities.
"I don't think we have as many boreholes. Do we even have as many factories? But 700 churches, which you even had to close? This has been a mess!"
The closure this week by the Rwanda Governance Board (RGB) was meant to tighten rules on registration and functioning of churches in the country in the face of rising cases of fraud and security concerns.
Religious and rights groups said the closure amounts to infringement on the right to worship, but the government says the crackdown is timely, in the face of thousands of mushrooming churches in the largely Christian country.
President Kagame, while officiating at the closure of a four-day national leadership retreat on Thursday, commented on the ongoing operation saying that he was surprised by the large number of churches. He pointed out that had there been proper planning, the situation would not have got to a level where the government has to close churches.
He said that Rwanda has not reached a level where it needs all these churches, noting that such a big number of churches is suitable in bigger and developed economies that have the means and systems to sustain them -which is one of the two scenarios such a development can be explained.
"The second scenario is that you will find such a mess [of churches] in societies which have nothing like ours for different reasons. In Rwanda and Africa, there are those who want to see us in such chaos. When authorities intervene and stop them, they lament that it is a human rights abuse. People should have a right to worship in whatever church, they say," said Mr Kagame.
He however said that Rwandans do not have the luxury and means to sustain such churches, supporting the move to shut them down.
The heads of Pentecostal churches which are the most affected have lamented the decision to close churches, which they said was hastily implemented.
"We needed more time to put things in order and later an inspection would determine which churches to close," said Bishop Liliane Mukabadege of Mountain of Hope, pointing out that some of the closed churches can meet the standards given a grace period.
Observers say that crackdown could set back planned investments which faith-based organisations were making including in the media, schools and hospitals.
It is suspected that a recent case involving Amazing Grace FM, a Christian-run radio station accused of airing a hateful sermon against women, put churches in the spotlight. In the sermon aired on January 29, a pastor Nicolas Niyibikora vented against women calling them "evil" and "against God's plan".
The radio has since been temporarily closed and fined Rwf2 million ($2,320) for undermining state security and Rwandan culture.
RGB, which registers faith-based and civil society organisations, says there are loopholes in the current law which were deemed not strict enough to address the issues that emerged after its enactment in 2013.
Anastase Shyaka, RGB chief executive officer, said the law allowed churches to start and register later while preachers underwent no licensing process as there are no specific requirements regarding who should practice, standards for places of worship and management, among other things.
"The same way other professions require some training or qualifications-it should also apply to preachers to avoid people who call themselves bishops, pastors or apostles when they have not acquired it through training," said Mr Shyaka.
RGB officials say this was partly to blame for the several malpractices and internal wrangles that characterised churches and faith-based organisations in the country.