Rwamagana District authorities have ordered the closure of 75 churches over failure to meet minimum standards.
The decision was announced on Monday during a meeting between district leaders and representatives of religious organisations operating in the district.
The district has 427 churches, among which only 352 have operating licenses, according to officials.
"We will not allow people to worship God in thatched structures (nyakatsi)," Radjab Mbonyumuvunyi, the district mayor, told the religious leaders.
"Today no one in Rwanda lives in a thatched house anymore, why should our people worship God from such structures?" he posed.
The decision follows a survey carried out recently by the district, which indicated that some churches operated in makeshift tents, commercial houses, and homes.
None of those will be called a church anymore, the mayor said.
The move is part of a broader crackdown on places of worship that fall short of minimum requirements, including on quality of the buildings and noise pollution.
The effort is being spearheaded by local governments in concert with the Rwanda Governance Board (RGB), which oversees operations of faith-based organisations and civil society.
Over the last couple of weeks authorities closed down up to 714 churches in Kigali alone, with some asked to upgrade before they can reopen.
Mayor Mbonyumuvunyi said only churches with licenses will be allowed to operate.
Churches with construction permits were also given two months to complete the works.
All the churches were also asked to pave their compounds or make them green by planting lawn, trees and flowers.
Participants in the meeting acknowledged there is disorder in the way some churches are set up and promised to work on their weaknesses.
"We do not like such disorder either. We also believe that clerics must be qualified in theology because even Jesus' disciples spent three years with Him learning," said Bishop Constantin Gasore, of Restoration Church Rwamagana.
"Places of worship have now become like business, otherwise what explains the mushrooming churches every other day?... are we all not preaching the gospel of Jesus Christ?" said Gasore.
Among other things, every church is required to ensure hygiene, be soundproof, and put in place rainwater harvesting system to prevent running water from damaging homes.
"How can you have both men and women share a toilet?" the Rwamagana mayor posed.
He said the decision was taken in the interest of the people.
Mbonyumuvunyi advised the religious leaders to always reflect on their role before opening churches, and told those whose churches were suspended that it is not the end of the road.
"Go back and look at where you fell short and do the necessary," he advised them.