The two lenders have confirmed the development which follows speculations on the likelihood of the merger.
NIC Bank, whose full name is National Industrial Credit Bank has presence in Kenya, Uganda and Tanzania.
According to a letter signed by the chairpersons of the boards of the two banks, the two entities could end up in a merger after the discussions.
The merger could create one of the largest financial services group in the region with experts saying that it could place them favourably for growth in banking and wealth management.
A joint statement by both firms noted that merging would put their customer base at about 38 million across the region.
"It is the view of the two Boards that a potential merger would bring together the best in class retail and corporate banks with strong potential for growth in all aspects of banking and wealth management. A combined entity would create a complementary base of over 38 million customers, a strong digital proposition and a robust corporate and asset finance offering," the statement read in part.
The move also comes amidst a development by regulators across the region requiring banks to raise their respective capital.
In Rwanda, the Central Bank will require local to raise their minimum capital from the current Rwf 5b to Rwf 20b according to a new draft licensing regulation for the sector.
This presents commercial banks a task to raise the capital fourfold which most admit is quite an uphill task with a transition period has been set at 5 years.
Other countries in the region have also raised the minimum capital requirements by between 200 per cent and 150 per cent.
The two banks are still tight-lipped on the terms of the deliberations but maintained that it will not affect current operations of the two banks.
"The Boards of Directors wish to emphasize that the discussions are preliminary in nature and as such only limited information can be disclosed at this stage. The two entities are committed to the highest standards of governance and regulatory disclosure and will update the market as and when there are any material developments," the statement read further.
CBA entered the Rwandan market in 2017 after acquiring the now defunct Crane Bank Rwanda.
It is believed that a merger could put CBA Rwanda in a better position as it could expose them to more capital as well as expertise to further penetrate the market.