A win for the South African senior men's national team, who beat Benin at Moses Mabhida Stadium a few days ago, would have seen them take a commanding lead in the group.
But as fate would have it, the home side scored two goals inside the opening half an hour and despite a late rally by coach Hugo Broos' charges, the pressure yielded nothing.
Rwanda took the lead through Innocent Nshuti in the 12th minute with a low shot that took the visitors' goalkeeper Ronwen Williams by surprise.
After the goal, the home side looked more galvanized and continued to apply pressure, and created a number of half chances.
Gilbert Mugisha got the second for Rwanda in the 28th minute when the striker took advantage of confusion between Khuliso Mudau and Williams to slot home.
Bafana Bafana applied pressure towards the end but the hosts' rearguard stood firm to secure a famous victory.