The hosts began day four 118-2, needing another 122 to win, but captain Dean Elgar hit a brilliant 96 not out to lead his side to a seven-wicket win.
The opener batted for more than five hours on a difficult pitch while Rassie van der Dussen made an important 40.
The third and final Test begins on Tuesday in Cape Town.
Having taken a number of blows while batting late on day three, Elgar resumed on 46 after the first two sessions of the fourth day were wiped out by rain.
He continued to battle in gloomy conditions which suited India’s seam bowlers and put on 82 with Van der Dussen.
Van der Dussen edged Mohammed Shami to first slip for 40 with 65 runs still needed but Temba Bavuma made 23 not out, having been dropped on nought one-handed by Shardul Thakur in his follow-through.
India, the number one ranked Test team in the world, were led by KL Rahul in the absence of injured captain Virat Kohli.
Fittingly Elgar hit the winning runs with former South Africa captain and current director of cricket Graeme Smith describing it as one of the side’s “greatest Test wins”.
“It wasn’t easy,” said Elgar, who was appointed captain of the struggling Proteas in March 2021.
“There were some tough conditions we had to face as a batting unit.
“I think some amber nectar is going to sooth those bruises.
“I like to think the knocks I take make me extra motivated to perform. Some will call it stupid or some will call it brave.” (Courtesy BBC)