He also vowed to create jobs in a country where some estimates say 90% of people are unemployed.
“We want to grow our economy, we want peace, we want jobs, jobs, jobs,” he told a cheering crowd in Harare.
Mr Mnangagwa, who fled to South Africa two weeks ago, is to be made the new president on Friday, state TV said.
His dismissal led the ruling party and the military to intervene and force an end to Mr Mugabe’s 37-year long rule.
He told supporters at the headquarters of the ruling Zanu-PF party that he had been the subject of several assassination plots and thanked the army for running the “process” of removing Mr Mugabe peacefully.
The news that 93-year-old Mr Mugabe was stepping down sparked wild celebrations across the country late into Tuesday night.
It came in the form of a letter read out in parliament on Tuesday, abruptly halting impeachment proceedings against him.
In it, Mr Mugabe said he was resigning to allow a smooth and peaceful transfer of power, and that his decision was voluntary.
A spokesman for the ruling Zanu-PF party said Mr Mnangagwa, 71, would serve the remainder of Mr Mugabe’s term until elections that are due to be held by September 2018.
Nicknamed the “crocodile” because of his political cunning, Mr Mnangagwa met South African President Jacob Zuma before leaving for Zimbabwe.
Mr Mnangagwa’s firing by Mr Mugabe two weeks ago triggered an unprecedented political crisis in the country.
It had been seen by many as an attempt to clear the way for Grace Mugabe to succeed her husband as leader and riled the military leadership, which stepped in and put Mr Mugabe under house arrest.
Under the constitution, the role of successor would normally go to a serving vice-president, and one still remains in post – Phelekezela Mphoko.
However, Mr Mphoko – a key ally of Mrs Mugabe – has just been fired by Zanu-PF and is not believed to be in the country. In his absence, the party has nominated Mr Mnangagwa, the speaker of parliament confirmed.
Some have questioned whether the handover to Mr Mnangagwa will bring about real change in the country.
He was national security chief at a time when thousands of civilians died in post-independence conflict in the 1980s, though he denies having blood on his hands.
Opposition leader Morgan Tsvangirai told the BBC he hoped that Zimbabwe was on a “new trajectory” that would include free and fair elections.
He said Mr Mugabe should be allowed to “go and rest for his last days”.
Prominent opposition politician David Coltart tweeted: “We have removed a tyrant but not yet a tyranny.”
African Union president Alpha Condé said he was “truly delighted” by the news, but expressed regret at the way Mr Mugabe’s rule had ended.
“It is a shame that he is leaving through the back door and that he is forsaken by the parliament,” he said.
At 93, Mr Mugabe was – until his resignation – the world’s oldest leader. He once proclaimed that “only God” could remove him.
Lawmakers from the ruling party and opposition roared with glee when his resignation letter was read aloud in parliament on Wednesday. (Courtesy BBC)