The group’s mission in Washington DC expressed its “shock, dismay and outrage” and said the Trump administration misunderstood Africans.
The US leader made the alleged remark in a Thursday meeting on immigration.
But Mr Trump has denied using the language reported.
He has been backed by two Republicans who were at the White House meeting, but Democratic Senator Dick Durbin said Mr Trump called African countries “shitholes” several times and used “racist” language.
On Friday, Mr Trump on Friday tweeted that his language he used at the private meeting with lawmakers to discuss immigration legislation had been “tough”.
But he added that the words attributed to him were “not the language used”.
African Union said the “remarks dishonour the celebrated American creed and respect for diversity and human dignity”.
It added: “While expressing our shock, dismay and outrage, the African Union strongly believes that there is a huge misunderstanding of the African continent and its people by the current Administration.
“There is a serious need for dialogue between the US Administration and the African countries.”
The pan-African grouping represents 55 member states throughout the continent. It succeeded the Organisation of African Unity – which originated in the decolonisation struggles of the early 1960s – in 2002.
The remarks were allegedly made when lawmakers visited him on Thursday to discuss a bipartisan proposal that would impose new restrictions on immigration but protect the so-called “Dreamers” – hundreds of thousands of immigrants brought to the US illegally as children – from deportation.
Mr Trump was said to have told them that instead of granting temporary residency to citizens of countries hit by natural disasters, war or epidemics, the US should instead be taking in migrants from countries like Norway. (Courtesy BBC)