In a statement, the Palace said “recollections may vary” but the matters will be addressed privately.
Meghan told Oprah Harry had been asked by an unnamed family member “how dark” their son Archie’s skin might be.
The Palace said the Sussexes would “always be much loved family members”.
The response from Buckingham Palace came after crisis meetings involving senior royals.
The Palace had been under growing pressure to respond to the interview in which Meghan – the first mixed-race member of the modern Royal Family – said that questions had been asked about their son’s skin colour.
Prince Harry later clarified to Oprah that the comments were not made by either the Queen or the Duke of Edinburgh.
In other developments, Piers Morgan has left ITV’s Good Morning Britain show following a row over comments he made about the Duchess of Sussex.
The statement from the Palace, which came a day and a half after the interview was first broadcast in the US, said: “The whole family is saddened to learn the full extent of how challenging the last few years have been for Harry and Meghan.
“The issues raised, particularly that of race, are concerning. Whilst some recollections may vary, they are taken very seriously and will be addressed by the family privately.
“Harry, Meghan and Archie will always be much loved family members.”
It is understood the royals wanted to carefully consider their response and to give the British public an opportunity to watch the interview first when it was broadcast on Monday evening.
The royals are said to consider this a family matter and to believe they should be given the opportunity to discuss the issues privately.