A Shia cultural organisation was the target but the Afghan Voice news agency was also hit. So-called Islamic State said it was behind the attack.
The interior ministry told the BBC an explosion at the Shia centre was followed by at least two more blasts.
IS has been behind a number of attacks on Shia targets across the country in recent months.
Students were among those who had gathered at the Shia centre for a discussion forum.
The interior ministry said the event was to mark the 38th anniversary of the Soviet invasion of Afghanistan.
The initial explosion was followed by at least two others, although the ministry said these did not cause any fatalities.
The health ministry’s latest figures say 41 people died and 84 were hurt, with women and children among the casualties.
Student Mohammad Hasan Rezayee told Tolo News: “We were inside the hall in the second row when an explosion from behind took place. After the blast there was fire and smoke inside the building and everyone was pleading for help.”
Another witness, Sayed Jan, told reporters from his hospital bed: “There was a book reading event and academic discussion, and I was one of the participants. During the speech a huge bang was heard and smoke rose from inside the hall.
“My face was burning. I fell down from the chair and I saw the other colleagues around me on the ground. The smoke was everywhere.”
Sayed Abbas Hussaini, a journalist at Afghan Voice, told Reuters that one reporter at the agency had been killed and two wounded.
Distraught relatives gathered at local hospitals, which are treating the dozens of wounded people. (Courtesy BBC)