At least 11 people have also been injured, several of them critically.
The eruption sent rocks raining down over a kilometre-wide area near Kusatsu in central Japan, say local media.
Mount Kusatsu-Shirane, 150km (93 miles) north-west of Tokyo, erupted on Tuesday but officials are still investigating whether it caused the avalanche.
Images of the volcano showed a vent on its side appeared to have blown out, while video footage shows a large cloud of black ash and rock being thrown down the mountainside.
Japan’s Meteorological Agency has warned people not to climb the mountain, saying further eruptions cannot be ruled out.
According to state broadcaster NHK, some of the volcanic debris has been found more than 1km (0.6 miles) from the mountain.
Several of the injured people were hit by falling rocks.
One eyewitness at the ski resort told NHK: “There was this huge boom, and a big plume of totally black smoke rose up. I had absolutely no idea what had happened.”
Another eyewitness described seeing “black smoke rose from the top of the mountain”.
“We were told to evacuate inside 30 minutes later,” the skier said.
Among the injured were four people who were riding a ropeway gondola at the time. Images showed the windows of the gondola had shattered.
At least 78 skiers were still trapped at a rest house on top of the mountain by Tuesday afternoon, according to a local town official.
The man who died was one of six members of the military taking part in an exercise in the area, said Defence Minister Itsunori Onodera.
The other five were rescued, but some of them were injured, he said.
The Fire and Disaster Management Agency have warned that rocks could yet rain down as far as 2km from the peak.
Japan is one of the world’s most seismically active nations and closely monitors its volcanoes. (Courtesy BBC)