Japanese police said the men were found at Yurihonjo city’s marina late on Thursday, and taken into custody.
The men had told authorities that their boat experienced difficulties and they ended up drifting into Japanese waters.
Japan regularly sees North Korean fishing boats straying into its territory, and its coastguard has occasionally had to rescue fishermen.
But sometimes the boats – often simple or rickety vessels – are found with bodies onboard, indicating that the vessels had been drifting for some time.
In the latest incident, Japanese authorities said the men were able to walk unaided. A Korean interpreter has been brought in to help with obtaining details of their story, reported broadcaster NHK.
Hachiro Okonogo, chairman of the National Public Safety Commission, told reporters that the men said they drifted over to Yurihonjo, in the north east of Japan, “after their ship experienced troubles”.
It is not known whether the men intend to stay in Japan or return to North Korea.
Defections by boat to Japan are not common given the distance, though it has happened before – in 2011, nine North Koreans tried to sail to South Korea but ended up across the sea in Japan.
Over the summer North Korea fired two missiles over Japan, angering Tokyo. This month also saw the high profile defection of a North Korean soldier to South Korea across the Demilitarised Zone. (Courtesy BBC)