More than 75,000 residents in the southern prefectures of Kumamoto and Kagoshima were asked to evacuate following pounding rains overnight.
Television footage showed large areas of Hitoyoshi town in Kumamoto inundated in muddy waters that gushed out from the Kuma River. Many cars were submerged up to their windows.
Mudslides smashed into houses and floodwaters carried trunks from uprooted trees. Several people were standing atop a convenience store as they waited for rescuers.
media reports stated that about 13 people were reported missing. Kumamoto officials say they were still assessing the extent of damage.
Prime Minister Shinzo Abe set up a task force, vowing to do utmost to rescue the missing.
The Japan Meteorological Agency earlier issued warnings of extraordinary rain in parts of Kumamoto, about 1,000 kilometers (600 miles) southwest of Tokyo, but later downgraded them as the rainfall estimated at 100 milimeters (4 inches) per hour subsided.
Kumamoto Gov. Ikuo Kabashima said he requested help from the Japan Self-Defense Forces.