The attack happened in the city of Sagamihara, located west of Tokyo, at a building believed to include services for disabled people.
At least 19 people died in the incident and dozens more were injured after the man – reportedly in his 20s – launched the night time attack.
Japanese media said staff called police at 2.30 a.m. local time on Tuesday with reports of a man armed with a knife on the grounds of the Tsukui Yamayuri Garden facility.
The suspect reportedly went into a police station and said, "I did it" before being arrested.
Police officials later confirmed that the suspect had told them he was a former worker of the Tsukui Yamayuri Garden facility.
The man arrested over a deadly knife attack in Japan is a former employee of the targeted facility, according to unconfirmed reports.