The purported message claims one hostage has been killed and demands a prisoner exchange for the other.
The Associated Press could not verify the contents of the message, which varied greatly from previous videos released by the Islamic State group, which now holds a third of both Syria and Iraq.
Chief Cabinet Secretary Yoshihide Suga said Cabinet ministers were holding an emergency meeting about the new message.
Prime Minister Shinzo Abe told reporters as he rushed into his office that the release of the new message was “an outrageous and unforgivable act. We demand their immediate release.”
Kyodo News agency said the same video has been emailed to the wife of one of the hostages.
The Islamic State group had threatened on Tuesday to behead the hostages within 72 hours unless it received a $200 million ransom.
Japan has scrambled for a way to secure the release of 47-year-old Kenji Goto, a journalist, and Haruna Yukawa, a 42-year-old adventurer fascinated by war. Japanese diplomats had left Syria as the civil war there escalated, adding to the difficulty of contacting the militants holding the hostages.
Goto’s mother made an appeal for his rescue.
“Time is running out. Please, Japanese government, save my son’s life,” said Junko Ishido. “My son is not an enemy of the Islamic State.”
Ishido said she was astonished and angered to learn from her daughter-in-law that Goto had left for Syria less than two weeks after his child was born in October to try to rescue Yukawa.
Japanese officials have not directly said whether they are considering paying any ransom. Japan has joined other major industrial nations in the Group of Seven in opposing ransom payments. U.S. and British officials said they advised against paying.