"Players have resolved not to play the match any longer as NFF officials are making plans to fly the team back home," said NFF director of communications, Ademola Olajire.
A formal complaint by Nigeria has also been sent to the Confederation of African Football (CAF).
The Nigerian team have been held at an abandoned airport in Libya for more than 15 hours since their chartered flight landed Sunday.
The Nigerian embassy in Libya reportedly cannot intervene as they would need permission to do so from the Libyan government.
Captain William Troost-Ekong said the team will not play the game and they will not make a three-hour bus ride from Al-Abraq to Benina, where the qualifier was scheduled to be played on Tuesday, for safety reasons.
"As the team captain together with the team we have decided that we will NOT play this game," he posted on social media.
"Let them have the points.
"We will not accept to travel anywhere by road here even with security it's not safe. We can only imagine what the hotel or food would be like given to us if we continued."
Former African footballer of the year Victor Ikpeba, who accompanied the team to Libya, has urged for stiff sanctions against Libya and supported the decision to boycott the match.
"If CAF know their job, Libya ought to be banned from international football," he told AFP.
"This is a high-risk country and one really wonders who approved for Libya to be playing their games at home.
"I played for the Super Eagles for 10 years and I never experienced what I have in Libya in the last hours."
"The team are not safe, those of us travelling with them are also not safe," he added.
"We were locked up in an abandoned airport for more than 10 hours like hostages.
"The team will call off their bluff and not play this match."
Soon after, Troost-Ekong gave his followers another update, suggesting that fuel in the airport needed for the plane was deliberately withheld to delay his side.
The Libyan Football Federation have "firmly rejected" all accusations of wrongdoing as they blamed it on a "misunderstanding".
In a statement on X, they explained: "We firmly reject any claims that suggest foul play or sabotage in this situation.
"It is important to highlight that just last week, our own national team faced significant challenges upon our arrival in Nigeria for the third match of the Africa Cup of Nations qualification."
On Friday, Nigeria beat Libya 1-0 in the southern city of Uyo.
They lead Group D with seven points from three matches, while Libya are bottom of the standings on one point.
