Taliban fighters stand guard at the entrance gate of Panjshir Province. AFP
The Panjshir Valley is famed for being a site of resistance by Afghans against Soviet forces in the 1980s and as a base for rebels opposed to Taliban rule during the first stint in power in the late 1990s.
The National Resistance Front (NRF) were the last to hold out against the Taliban's takeover of the country last year by retreating to the valley.
Headed by the son of late anti-Taliban commander Ahmad Shah Massoud, NRF forces last week announced an offensive against the Taliban — their first since the hardliners seized power in August.
Afghans wave a flag above the portrait of late commander Ahmad Shah Massoud in Panjshir province. File/AFP
Both sides claim to have killed dozens of each other's fighters in recent days.
"We could only pick up one or two items of clothing," Lutfullah Bari told the media, saying he fled with dozens of families.
"Like us... (the families) are now living with their relatives in different areas of Kabul," he added.
Farid Ahmad, a father-of-10, said he left his district with several other people because of fighting.
Another civilian, Aimal Rahimi, said people "are afraid and escaping to save their lives".
Afghan resistance movement and anti-Taliban uprising forces personnel stand guard in Panjshir province. AFP
Taliban commanders in Panjshir however told AFP the fighting had stopped.
"They (NRF fighters) have escaped to the mountains," said Abdul Hamid Khurasani, head of the Taliban's elite Badri unit in Panjshir.
"The situation is now normal and peaceful."
The NRF said their offensive would continue across 12 provinces where their forces had a presence, mostly in the north of Afghanistan.
Massoud, the group's most revered figure and known as the "Lion of Panjshir", was assassinated in 2001 by Al-Qaeda, two days before the September 11 attacks in the United States.
His son, Ahmad Massoud, has since picked up the mantle against Taliban forces, denouncing the militant regime as "illegitimate".