: Annuminas would have been pretty safe, however, being far removed from most of the fighting. It may be that the people simply followed the armies. Or perhaps Fornost was a fortress until Amlaith settled there, and then he just drew in as many people as he could.

While Annuminas was a fair distance from the 'front lines' (probably extending from the upper North Downs to the confluence of the Greyflood, depending on the exact point in time), it was undefended from any attack originating in the north. The armies of Carn-Dum could have easily marched north of the North Downs (skirting Fornost), through the wastelands of northern Eriador, and southwest to Annuminas. Assuming the Witch-king had enough sense to post a rearguard to the east to prevent an attack by Arthedain, such a move could have been made virtually uninhibited.

Also, there do not seem to have been any walls or forts between the North Downs and the Emyn Uial. And, since the Dunedain where certainly capable of building large-scale fortifications (as evidenced in the Hadrian-like wall built south of the East-West Road to defend Cardolan), I can only assume that they had no reason to do so. This would suggest to me that by the time Angmar became a threat (ca. TA 1356), the area was of no great military value, or contained nothing worth defending.

For what its worth, the Encyclopedia of Arda lists the following in a footnote to the entry for Annuminas:

At the Council of Elrond, Elrond recounts some of the history of Annúminas: '...after the war and slaughter of the Gladden Fields the Men of Westernesse were diminished, and their city of Annúminas beside Lake Evendim fell into ruin; and the heirs of Valandil removed and dwelt at Fornost...' This suggests an early abandonment, perhaps even by Valandil's immediate successor, Eldacar. The earliest date possible, then, is 249 (Third Age), the year of Eldacar's accession, though the phrase 'heirs of Valandil' suggests that Annúminas was abandoned somewhat later than this.

It seems reasonable to assume that Annúminas was abandoned some centuries later, at the time of the break-up of Arnor in 861 (Third Age), but Tolkien nowhere states this explicitly.