You may notice some locations appear to be on fire on the Empire Map. This happens when the Barbarian presence there starts rebuilding their destroyed huts. Be sure to visit these locations and destroy any wandering Barbarian armies you may find to put a stop to their rebuilding efforts!