You probably entered a location after its timer expired, to leave the zone you first need to move to any other location on the map, and only then from there to your base.