Acetic Acid (Vinegar) can alter the structure of clay pots if soaked too long. Initially, acids will liberate minerals such as magnesium and calcium that cause unsightly scale. If allowed to continue soaking, acids will break down the iron and aluminum silicates that form clay.
Thoroughly rinse clay pots after soaking in Vinegar. Don't soak any longer than necessary.