Minute concentrations of suspended particles can dramatically alter the behavior of a drying droplet. After a period of isotropic shrinkage, similar to droplets of a pure liquid, these droplets suddenly buckle like an elastic shell. While linear elasticity is able to describe the morphology of the buckled droplets, it fails to predict the onset of buckling. Instead, we find that buckling is coincident with a stress-induced fluid to solid transition in a shell of particles at a droplet's surface, occurring when attractive capillary forces overcome stabilizing electrostatic forces between particles. When replacing water by a mixture of ethanol and water, the buckling threshold changes. For a fixed colloid solid fraction, the buckling threshold evolves as a function of ethanol content, due to changes of the solvent mixture physical properties, such as viscosity and evaporation rate. A simplified model predicting the qualitative behaviour of the buckling threshold as a function of the initial ethanol mass fraction has been developed that fits well experimental results.