This article is from the Ozone Depletion FAQ, by Robert Parson firstname.lastname@example.org with numerous contributions by others.
This does happen to some extent - it's called "self-healing" - and
has the effect of moving ozone from the upper to the lower
stratosphere. Recall that ozone is _created_ by UV with wavelengths
less than 240 nm, but functions by _absorbing_ UV with wavelengths
greater than 240 nm. The peak of the ozone absorption band is at ~250
nm, and the cross-section falls off at shorter wavelengths. The O2
and O3 absorption bands do overlap, though, and UV radiation between
200 and 240 nm has a good chance of being absorbed by _either_ O2 or
O3. [Rowland and Molina 1975] (Below 200 nm the O2 absorption
cross-section increases dramatically, and O3 absorption is
insignificant in comparison.) Since there is some overlap, a decrease
in ozone does lead to a small increase in absorption by O2. This is a
weak feedback, however, and it does not compensate for the ozone
destroyed. Negative feedback need not imply stability, just as
positive feedback need not imply instability.
Numerical calculations of ozone depletion take the "self-healing"
phenomenon into account, by letting the perturbed ozone layer come
into equilibrium with the exciting radiation.