This article is from the Car Audio FAQ, by Ian D. Bjorhovde (firstname.lastname@example.org) with numerous contributions by others.
Magnet *size* is meaningless!
Every speaker will have an optimal BL ((see Section 4.1),) product,
the field strength in the air gap multiplied by the length of the voice
coil wire in the field.
If the BL product is too low, the speaker is electrically not very well
damped (which will result in a woofer with a high Qts). A bump in
frequency response and a level drop in midband efficiency may be the
result. If the BL product is too high, the speaker is electrically
overdamped (Low Qts woofer). A very high midband efficiency, but the
driver starts to roll of early.
An high BL product can be achieved in a number of ways: increase field
strength; or increase wire length in magnetic gap.
The increase in field strength is limited; so some manufacturers use
very thin wire for the voice coil, as such they can achieve a high BL
product with a low field strength (cheap magnet). Or they use an 8 layer
voice coil... needless to say that electrical power handling will
Long stroke woofers, having only a part of the voice coil in the air
gap, need a very high field strength to achieve a high BL product. Often
this means a big magnet as well...
Use magnet size as an indication, but as nothing more than that.