This article is from the PGP FAQ, by Jeff Licquia firstname.lastname@example.org with numerous contributions by others.
No, not unless they have also stolen your secret pass phrase, or if
your pass phrase is susceptible to a brute-force attack. Neither part
is useful without the other. You should, however, revoke that key and
generate a fresh key pair using a different pass phrase. Before
revoking your old key, you might want to add another user ID that
states what your new key id is so that others can know of your new