This article is from the alt.usage.english FAQ, by Mark Israel firstname.lastname@example.org with numerous contributions by others.
Those who round their lips when they say it would probably
transcribe it /dOg/; those who don't round their lips, /dAg/.
Very few people in North America distinguish all three vowels
/A/, /A./, and /O/. Speakers in Eastern and Southern U.S. merge
/A./ and /A/, so that "bother" and "father" rhyme. Speakers in
Western U.S. and in Canada merge /A./ and /O/, so that "cot" and
"caught", "Don" and "Dawn" are pronounced alike. Some speakers
merge all three vowels. The Oxford Companion to the English
Language says: "The merger of vowels in "tot" and "taught" begins
in a narrow band in central Pennsylvania and spreads north and
south to influence the West, where the merger is universal. [...]
In New England, where the merger is beginning to occur, speakers
select the first vowel; in the Midland and West, the second vowel
is used for both." Although /A./ is seldom used to transcribe
American pronunciation, the vowel transcribed /O/ may sound like
/A./ to non-American speakers, or it may sound like /O/.
There is a further complication with "dog": U.S. dictionaries
give the pronunciations /dOg/, /dAg/ in that order (and similarly
with some other words ending in "-og", although which ones varies
from dictionary to dictionary). "Dawg", the name of the family dog
in the comic strip "Hi and Lois", may be intended to convey the
pronunciation /dOg/ to (or from) people who usually pronounce the
word /dAg/; or it may be intended as how a child in a community
where /A./ and /O/ are merged might misspell "dog".