This article is from the RSC/UKSC Cricket FAQ, by Ganesh with numerous contributions by others.
Basically if he has number X on his cap, it means he is
the Xth player to play for his country in that form of
Dianne van Dulken:
[Steve Waugh] had the brilliant idea a few years back
of getting it on their caps, so they are in touch with
their place in history, etc, etc.
I notice that both India and England have followed suit.