This article is from the Bicycles FAQ, by Mike Iglesias with numerous contributions by others.
Since posture is the problem, it is also the solution. Those of us who
suffer from low back pain need to be constantly vigilant. We need to
maintain some arch in our backs as much as possible.
Sitting is a particular problem. Most chairs, coaches, car seats, etc
provide little low back support. You can buy low-back support pads at some
drug stores. Try them before you buy them because they are not all
comfortable. Alternatively, you can fold a towel and put it behind your
low back. The key is to maintain some arch without being uncomfortable.
Position on the bike is also important. Get your bike fit checked at a
shop that you trust. You should also work on maintaining a flat back
when riding. One way to achieve this is to push your belly button toward
the top tube.