This article is from the Amiga FAQ, by Ignaz Kellerer with numerous contributions by others.
Many people would prefer to use a 3.5" instead of the smaller 2.5" drives as the former are much cheaper. This is possible, but you need a special cable to connect the drive to your IDE controller. Additionally you should put some isolating material between the drive and the main board. Some people report termic problems but I did not notice anything.
Some dealers offer a set which contains the cable, isolating material and for about 20$. See the magazines for more information.
Thomas Schuh (firstname.lastname@example.org)
If you are going to install an IDE-harddisk, regardless if 2.5" or 3.5", you should always consider that the it may not work flawlessly, if the drive and ROM-version have not been tested and approved by Commodore. Regarding the A1200, one should also remember that this Amiga has not been designed to hold 3.5" harddisk drives. The 2.5" drives, which are mainly thought for portable computers, may have some advantages (for example in power consumption, heat emission or shock resistance) that could pay off because of the small keyboard case of the A1200 and the limited power supply. With the larger 3.5" drives, ventilation inside the computer might also be disturbed. If it is necessary to remove some shielding inside the A1200 to make the drive fit inside, even radio interference may occur.
To sum it up: if 3.5" harddisk drives could really be used in the A1200 without any problems, you could be sure Commodore themselves wouldn't have used the slightly more expensive 2.5" drives in the A1200-HD.