This item is from the Yet Another Enhanced IDE/Fast-ATA/ATA-2 FAQ, by John Wehman and Peter den Haan with numerous contributions by others. (v1.92).
Nothing, in principle. A secondary IDE port has been reserved in the PC I/O map for ages (base address 0170h, IRQ 15), and adapters that could be configured as secondary have been available for quite some time, even though BIOS support was lacking. So while it is a part of EIDE, there is actually nothing new about this feature, except that the possibility of connecting tape drives and CD-ROMs to the ATA adapter has transformed four device support from a luxury into a necessity.
Actually, there is another reason to provide a secondary port for ATAPI devices. There are a number of advanced hardware features for harddisk interfaces, such as prefetch buffers and write behind, that may get in the way of ATAPI compatibility. This means that if the software drivers of an intelligent ATA-2 port are not ATAPI-aware, or simply don't work as they should, you may run into sticky problems. Especially if you have an ATAPI CD-ROM that doesn't support PIO mode 3 transfers, a secondary port that provides only basic ATA features is a good way to avoid a lot of headaches.
Finally, an ATAPI device on the primary port will cause Windows FastDisk drivers that aren't ATAPI aware to fail.
Nothing prevents you from defining more ports like the primary and the secondary one; in addition to these two, the tertiary and quaternary one are semi-standard. See section 10.4 for the port and IRQ assignments of all these ports.