This article is from the CD-Recordable FAQ, by Andy McFadden (firstname.lastname@example.org) with numerous contributions by others.
The TOC (Table Of Contents) identifies the start position and length of the
tracks on a disc. The TOC is present on all CDs. If it weren't, the disc
would be unreadable on a CD player or CD-ROM drive. CD recorders write the
TOC as part of "finalizing the disc. (Section (2-19) has some more details
about finalizing discs.)
A "directory" is a list of files. If you're a Mac user, you're probably
used to the term "folder". It's part of a filesystem, such as the ISO-9660
or HFS filesystem present on most CD-ROMs. Audio tracks don't have files,
so they don't have directories either.
There's nothing stopping you from writing a FAT16 or Linux ext2 filesystem
directly onto a CD-ROM. Whether or not you can read such a disc is a
different matter. (The Linux "mount" command should allow you to mount
just about anything read-only, but Windows may not be so willing.) The CD
specification defines the TOC, and there are well-defined standards for
certain filesystems, but [AFAIK] nothing in the CD spec requires that you
fill a data track with a certain kind of data.