This is one of the most ill-named parts of SharePoint Portal Server 2003. Here goes:
1. The Sites Directory area in a portal contains links (by default) to site collections created when a person uses the Create Site link in that directory. There is the ability to organize those sites any way you want. You can also add links to other sites in the directory, allowing you to create a one-stop shop for all your links to your information resources.
When creating a site via this method, in order to get that site listed in the Sites Directory, you must ensure that the List This Site in the Site Directory check box is enabled (See Figure 1). If this box is not checked when the site is created or when the link is added to the directory, the site will not be listed in the Sites Directory.
2. The Manage Crawls of Site Directory is a list of sites created when a person uses the Create Site link in the Sites Directory area. This list instructs Search on which sites to crawl. By default, each site created in the Sites Directory is automatically added in this list for crawling because the Include in Search Results check box is selected by default. The Manage Crawls of Site Directory list has three states for each link: Approved, Rejected or Requested. The reason the site is indexed is because the site is listed in this list and is given the Approved state, by default.
Now, you would think that the sites directory content source crawls the sites directory area in a portal. But that would not be correct. The sites directory content source crawls the Approved listings in the Manage Crawls of Site Directory list. The portal content source will crawl the Sites Directory area and the existence of those links, but it will not follow the link to site and crawl the site's content.
So, if you're unable to crawl and index your site's content, ensure that the site is listed in the Mange Crawls for Site Directory list and that it is approved. And please don't confuse the Sites Directory with the Sites Directory Content Source. They are really not related in any real way.
Bill English, Mindsharp