Are you including a site map page. Spiders can’t index pages that can’t be crawled. A site map will help spiders find all the important pages on your site, and help the spider understand your site’s hierarchy.
Shared publicly