# /robots.txt file for http://www.peterpap.com User-agent: * Disallow: /ews Disallow: /netsuitecatalogconnector Disallow: /p7pm Disallow: /psd Disallow: /dev Disallow: /v3flashslideshow Disallow: /_notes/ Disallow: /_baks/ Disallow: /MMWIP/ Disallow: /SitemapGenerator/ Sitemap: http://www.xxxx.com/sitemap.xml # You can use wildcards * # Also the $ anchors the end of a file else a= would mean a= AND a=123 # so a=$ ONLY means the url END with a= and not value # http://help.yahoo.com/l/us/yahoo/search/webcrawler/slurp-02.html # http://www.google.com/support/webmasters/bin/topic.py?topic=8843 # http://search.msn.com/docs/siteowner.aspx?t=SEARCH_WEBMASTER_REF_RestrictAccessToSite.htm