User-agent: * Allow: / Disallow: /ad* Disallow: /xj Disallow: /*.asp$ Disallow: /*.doc$ Disallow: /*.pdf$ Disallow: /*.gq$