请问如何禁谷歌和必应的蜘蛛搜索引擎
必应每天都有几千个爬行记录，就是老爬个人资料robots.txt我这样写对么，大佬帮纠正一下
#
# robots.txt for Discuz! X3
#
User-agent: *
Allow: /data/attachment/
Allow: /data/cache/
Allow: /uc_server/data/avatar/
Disallow: /api/
Disallow: /data/
Disallow: /source/
Disallow: /install/
Disallow: /template/default/
Disallow: /config/
Disallow: /uc_client/
Disallow: /uc_server/
Disallow: /admin.php
Disallow: /search.php
Disallow: /member.php
Disallow: /api.php
Disallow: /misc.php
Disallow: /connect.php
Disallow: /forum.php?mod=redirect*
Disallow: /forum.php?mod=post*
Disallow: /*?mod=misc*
Disallow: /*?mod=attachment*
Disallow: /*mobile=yes*
Disallow: */home
Disallow: */home.php?mod=spacecp*
Disallow: */home.php?mod=profile*
User-agent: Googlebot
Disallow: /
User-agent: bingbot
Disallow: /
User-agent: manbot
Disallow: /
我知道答案 回答被采纳将会获得1 贡献 已有0人回答
页:
[1]
