如何做搜尋引擎蜘蛛日誌分析
搜尋引擎蜘蛛日誌檔案是一種非常強大但未被站長充分利用的檔案，分析它可以獲取有關每個搜尋引擎如何爬取網站內容的相關資訊點，及檢視搜尋引擎蜘蛛在一段時間內的行為。
| IP地址(10) | 伺服器名稱 | 所屬國家 |
|---|---|---|
| 52.206.180.182 | ec2-52-206-180-182.compute-1.amazonaws.com | US |
| 54.152.233.61 | ec2-54-152-233-61.compute-1.amazonaws.com | US |
| 54.198.164.91 | ec2-54-198-164-91.compute-1.amazonaws.com | US |
| 54.163.99.156 | ec2-54-163-99-156.compute-1.amazonaws.com | US |
| 54.159.248.23 | ec2-54-159-248-23.compute-1.amazonaws.com | US |
| 52.91.58.248 | ec2-52-91-58-248.compute-1.amazonaws.com | US |
| 52.91.238.228 | ec2-52-91-238-228.compute-1.amazonaws.com | US |
| 54.87.155.146 | ec2-54-87-155-146.compute-1.amazonaws.com | US |
| 54.198.150.250 | ec2-54-198-150-250.compute-1.amazonaws.com | US |
| 54.89.144.148 | ec2-54-89-144-148.compute-1.amazonaws.com | US |
| IP地址(7) | 伺服器名稱 | 所屬國家 |
|---|---|---|
| 52.90.158.71 | ec2-52-90-158-71.compute-1.amazonaws.com | US |
| 52.91.170.37 | ec2-52-91-170-37.compute-1.amazonaws.com | US |
| 54.87.218.245 | ec2-54-87-218-245.compute-1.amazonaws.com | US |
| 54.242.157.130 | ec2-54-242-157-130.compute-1.amazonaws.com | US |
| 54.209.132.41 | ec2-54-209-132-41.compute-1.amazonaws.com | US |
| 52.91.160.3 | ec2-52-91-160-3.compute-1.amazonaws.com | US |
| 54.172.242.103 | ec2-54-172-242-103.compute-1.amazonaws.com | US |
| 52.207.224.181 | ec2-52-207-224-181.compute-1.amazonaws.com | US |
| 52.90.128.129 | ec2-52-90-128-129.compute-1.amazonaws.com | US |
| 52.87.245.25 | ec2-52-87-245-25.compute-1.amazonaws.com | US |
| 52.207.225.219 | ec2-52-207-225-219.compute-1.amazonaws.com | US |
| 54.174.102.105 | ec2-54-174-102-105.compute-1.amazonaws.com | US |
| 54.209.164.133 | ec2-54-209-164-133.compute-1.amazonaws.com | US |
| 54.86.94.178 | ec2-54-86-94-178.compute-1.amazonaws.com | US |
| 52.207.227.245 | ec2-52-207-227-245.compute-1.amazonaws.com | US |
| 52.23.156.89 | ec2-52-23-156-89.compute-1.amazonaws.com | US |
| IP地址(7) | 伺服器名稱 | 所屬國家 |
|---|---|---|
| 52.207.225.219 | ec2-52-207-225-219.compute-1.amazonaws.com | US |
| 54.174.102.105 | ec2-54-174-102-105.compute-1.amazonaws.com | US |
| 54.209.164.133 | ec2-54-209-164-133.compute-1.amazonaws.com | US |
| 54.86.94.178 | ec2-54-86-94-178.compute-1.amazonaws.com | US |
| 52.207.227.245 | ec2-52-207-227-245.compute-1.amazonaws.com | US |
| 52.23.156.89 | ec2-52-23-156-89.compute-1.amazonaws.com | US |
| 52.90.128.129 | ec2-52-90-128-129.compute-1.amazonaws.com | US |
| IP地址(3) | 伺服器名稱 | 所屬國家 |
|---|---|---|
| 52.91.189.245 | ec2-52-91-189-245.compute-1.amazonaws.com | US |
| 54.173.191.178 | ec2-54-173-191-178.compute-1.amazonaws.com | US |
| 54.85.105.140 | ec2-54-85-105-140.compute-1.amazonaws.com | US |
一般不需要攔截，尤其是如果你自己也受益於搜尋引擎優化服務。不過，如果你擔心伺服器資源佔用等問題，且您都不使用這些工具，當然也可以選擇攔截它們。
您可以通過在網站的 robots.txt 中設定使用者代理訪問規則來遮蔽 SalesIntelligent 或限制其訪問許可權。我們建議安裝 Spider Analyser 外掛，以檢查它是否真正遵循這些規則。
# robots.txt # 下列程式碼一般情況可以攔截該代理 User-agent: SalesIntelligent Disallow: /
您無需手動執行此操作，可通過我們的 Wordpress 外掛 Spider Analyser 來攔截不必要的蜘蛛或者爬蟲。
