
如何做搜尋引擎蜘蛛日誌分析
搜尋引擎蜘蛛日誌檔案是一種非常強大但未被站長充分利用的檔案,分析它可以獲取有關每個搜尋引擎如何爬取網站內容的相關資訊點,及檢視搜尋引擎蜘蛛在一段時間內的行為。
IP地址(235) | 伺服器名稱 | 所屬國家 |
---|---|---|
66.102.8.25 | google-proxy-66-102-8-25.google.com | US |
66.102.8.27 | google-proxy-66-102-8-27.google.com | US |
66.102.8.29 | google-proxy-66-102-8-29.google.com | US |
66.102.8.105 | google-proxy-66-102-8-105.google.com | US |
66.102.8.107 | google-proxy-66-102-8-107.google.com | US |
66.102.8.103 | google-proxy-66-102-8-103.google.com | US |
66.102.6.62 | google-proxy-66-102-6-62.google.com | US |
64.233.172.186 | google-proxy-64-233-172-186.google.com | US |
64.233.172.188 | google-proxy-64-233-172-188.google.com | US |
64.233.172.190 | google-proxy-64-233-172-190.google.com | US |
IP地址(93) | 伺服器名稱 | 所屬國家 |
---|---|---|
66.249.93.149 | google-proxy-66-249-93-149.google.com | US |
66.249.93.138 | google-proxy-66-249-93-138.google.com | US |
66.249.93.140 | google-proxy-66-249-93-140.google.com | US |
66.249.93.142 | google-proxy-66-249-93-142.google.com | US |
66.249.93.148 | google-proxy-66-249-93-148.google.com | US |
66.249.93.150 | google-proxy-66-249-93-150.google.com | US |
66.249.93.75 | google-proxy-66-249-93-75.google.com | BE |
66.249.93.77 | google-proxy-66-249-93-77.google.com | BE |
66.249.93.79 | google-proxy-66-249-93-79.google.com | BE |
66.249.93.132 | google-proxy-66-249-93-132.google.com | US |
IP地址(149) | 伺服器名稱 | 所屬國家 |
---|---|---|
66.249.88.128 | google-proxy-66-249-88-128.google.com | US |
66.249.93.142 | google-proxy-66-249-93-142.google.com | US |
66.249.93.140 | google-proxy-66-249-93-140.google.com | US |
66.249.93.146 | google-proxy-66-249-93-146.google.com | US |
66.249.93.147 | google-proxy-66-249-93-147.google.com | US |
66.102.8.205 | google-proxy-66-102-8-205.google.com | US |
66.102.8.207 | google-proxy-66-102-8-207.google.com | US |
66.102.8.209 | google-proxy-66-102-8-209.google.com | US |
66.249.93.138 | google-proxy-66-249-93-138.google.com | US |
66.249.93.150 | google-proxy-66-249-93-150.google.com | US |
IP地址(33) | 伺服器名稱 | 所屬國家 |
---|---|---|
66.249.93.179 | google-proxy-66-249-93-179.google.com | US |
66.249.93.181 | google-proxy-66-249-93-181.google.com | US |
66.249.93.177 | google-proxy-66-249-93-177.google.com | US |
66.249.93.97 | google-proxy-66-249-93-97.google.com | KR |
66.249.93.127 | google-proxy-66-249-93-127.google.com | US |
66.249.93.123 | google-proxy-66-249-93-123.google.com | US |
66.249.93.67 | google-proxy-66-249-93-67.google.com | US |
66.249.93.120 | google-proxy-66-249-93-120.google.com | US |
66.249.93.116 | google-proxy-66-249-93-116.google.com | US |
66.102.8.193 | google-proxy-66-102-8-193.google.com | US |
IP地址(7) | 伺服器名稱 | 所屬國家 |
---|---|---|
66.249.93.251 | google-proxy-66-249-93-251.google.com | US |
66.102.6.145 | google-proxy-66-102-6-145.google.com | US |
66.102.6.163 | google-proxy-66-102-6-163.google.com | US |
66.249.93.245 | google-proxy-66-249-93-245.google.com | US |
66.249.93.248 | google-proxy-66-249-93-248.google.com | US |
IP地址(2) | 伺服器名稱 | 所屬國家 |
---|---|---|
159.223.192.50 | 159.223.192.50 | US |
66.249.82.90 | google-proxy-66-249-82-90.google.com | TW |
66.249.93.103 | google-proxy-66-249-93-103.google.com | US |
66.249.93.105 | google-proxy-66-249-93-105.google.com | KR |
66.249.82.89 | google-proxy-66-249-82-89.google.com | TW |
66.102.8.36 | google-proxy-66-102-8-36.google.com | US |
66.249.93.100 | google-proxy-66-249-93-100.google.com | US |
64.233.172.103 | google-proxy-64-233-172-103.google.com | US |
66.249.93.89 | google-proxy-66-249-93-89.google.com | US |
66.249.93.91 | google-proxy-66-249-93-91.google.com | US |
66.249.93.93 | google-proxy-66-249-93-93.google.com | BE |
66.102.8.25 | google-proxy-66-102-8-25.google.com | US |
66.102.8.27 | google-proxy-66-102-8-27.google.com | US |
66.102.8.29 | google-proxy-66-102-8-29.google.com | US |
66.102.8.105 | google-proxy-66-102-8-105.google.com | US |
66.102.8.107 | google-proxy-66-102-8-107.google.com | US |
66.102.8.103 | google-proxy-66-102-8-103.google.com | US |
66.102.6.62 | google-proxy-66-102-6-62.google.com | US |
64.233.172.186 | google-proxy-64-233-172-186.google.com | US |
64.233.172.188 | google-proxy-64-233-172-188.google.com | US |
64.233.172.190 | google-proxy-64-233-172-190.google.com | US |
66.249.93.149 | google-proxy-66-249-93-149.google.com | US |
66.249.93.138 | google-proxy-66-249-93-138.google.com | US |
66.249.93.140 | google-proxy-66-249-93-140.google.com | US |
66.249.93.142 | google-proxy-66-249-93-142.google.com | US |
66.249.93.148 | google-proxy-66-249-93-148.google.com | US |
66.249.93.150 | google-proxy-66-249-93-150.google.com | US |
66.249.93.75 | google-proxy-66-249-93-75.google.com | BE |
66.249.93.77 | google-proxy-66-249-93-77.google.com | BE |
66.249.93.79 | google-proxy-66-249-93-79.google.com | BE |
66.249.93.132 | google-proxy-66-249-93-132.google.com | US |
66.249.88.128 | google-proxy-66-249-88-128.google.com | US |
66.249.93.146 | google-proxy-66-249-93-146.google.com | US |
66.249.93.147 | google-proxy-66-249-93-147.google.com | US |
66.102.8.207 | google-proxy-66-102-8-207.google.com | US |
66.102.8.209 | google-proxy-66-102-8-209.google.com | US |
66.102.8.205 | google-proxy-66-102-8-205.google.com | US |
66.249.93.179 | google-proxy-66-249-93-179.google.com | US |
66.249.93.181 | google-proxy-66-249-93-181.google.com | US |
66.249.93.177 | google-proxy-66-249-93-177.google.com | US |
66.249.93.67 | google-proxy-66-249-93-67.google.com | US |
66.249.93.97 | google-proxy-66-249-93-97.google.com | US |
66.249.93.123 | google-proxy-66-249-93-123.google.com | US |
66.249.93.127 | google-proxy-66-249-93-127.google.com | US |
66.249.93.120 | google-proxy-66-249-93-120.google.com | US |
66.249.93.116 | google-proxy-66-249-93-116.google.com | US |
66.102.8.193 | google-proxy-66-102-8-193.google.com | US |
66.102.6.145 | google-proxy-66-102-6-145.google.com | US |
66.102.6.163 | google-proxy-66-102-6-163.google.com | US |
66.249.93.245 | google-proxy-66-249-93-245.google.com | US |
66.249.93.248 | google-proxy-66-249-93-248.google.com | US |
66.249.93.251 | google-proxy-66-249-93-251.google.com | US |
66.249.93.193 | google-proxy-66-249-93-193.google.com | US |
66.249.82.96 | google-proxy-66-249-82-96.google.com | US |
78.153.140.223 | hostglobal.plus | GB |
IP地址(2) | 伺服器名稱 | 所屬國家 |
---|---|---|
66.249.93.193 | google-proxy-66-249-93-193.google.com | US |
66.249.82.96 | google-proxy-66-249-82-96.google.com | US |
一般不攔截。此類爬蟲通常由諸如Pingdom, Gtmetrix和PageSpeed Insights等網站測速工具發起,且一般為網站所有者所為。
您可以通過在網站的 robots.txt 中設定使用者代理訪問規則來遮蔽 Google Page Speed Insights 或限制其訪問許可權。我們建議安裝 Spider Analyser 外掛,以檢查它是否真正遵循這些規則。
# robots.txt # 下列程式碼一般情況可以攔截該代理 User-agent: Google Page Speed Insights Disallow: /
您無需手動執行此操作,可通過我們的 Wordpress 外掛 Spider Analyser 來攔截不必要的蜘蛛或者爬蟲。
您可以通過谷歌的線上工具獲取 PageSpeed 得分並使用 PageSpeed 建議提高您網站的載入速度。
當您使用Google Page Speed Insights執行任何網站速度測試時,就會通過網路爬蟲“Google Page Speed Insights”對您的網站進行模擬訪問測試,並根據PageSpeed Insights標準進行評分。
如果您希望您的網站獲得較好的PageSpeed Insights的評分,可以參考我們寫的教學。