Commit89f78bd

authored

添加nginx防止爬虫爬取配置 (Chanzhaoyu#1187)

* 添加nginx防止爬虫爬取配置* Update nginx.conf* Update README.md

1 parentd598dc6 commit89f78bdCopy full SHA for 89f78bd

File tree

+23

-0

lines changed

+23

-0

lines changed

Lines changed: 16 additions & 0 deletions

Original file line number	Diff line number	Diff line change
`@@ -24,6 +24,7 @@`
`24`	`24`	`- [Docker 参数示例](#docker-参数示例)`
`25`	`25`	`- [Docker build \& Run](#docker-build--run)`
`26`	`26`	`- [Docker compose](#docker-compose)`
	`27`	`+- [防止爬虫抓取](#防止爬虫抓取)`
`27`	`28`	`- [使用 Railway 部署](#使用-railway-部署)`
`28`	`29`	`- [Railway 环境变量](#railway-环境变量)`
`29`	`30`	`- [手动打包](#手动打包)`
`@@ -234,6 +235,21 @@ services:`
`234`	`235`	```
`235`	`236`	-`OPENAI_API_BASE_URL` 可选，设置 `OPENAI_API_KEY` 时可用
`236`	`237`	-`OPENAI_API_MODEL`可选，设置 `OPENAI_API_KEY` 时可用
	`238`	`+`
	`239`	`+#### 防止爬虫抓取`
	`240`	`+`
	`241`	`+nginx`
	`242`	`+`
	`243`	+将下面配置填入nginx配置文件中，可以参考 `docker-compose/nginx/nginx.conf` 文件中添加反爬虫的方法
	`244`	`+`
	`245`	+```
	`246`	`+# 防止爬虫抓取`
	`247`	`+if ($http_user_agent ~* "360Spider\|JikeSpider\|Spider\|spider\|bot\|Bot\|2345Explorer\|curl\|wget\|webZIP\|qihoobot\|Baiduspider\|Googlebot\|Googlebot-Mobile\|Googlebot-Image\|Mediapartners-Google\|Adsbot-Google\|Feedfetcher-Google\|Yahoo! Slurp\|Yahoo! Slurp China\|YoudaoBot\|Sosospider\|Sogou spider\|Sogou web spider\|MSNBot\|ia_archiver\|Tomato Bot\|NSPlayer\|bingbot")`
	`248`	`+{`
	`249`	`+ return 403;`
	`250`	`+}`
	`251`	+```
	`252`	`+`
`237`	`253`	`### 使用 Railway 部署`
`238`	`254`
`239`	`255`	`[![Deploy on Railway](https://railway.app/button.svg)](https://railway.app/new/template/yytmgc)`

Lines changed: 7 additions & 0 deletions

Original file line number	Diff line number	Diff line change
`@@ -3,6 +3,13 @@ server {`
`3`	`3`	`server_name localhost;`
`4`	`4`	`charset utf-8;`
`5`	`5`	`error_page500502503504 /50x.html;`
	`6`	`+`
	`7`	`+# 防止爬虫抓取`
	`8`	`+if($http_user_agent~*"360Spider\|JikeSpider\|Spider\|spider\|bot\|Bot\|2345Explorer\|curl\|wget\|webZIP\|qihoobot\|Baiduspider\|Googlebot\|Googlebot-Mobile\|Googlebot-Image\|Mediapartners-Google\|Adsbot-Google\|Feedfetcher-Google\|Yahoo! Slurp\|Yahoo! Slurp China\|YoudaoBot\|Sosospider\|Sogou spider\|Sogou web spider\|MSNBot\|ia_archiver\|Tomato Bot\|NSPlayer\|bingbot")`
	`9`	`+{`
	`10`	`+return403;`
	`11`	`+}`
	`12`	`+`
`6`	`13`	`location /{`
`7`	`14`	`root /usr/share/nginx/html;`
`8`	`15`	`try_files$uri /index.html;`

Comments

(0)