René's URL Explorer Experiment

Title: GitHub - comant/CrawlerTutorial: 爬虫入门、爬虫进阶、高级爬虫 · GitHub

Open Graph Title: GitHub - comant/CrawlerTutorial: 爬虫入门、爬虫进阶、高级爬虫

X Title: GitHub - comant/CrawlerTutorial: 爬虫入门、爬虫进阶、高级爬虫

Description: 爬虫入门、爬虫进阶、高级爬虫. Contribute to comant/CrawlerTutorial development by creating an account on GitHub.

Open Graph Description: 爬虫入门、爬虫进阶、高级爬虫. Contribute to comant/CrawlerTutorial development by creating an account on GitHub.

X Description: 爬虫入门、爬虫进阶、高级爬虫. Contribute to comant/CrawlerTutorial development by creating an account on GitHub.

Opengraph URL: https://github.com/comant/CrawlerTutorial

X: @github

direct link

Domain: github.com

route-pattern	/:user_id/:repository
route-controller	files
route-action	disambiguate
fetch-nonce	v2:8d6b03dc-f8e8-b060-3566-62e00896e20b
current-catalog-service-hash	f3abb0cc802f3d7b95fc8762b94bdcb13bf39634c40c357301c4aa1d67a256fb
request-id	CB8C:13BA54:AA3FAFD:AD82A0A:6A50A8EC
html-safe-nonce	738e7f97b5dcd87cd59904ff7cd06ff66d1b08452d5f38aa557b2752cdbfc0ed
visitor-payload	eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJDQjhDOjEzQkE1NDpBQTNGQUZEOkFEODJBMEE6NkE1MEE4RUMiLCJ2aXNpdG9yX2lkIjoiNjQyODAzNDg2MjI4NDg0MzI0NCIsInJlZ2lvbl9lZGdlIjoic2VhIiwicmVnaW9uX3JlbmRlciI6InNlYSJ9
visitor-hmac	d1dde6fbef9201b485798c7fae0845feb73f0371e665a8dbf93d1ca4a2cf3cf9
hovercard-subject-tag	repository:1198597397
github-keyboard-shortcuts	repository,copilot
google-site-verification	Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-url	https://collector.github.com/github/collect
analytics-location	//
fb:app_id	1401488693436528
apple-itunes-app	app-id=1477376905, app-argument=https://github.com/comant/CrawlerTutorial
twitter:image	https://opengraph.githubassets.com/157112a098852ff21cb9189395e2507b660b468d4db3be55d72fbfbc9f035992/comant/CrawlerTutorial
twitter:card	summary_large_image
og:image	https://opengraph.githubassets.com/157112a098852ff21cb9189395e2507b660b468d4db3be55d72fbfbc9f035992/comant/CrawlerTutorial
og:image:alt	爬虫入门、爬虫进阶、高级爬虫. Contribute to comant/CrawlerTutorial development by creating an account on GitHub.
og:image:width	1200
og:image:height	600
og:site_name	GitHub
og:type	object
hostname	github.com
expected-hostname	github.com
None	d6dc8294eb500fa36bbc57472d61fe87c522f9c3c1d64f70f4926f66a66a7efb
turbo-cache-control	no-cache
go-import	github.com/comant/CrawlerTutorial git https://github.com/comant/CrawlerTutorial.git
octolytics-dimension-user_id	3064483
octolytics-dimension-user_login	comant
octolytics-dimension-repository_id	1198597397
octolytics-dimension-repository_nwo	comant/CrawlerTutorial
octolytics-dimension-repository_public	true
octolytics-dimension-repository_is_fork	true
octolytics-dimension-repository_parent_id	776617940
octolytics-dimension-repository_parent_nwo	NanmiCoder/CrawlerTutorial
octolytics-dimension-repository_network_root_id	776617940
octolytics-dimension-repository_network_root_nwo	NanmiCoder/CrawlerTutorial
turbo-body-classes	logged-out env-production page-responsive
disable-turbo	false
browser-stats-url	https://api.github.com/_private/browser/stats
browser-errors-url	https://api.github.com/_private/browser/errors
release	7ac0ad2f2c7e4b9056617355fd9e33e22b0c8df9
ui-target	full
theme-color	#1e2327
color-scheme	light dark

Links:

Skip to content	https://github.com/comant/CrawlerTutorial#start-of-content
	https://github.com/
Sign in	https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fcomant%2FCrawlerTutorial
GitHub CopilotWrite better code with AI	https://github.com/features/copilot
GitHub Copilot appDirect agents from issue to merge	https://github.com/features/ai/github-app
MCP RegistryNewIntegrate external tools	https://github.com/mcp
ActionsAutomate any workflow	https://github.com/features/actions
CodespacesInstant dev environments	https://github.com/features/codespaces
IssuesPlan and track work	https://github.com/features/issues
Code ReviewManage code changes	https://github.com/features/code-review
GitHub Advanced SecurityFind and fix vulnerabilities	https://github.com/security/advanced-security
Code securitySecure your code as you build	https://github.com/security/advanced-security/code-security
Secret protectionStop leaks before they start	https://github.com/security/advanced-security/secret-protection
Why GitHub	https://github.com/why-github
Documentation	https://docs.github.com
Blog	https://github.blog
Changelog	https://github.blog/changelog
Marketplace	https://github.com/marketplace
View all features	https://github.com/features
Enterprises	https://github.com/enterprise
Small and medium teams	https://github.com/team
Startups	https://github.com/enterprise/startups
Nonprofits	https://github.com/solutions/industry/nonprofits
App Modernization	https://github.com/solutions/use-case/app-modernization
DevSecOps	https://github.com/solutions/use-case/devsecops
DevOps	https://github.com/solutions/use-case/devops
CI/CD	https://github.com/solutions/use-case/ci-cd
View all use cases	https://github.com/solutions/use-case
Healthcare	https://github.com/solutions/industry/healthcare
Financial services	https://github.com/solutions/industry/financial-services
Manufacturing	https://github.com/solutions/industry/manufacturing
Government	https://github.com/solutions/industry/government
View all industries	https://github.com/solutions/industry
View all solutions	https://github.com/solutions
AI	https://github.com/resources/articles?topic=ai
Software Development	https://github.com/resources/articles?topic=software-development
DevOps	https://github.com/resources/articles?topic=devops
Security	https://github.com/resources/articles?topic=security
View all topics	https://github.com/resources/articles
Customer stories	https://github.com/customer-stories
Events & webinars	https://github.com/resources/events
Ebooks & reports	https://github.com/resources/whitepapers
Business insights	https://github.com/solutions/executive-insights
GitHub Skills	https://skills.github.com
Documentation	https://docs.github.com
Customer support	https://support.github.com
Community forum	https://github.com/orgs/community/discussions
Trust center	https://github.com/trust-center
Partners	https://github.com/partners
View all resources	https://github.com/resources
GitHub SponsorsFund open source developers	https://github.com/open-source/sponsors
Security Lab	https://securitylab.github.com
Maintainer Community	https://maintainers.github.com
Accelerator	https://github.com/open-source/accelerator
GitHub Stars	https://stars.github.com
Archive Program	https://archiveprogram.github.com
Topics	https://github.com/topics
Trending	https://github.com/trending
Collections	https://github.com/collections
Enterprise platformAI-powered developer platform	https://github.com/enterprise
GitHub Advanced SecurityEnterprise-grade security features	https://github.com/security/advanced-security
Copilot for BusinessEnterprise-grade AI features	https://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 support	https://github.com/enterprise/premium-support
Pricing	https://github.com/pricing
Search syntax tips	https://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
documentation	https://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
Sign in	https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fcomant%2FCrawlerTutorial
Sign up	https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E&source=header-repo&source_repo=comant%2FCrawlerTutorial
Reload	https://github.com/comant/CrawlerTutorial
Reload	https://github.com/comant/CrawlerTutorial
Reload	https://github.com/comant/CrawlerTutorial
comant	https://github.com/comant
CrawlerTutorial	https://github.com/comant/CrawlerTutorial
NanmiCoder/CrawlerTutorial	https://github.com/NanmiCoder/CrawlerTutorial
Notifications	https://github.com/login?return_to=%2Fcomant%2FCrawlerTutorial
Fork 0	https://github.com/login?return_to=%2Fcomant%2FCrawlerTutorial
Star 0	https://github.com/login?return_to=%2Fcomant%2FCrawlerTutorial
Code	https://github.com/comant/CrawlerTutorial
Pull requests 0	https://github.com/comant/CrawlerTutorial/pulls
Actions	https://github.com/comant/CrawlerTutorial/actions
Projects	https://github.com/comant/CrawlerTutorial/projects
Security and quality 0	https://github.com/comant/CrawlerTutorial/security
Insights	https://github.com/comant/CrawlerTutorial/pulse
Code	https://github.com/comant/CrawlerTutorial
Pull requests	https://github.com/comant/CrawlerTutorial/pulls
Actions	https://github.com/comant/CrawlerTutorial/actions
Projects	https://github.com/comant/CrawlerTutorial/projects
Security and quality	https://github.com/comant/CrawlerTutorial/security
Insights	https://github.com/comant/CrawlerTutorial/pulse
	https://github.com/comant/CrawlerTutorial
Branches	https://github.com/comant/CrawlerTutorial/branches
Tags	https://github.com/comant/CrawlerTutorial/tags
	https://github.com/comant/CrawlerTutorial/branches
	https://github.com/comant/CrawlerTutorial/tags
77 Commits	https://github.com/comant/CrawlerTutorial/commits/main/
	https://github.com/comant/CrawlerTutorial/commits/main/
.github/workflows	https://github.com/comant/CrawlerTutorial/tree/main/.github/workflows
.github/workflows	https://github.com/comant/CrawlerTutorial/tree/main/.github/workflows
docs	https://github.com/comant/CrawlerTutorial/tree/main/docs
docs	https://github.com/comant/CrawlerTutorial/tree/main/docs
源代码	https://github.com/comant/CrawlerTutorial/tree/main/%E6%BA%90%E4%BB%A3%E7%A0%81
源代码	https://github.com/comant/CrawlerTutorial/tree/main/%E6%BA%90%E4%BB%A3%E7%A0%81
爬虫进阶	https://github.com/comant/CrawlerTutorial/tree/main/%E7%88%AC%E8%99%AB%E8%BF%9B%E9%98%B6
爬虫进阶	https://github.com/comant/CrawlerTutorial/tree/main/%E7%88%AC%E8%99%AB%E8%BF%9B%E9%98%B6
.gitattributes	https://github.com/comant/CrawlerTutorial/blob/main/.gitattributes
.gitattributes	https://github.com/comant/CrawlerTutorial/blob/main/.gitattributes
.gitignore	https://github.com/comant/CrawlerTutorial/blob/main/.gitignore
.gitignore	https://github.com/comant/CrawlerTutorial/blob/main/.gitignore
CLAUDE.md	https://github.com/comant/CrawlerTutorial/blob/main/CLAUDE.md
CLAUDE.md	https://github.com/comant/CrawlerTutorial/blob/main/CLAUDE.md
README.md	https://github.com/comant/CrawlerTutorial/blob/main/README.md
README.md	https://github.com/comant/CrawlerTutorial/blob/main/README.md
package-lock.json	https://github.com/comant/CrawlerTutorial/blob/main/package-lock.json
package-lock.json	https://github.com/comant/CrawlerTutorial/blob/main/package-lock.json
package.json	https://github.com/comant/CrawlerTutorial/blob/main/package.json
package.json	https://github.com/comant/CrawlerTutorial/blob/main/package.json
README	https://github.com/comant/CrawlerTutorial
	https://github.com/comant/CrawlerTutorial#关于作者
Github万星开源自媒体爬虫仓库MediaCrawler作者	https://github.com/NanmiCoder/MediaCrawler
	https://github.com/comant/CrawlerTutorial#查看教程
https://nanmicoder.github.io/CrawlerTutorial/	https://nanmicoder.github.io/CrawlerTutorial/
查看B站合集地址	https://space.bilibili.com/434377496/channel/collectiondetail?sid=4035213&ctype=0
	https://github.com/comant/CrawlerTutorial#爬虫入门
01_为什么要写这个爬虫教程	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E5%85%A5%E9%97%A8/01_%E4%B8%BA%E4%BB%80%E4%B9%88%E8%A6%81%E5%86%99%E8%BF%99%E4%B8%AA%E7%88%AC%E8%99%AB%E6%95%99%E7%A8%8B.md
02_个人学会爬虫能赚钱吗	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E5%85%A5%E9%97%A8/02_%E4%B8%AA%E4%BA%BA%E5%AD%A6%E4%BC%9A%E7%88%AC%E8%99%AB%E8%83%BD%E8%B5%9A%E9%92%B1%E5%90%97.md
03_网络爬虫到底是什么	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E5%85%A5%E9%97%A8/03_%E7%BD%91%E7%BB%9C%E7%88%AC%E8%99%AB%E5%88%B0%E5%BA%95%E6%98%AF%E4%BB%80%E4%B9%88.md
04_爬虫的基本工作原理	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E5%85%A5%E9%97%A8/04_%E7%88%AC%E8%99%AB%E7%9A%84%E5%9F%BA%E6%9C%AC%E5%B7%A5%E4%BD%9C%E5%8E%9F%E7%90%86.md
05_常用的抓包工具有那些	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E5%85%A5%E9%97%A8/05_%E5%B8%B8%E7%94%A8%E7%9A%84%E6%8A%93%E5%8C%85%E5%B7%A5%E5%85%B7%E6%9C%89%E9%82%A3%E4%BA%9B.md
06_为什么说用Python写爬虫有天生优势	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E5%85%A5%E9%97%A8/06_%E4%B8%BA%E4%BB%80%E4%B9%88%E8%AF%B4%E7%94%A8Python%E5%86%99%E7%88%AC%E8%99%AB%E6%9C%89%E5%A4%A9%E7%94%9F%E4%BC%98%E5%8A%BF.md
07_Python常见的网络请求库	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E5%85%A5%E9%97%A8/07_Python%E5%B8%B8%E8%A7%81%E7%9A%84%E7%BD%91%E7%BB%9C%E8%AF%B7%E6%B1%82%E5%BA%93.md
08_爬虫入门实战1_静态网页数据提取	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E5%85%A5%E9%97%A8/08_%E7%88%AC%E8%99%AB%E5%85%A5%E9%97%A8%E5%AE%9E%E6%88%981_%E9%9D%99%E6%80%81%E7%BD%91%E9%A1%B5%E6%95%B0%E6%8D%AE%E6%8F%90%E5%8F%96.md
09_爬虫入门实战2_动态数据提取	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E5%85%A5%E9%97%A8/09_%E7%88%AC%E8%99%AB%E5%85%A5%E9%97%A8%E5%AE%9E%E6%88%982_%E5%8A%A8%E6%80%81%E6%95%B0%E6%8D%AE%E6%8F%90%E5%8F%96.md
10_爬虫入门实战3_数据存储实现	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E5%85%A5%E9%97%A8/10_%E7%88%AC%E8%99%AB%E5%85%A5%E9%97%A8%E5%AE%9E%E6%88%983_%E6%95%B0%E6%8D%AE%E5%AD%98%E5%82%A8%E5%AE%9E%E7%8E%B0.md
11_爬虫入门实战4_高效率的爬虫实现	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E5%85%A5%E9%97%A8/11_%E7%88%AC%E8%99%AB%E5%85%A5%E9%97%A8%E5%AE%9E%E6%88%984_%E9%AB%98%E6%95%88%E7%8E%87%E7%9A%84%E7%88%AC%E8%99%AB%E5%AE%9E%E7%8E%B0.md
	https://github.com/comant/CrawlerTutorial#爬虫进阶
01_工程化爬虫开发规范	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E8%BF%9B%E4%BB%B7/01_%E5%B7%A5%E7%A8%8B%E5%8C%96%E7%88%AC%E8%99%AB%E5%BC%80%E5%8F%91%E8%A7%84%E8%8C%83.md
02_反爬虫对抗基础_请求伪装	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E8%BF%9B%E4%BB%B7/02_%E5%8F%8D%E7%88%AC%E8%99%AB%E5%AF%B9%E6%8A%97%E5%9F%BA%E7%A1%80_%E8%AF%B7%E6%B1%82%E4%BC%AA%E8%A3%85.md
03_代理IP的使用与管理	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E8%BF%9B%E4%BB%B7/03_%E4%BB%A3%E7%90%86IP%E7%9A%84%E4%BD%BF%E7%94%A8%E4%B8%8E%E7%AE%A1%E7%90%86.md
04_Playwright浏览器自动化入门	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E8%BF%9B%E4%BB%B7/04_Playwright%E6%B5%8F%E8%A7%88%E5%99%A8%E8%87%AA%E5%8A%A8%E5%8C%96%E5%85%A5%E9%97%A8.md
05_Playwright进阶_反检测与性能优化	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E8%BF%9B%E4%BB%B7/05_Playwright%E8%BF%9B%E9%98%B6_%E5%8F%8D%E6%A3%80%E6%B5%8B%E4%B8%8E%E6%80%A7%E8%83%BD%E4%BC%98%E5%8C%96.md
06_登录认证_Cookie与Session管理	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E8%BF%9B%E4%BB%B7/06_%E7%99%BB%E5%BD%95%E8%AE%A4%E8%AF%81_Cookie%E4%B8%8ESession%E7%AE%A1%E7%90%86.md
07_登录认证_扫码与短信登录实现	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E8%BF%9B%E4%BB%B7/07_%E7%99%BB%E5%BD%95%E8%AE%A4%E8%AF%81_%E6%89%AB%E7%A0%81%E4%B8%8E%E7%9F%AD%E4%BF%A1%E7%99%BB%E5%BD%95%E5%AE%9E%E7%8E%B0.md
08_验证码识别与处理	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E8%BF%9B%E4%BB%B7/08_%E9%AA%8C%E8%AF%81%E7%A0%81%E8%AF%86%E5%88%AB%E4%B8%8E%E5%A4%84%E7%90%86.md
09_数据清洗与预处理	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E8%BF%9B%E4%BB%B7/09_%E6%95%B0%E6%8D%AE%E6%B8%85%E6%B4%97%E4%B8%8E%E9%A2%84%E5%A4%84%E7%90%86.md
10_数据分析与可视化	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E8%BF%9B%E4%BB%B7/10_%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90%E4%B8%8E%E5%8F%AF%E8%A7%86%E5%8C%96.md
11_进阶综合实战项目	https://github.com/comant/CrawlerTutorial/blob/main/docs/%E7%88%AC%E8%99%AB%E8%BF%9B%E4%BB%B7/11_%E8%BF%9B%E9%98%B6%E7%BB%BC%E5%90%88%E5%AE%9E%E6%88%98%E9%A1%B9%E7%9B%AE.md
	https://github.com/comant/CrawlerTutorial#高级爬虫
	https://github.com/comant/CrawlerTutorial#爬虫交流群
	https://github.com/comant/CrawlerTutorial/blob/main/docs/static/images/QIWEI.png
	https://github.com/comant/CrawlerTutorial#免责声明
	https://github.com/comant/CrawlerTutorial#star-history
	https://star-history.com/#NanmiCoder/CrawlerTutorial&Date
nanmicoder.github.io/CrawlerTutorial/	https://nanmicoder.github.io/CrawlerTutorial/
Readme	https://github.com/comant/CrawlerTutorial#readme-ov-file
Please reload this page	https://github.com/comant/CrawlerTutorial
Activity	https://github.com/comant/CrawlerTutorial/activity
0 forks	https://github.com/comant/CrawlerTutorial/forks
Report repository	https://github.com/contact/report-content?content_url=https%3A%2F%2Fgithub.com%2Fcomant%2FCrawlerTutorial&report=comant+%28user%29
Releases	https://github.com/comant/CrawlerTutorial/releases
Packages 0	https://github.com/users/comant/packages?repo_name=CrawlerTutorial
Please reload this page	https://github.com/comant/CrawlerTutorial
Contributors	https://github.com/comant/CrawlerTutorial/graphs/contributors
Please reload this page	https://github.com/comant/CrawlerTutorial
	https://github.com
Terms	https://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacy	https://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Security	https://github.com/security
Status	https://www.githubstatus.com/
Community	https://github.community/
Docs	https://docs.github.com/
Contact	https://support.github.com?tags=dotcom-footer

Viewport: width=device-width

URLs of crawlers that visited me.