A web crawler based on requests-html, mainly targets for url validation test.
Stars
31
Forks
12
Watchers
31
Open Issues
1
Overall repository health assessment
No package.json found
This might not be a Node.js project
bugfix: when response content-type is not specified, and this url is external link, it should be considered as external other than IGNORE
2b62b8eView on GitHubsome links can not be visited with HEAD method and will return 404 status code, so we shall recheck with GET method.
02a0a2dView on GitHub