Extremely fast and efficient Ruby domain spider
Stars
36
Forks
18
Watchers
36
Open Issues
1
Overall repository health assessment
No package.json found
This might not be a Node.js project
31
commits
1
commits
Forgot to merge URL's before checking if there's an image or internal. All relative URL's weren't marked as internal and disregarded such as '/info/here.html'.
90f0ee6View on GitHubAdded proxy list options to initializer for crawler use.
92507b9View on GitHub