Tree @master (Download .tar.gz)
Commit History
@master
git clone https://simonvolpert.com/fediscover/
- Update minimum required Python version in README Simon Volpert 1 year, 10 months ago
- Update README Simon Volpert 4 years ago
- Convert a username into a profile URL before crawling Simon Volpert 4 years ago
- Force crawling the next URL when a profile URL is passed on the command line Simon Volpert 5 years ago
- Remove obsolete comment Simon Volpert 5 years ago
- Unify reporting of blacklisted URLs Simon Volpert 5 years ago
- Remove "skip" option Simon Volpert 5 years ago
- Keep crawling on transient connection failure, retry failed URLs once Simon Volpert 5 years ago
- Update README Simon Volpert 5 years ago
- Keep crawling until there are users in the queue or the URL queue runs out Simon Volpert 5 years ago
- Add a "verbose" option to print every URL being processed Simon Volpert 5 years ago
- Add skipped URL cound in the post-crawl report Simon Volpert 5 years ago
- Add blacklisting Simon Volpert 5 years ago
- Remove duplicate matches URLs from pages when scraping Simon Volpert 5 years ago
- Automatically crawl the next URL if the profile queue is empty Simon Volpert 5 years ago
- Add following/follower URLs to queue when crawling a profile URL Simon Volpert 5 years ago
- Refactor URL loading code Simon Volpert 5 years ago
- Refactor URL crawling Simon Volpert 5 years ago
- Move page scraping code into its own function Simon Volpert 5 years ago
- Add some code comments Simon Volpert 5 years ago
- Add "dry-run" and "random" options Simon Volpert 5 years ago
- Normalize profile URLs passed on the command line before processing Simon Volpert 5 years ago
- Put print-to-standard-error code into its own function Simon Volpert 5 years ago
- Move repeating URL caching code into a function Simon Volpert 5 years ago
- Move newly processed link count to the appropriate session state containers Simon Volpert 5 years ago
- Print all unimportant messages to standard error Simon Volpert 5 years ago
- Reword follow page regex to satisfy the linter Simon Volpert 5 years ago
- Fix use-before-declaring bug Simon Volpert 5 years ago
- Process profile URLs passed to "crawl" correctly Simon Volpert 5 years ago
- Insert additional profile pages in the front of the crawlable URL list Simon Volpert 5 years ago