Merge pull request #5 from ContinuumIO/youngblood/convert_to_CrawlSpider

split comments from posts into separate doctypes

increase number of comments per page to reddit max (500)

make records added to ES unique; add sane index

fix crawlspider; fix fields in link_page parsing

Add reddit link page item. Pass parsing from link page to comment page.

adds depth_finder_spider.py and scraped-date property to comment Items

converts RedditSpider to a CrawlSpider, adds scrape_datetime to RedditCommentItem

Merge pull request #3 from ContinuumIO/bhard/restructure

Add scrapinghub as a channel to environment.yml.

Renamed comment spider, added environment.yml

Merge pull request #1 from ContinuumIO/docker-images

Merge pull request #2 from ContinuumIO/scrape_reddit_comments

initial commit of reddit scraper. Dumps comments to elasticsearch.

initial commit of scrapy Dockerfile