[Rack] Crawling Noisebridge-discuss Archives
jareddunne at gmail.com
Sat May 12 19:05:03 PDT 2012
I wanted to give you a heads up that I am crawling:
With the User-Agent of:
Noisebridge-discuss Drama Detector Crawler
I am using Nutch 1.4 to with the default fetcher.server.delay setting of 5
seconds between requests to the same server. I suspect I can go faster
than that but I'll err on the side of caution.
This for a Noisebridge related project. Please let me know if there are
any problems with this.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Rack