[Rack] Crawling Noisebridge-discuss Archives
matt at nycresistor.com
Mon May 14 10:12:00 PDT 2012
put it up on torrent?
On Sat, May 12, 2012 at 10:54 PM, Andy Isaacson <adi at hexapodia.org> wrote:
> On Sat, May 12, 2012 at 07:05:03PM -0700, Jared Dunne wrote:
>> I wanted to give you a heads up that I am crawling:
>> With the User-Agent of:
>> Noisebridge-discuss Drama Detector Crawler
>> I am using Nutch 1.4 to with the default fetcher.server.delay setting of 5
>> seconds between requests to the same server. I suspect I can go faster
>> than that but I'll err on the side of caution.
>> This for a Noisebridge related project. Please let me know if there are
>> any problems with this.
> There's a mbox if that would be easier.
> Rack mailing list
> Rack at lists.noisebridge.net
More information about the Rack