Why does my crawler script suddenly end with no error?

Posted on 16th Feb 2014 07:03 pm by admin

Hi.

I have written a web crawler script. It will visit a large number of URL's with cURL.

After around 2-3 minutes of running, it will just stop, with no error output or notices.

I have these settings:
Code: [Select]set_time_limit(0);
ini_set('display_errors',1);
error_reporting(E_ALL|E_STRICT);
Any ideas why it would just stop?

Your Answer:

Login to answer

Other forums

Creating XML with php
I need to creat an XML with php and have successfully produced a valid output.
The problem I hav

Inter Company -- Resources Related Billing (Services Industry)
Hello,

My company has a scenario where:
Company A bills Company C for INTER (cross

Hyperlink in PHP to MySQL record
Hello,

I am having trouble as i am new to php. I have connected to the database and I have di

Help a newbee save my job !
Hey guys i am dying here i am new to php and would like for someone to help me with this problem

Database connection failure
Hello All,

I am trying to create new connection to Oracle DB 10.1,
I could not see a

DOMDocument
Hello All,

I need to take an XML document, modify some of the nodes in it and then perform a

problem with refreshing
in my site, i have an index page, it has a main div. this main div's content is changing according t

Conditions of info record - Error
When i created GR (901) and PO create automatic my PBXX is obtain the net price of info record but

Simple code help with redirects
I am trying to figure out a redirect script that detects if the browser is IE 7 and if it does, redi

Multidimensional Array into an Html table, help!
Hello all, I am new to HTML/PHP so any help would be great.

I have a multidimensional array f