Why does my crawler script suddenly end with no error?

Posted on 16th Feb 2014 07:03 pm by admin

Hi.

I have written a web crawler script. It will visit a large number of URL's with cURL.

After around 2-3 minutes of running, it will just stop, with no error output or notices.

I have these settings:
Code: [Select]set_time_limit(0);
ini_set('display_errors',1);
error_reporting(E_ALL|E_STRICT);
Any ideas why it would just stop?

Your Answer:

Login to answer

Other forums

Doubles are giving me problems
Ok so, first of all i made a double = 0.05, but when running the debugger it shows up as 0.04999. I

PHP / MySQL Associative Multidimensional Array:
Hello.

I have data in a MySQL Table that adheres to the below: (note, no index, could add if

Calling variables
I want to call this to my website but it doesn't display what I want it to,

Code: <h5&

Filtering for a phrase using pregmatch
Here is my code which looks for addresses which start with ' src=" '

Code: [Select]preg_

Accessing Infotype data in dialog program
Hi All
In Dialog programs attributes I didn't see any logical database field. How can I access

Procedural to OOP
John Kleijn said that to avoid writing "crappy code", we should learn OOP and common OO pr

IIS & NW MII on the same server
Hi,

We're weighing the possibility to run both IIS and MII (NetWeaver) on the same server

How to display value in drop down list after form has been reloaded
Please bear with me as I am very new to php and html.

I have a form with several drop down me

CHMOD script
Hi,

I need a script to read all files in a folder and set to 777.

Can anyone help out

Gaining access to Drop Down Menu Variable
code is below, I am creating a drop down menu. I want to echo the value selected in the drop down me