Why does my crawler script suddenly end with no error?

Posted on 16th Feb 2014 07:03 pm by admin

Hi.

I have written a web crawler script. It will visit a large number of URL's with cURL.

After around 2-3 minutes of running, it will just stop, with no error output or notices.

I have these settings:
Code: [Select]set_time_limit(0);
ini_set('display_errors',1);
error_reporting(E_ALL|E_STRICT);
Any ideas why it would just stop?

Your Answer:

Login to answer

Other forums

mysql_affected_rows() usage
Possibly a MySQL issue, but the function that is not behaving in the anticipated way is a PHP functi

How to schedule the a job?
Hi,
I have prepared a task ( i.e. a procedure is prepare the file and send the same to con

PHP MySQL Return Results Issue
Hi guys,

I'm running joomla and made a module which is using ajax.
Basically i just have a

Xacute search within SQL results
I have a query that is pulling data, and I want to return a specific value from the results of that

How can i steam a video on my PHP site? (non YouTube)
Basically I have a PHP site and don't want to give page rank to YouTube. I just want to stream a bas

How do I get the row number from from an sql table query
Hi all

I have a table that I query and it returns a number of rows.
mysql_num_rows($query)

Using unserialize()
Hi there.

I have some data in my database that is serialized.

e.g.

a:2:{i:0

Database 'Validation'
Hi everyone,
I'm trying to validate the password entered by the user with the password in the dat

Menu restriction
How can I restrict the individual menu that would appear when a user logs in so that all users are c

creating a 1 to 100 in a table
hi guys I am a newb in php need some help.

I have a table with 1 column and 100 rows and in