Why does my crawler script suddenly end with no error?

Posted on 16th Feb 2014 07:03 pm by admin

Hi.

I have written a web crawler script. It will visit a large number of URL's with cURL.

After around 2-3 minutes of running, it will just stop, with no error output or notices.

I have these settings:
Code: [Select]set_time_limit(0);
ini_set('display_errors',1);
error_reporting(E_ALL|E_STRICT);
Any ideas why it would just stop?

Your Answer:

Login to answer

Other forums

UL and LI Add Form
The idea I want here is when the user click on a character name from the drop down select bar at the

Gaining access to Drop Down Menu Variable
code is below, I am creating a drop down menu. I want to echo the value selected in the drop down me

Mastering Regular Expressions, Third Edition, is available.
The Third Edition of Mastering Regular Expressions is available.

QuoteThis third edition is 5

Multiple Do / While Statements?
I'm still very new to PHP, and running in to a problem when trying to execute a do/while loop inside

cstdatomic (c++0x std::atomic) / g++ 4.4
Hello,

I'm trying to use cstdatomic (std::atomic in the upcoming c++0x standard) in g++

private constructor
Hello,

Can we create a constructor as private? If yes, what is the use of it? If no, why can'

get multiple rows
Hi I want to get multiple rows from a db

I am using this for the db query
Code: public

Preg_match unknown modifyer
Hello,

Im trying to write a little script for my forums i need to get the reply from my forum

PHP4 to PHP5 Conversion
Hi Everyone,

I am working on a site that is built up on PHP4 and each page is being started f

imap: how to save a copy of sent emails to sent elements
Hello my friends,

I am writing an online emailing application with inbox outbox/sent elements