Why does my crawler script suddenly end with no error?

Posted on 16th Feb 2014 07:03 pm by admin

Hi.

I have written a web crawler script. It will visit a large number of URL's with cURL.

After around 2-3 minutes of running, it will just stop, with no error output or notices.

I have these settings:
Code: [Select]set_time_limit(0);
ini_set('display_errors',1);
error_reporting(E_ALL|E_STRICT);
Any ideas why it would just stop?

Your Answer:

Login to answer

Other forums

quick question about System Change Number(SCN) in FlashBack Query topic
i know tht we can get the SCN of the database using flashback concept as follows

SQL>

Get word number x from string?
How can I use a function to loop through a string, and "have a look at" every word in the

Save username into DB
OkaY so I got my blog to actually save the posts and whatever, all I need now is for it to keep the

php automatically escaping single quotes
I'm trying to test out my security a bit and I've noticed that php is escaping my single quotes. For

What am I missing here? Help!
Hello all!. I can't seem to get this working right. Well - it renders right, but something is goin

Best way to cross matching large datasets
Hi,

Im running a script where am I cross matching about 200 000 data sets with each other. Ea

Filtering Tables
Hello Thanks for any help any one can give Im very new to PHP..

I need to pull data for clien

Can php detect connection speed and thereafter swap out elements?
No code to post, lads, yet. Just looking for a yay or nay at this stage:

Can a php script be

Basic Forum Tutorial
Hi, I'm new to PHP. I want to build a basic forum for my site using PHP and MySQL. I've searched t

connect() succeeds but accept() does not?
Please help me with this problem. I'm working on a time-sensitive project where I'm using TCP socket