Why does my crawler script suddenly end with no error?

Posted on 16th Feb 2014 07:03 pm by admin

Hi.

I have written a web crawler script. It will visit a large number of URL's with cURL.

After around 2-3 minutes of running, it will just stop, with no error output or notices.

I have these settings:
Code: [Select]set_time_limit(0);
ini_set('display_errors',1);
error_reporting(E_ALL|E_STRICT);
Any ideas why it would just stop?

Your Answer:

Login to answer

Other forums

Formatting echo from database
So I have a database that stores First and last names, then echos them back to a website, as of now

How to ... (FAQs)
... get e-mail notifications As several people asked how to get e-mail notifications when new posti

line breaks in between fetched file names
Hi,

I have this code:
Code: <?php
if($dir = opendir('files')){
while (($f

adding 0 to numbers
Hey guys i need to create a 00001 number in a loop. Is there a better way of forcing a 5 digit numbe

PHP and SMS
Hi all forum members. I am new here and am unsure what category shoild I post this in.
Moderator

Help with PHP and checkboxes
I am posting this for a friend of mine, i wrote a small script for her to process her form data to a

Undefined Variable: PHP_SELF, pls help
Hi,

Im a newbie on PHP / MySQL programming and Im running a script to search one field on my

Insert Failing.
Hey,
I am making a Sign up page for a website, but the insert query into the Database does not se

small inaccuracies
I have this code to convert fractional base 10 into base 2:

while($num > 0)

Displaying Site Logo/Icon on a Title tag
really need some help out here. I want to insert a Logo/Icon just like the one close to the "PH