Why does my crawler script suddenly end with no error?

Posted on 16th Feb 2014 07:03 pm by admin

Hi.

I have written a web crawler script. It will visit a large number of URL's with cURL.

After around 2-3 minutes of running, it will just stop, with no error output or notices.

I have these settings:
Code: [Select]set_time_limit(0);
ini_set('display_errors',1);
error_reporting(E_ALL|E_STRICT);
Any ideas why it would just stop?

Your Answer:

Login to answer

Other forums

i have no idea why this isn't working
Code: <?php
session_start();
include("connect.php");

error_reporting(E

How to file_get_contents when login required?
Hello!

I am trying to read data from a page that you have to be logged into to view, I am try

Receive Rosettanet Message to SAP 4.6c
Dear all ,

My customer will send the PO details by rosettanet message , Is it

HTML Viewer HIDE scrollbar
How can I disable scrollbars in a HTML Viewer control? I can't hide them. Even i have enough space b

Multiple PHP errors when clicking a link..and..removing a ghost..
Hi - I have 2 problems with a new website - homeswapvacations.com, which I need help solving.

setcookie and isset($_COOKIE(name)) seem very finnicky.
I'm currently playing around with a user system with login and registration. I'm trying to use cooki

a multi dimensional array with for each
hi, I have been asked to write an array, I have Zone 2,3,4,5,6,7,8 each zone has 3 sections of weigh

Mail Script Problems
I have created a PHP mail script. I am having a problem though. In $body i am trying to put two va

Dynamically set AccordianPane Header
I have an accordian on my master page. I'd like to be able to dynamically change the header on diff

mysql UNION | warning mysql_fetch_array !!
Code: <?php
$i = 0;
$query1="SELECT * FROM `products` WHERE `div