Why does my crawler script suddenly end with no error?

Posted on 16th Feb 2014 07:03 pm by admin

Hi.

I have written a web crawler script. It will visit a large number of URL's with cURL.

After around 2-3 minutes of running, it will just stop, with no error output or notices.

I have these settings:
Code: [Select]set_time_limit(0);
ini_set('display_errors',1);
error_reporting(E_ALL|E_STRICT);
Any ideas why it would just stop?

Your Answer:

Login to answer

Other forums

Need help in log in and log out?
hi there all of u. i have recently created a site for someone. i have placed log in and log out and

PHP/Database issue
My friend is helping me make a database where you go to a certain webpage of my site and the page wi

What am I missing here? Help!
Hello all!. I can't seem to get this working right. Well - it renders right, but something is goin

Uploading Filetypes and placing them in seperate folders.
Hello,

first post , and asking for help im afraid. Very new to PHP, was making good progress

Not Inserting into DB
I can't get my Add new Division form to submit to the database. Anyone see my issue?

Code: &a

Parse String
I need to parse the string below into three variables. How would I do that?

john + 10/20/2008

When i am Canseling the Billing Document in VF11 I am getting Shortdump
Hi Experts

When i am Canseling the Billing Document in VF11 I am getting Shortdump.

PHP not creating file.
Its supposed to create a .ini file in the /scriptfiles/users/ with the name they entered into the fi

Ajax Issues - Update Panel / Timer. Intellisense doesn't know about them
Type 'System.Web.UI.ScriptManager' does not have a public property named 'UpdatePanel'. That is wha

Php mysql - select?
Hi i have this code:
Code: $iteminfo = mysql_query("SELECT desc FROM wc_items WHERE itemid =