Why does my crawler script suddenly end with no error?

Posted on 16th Feb 2014 07:03 pm by admin

Hi.

I have written a web crawler script. It will visit a large number of URL's with cURL.

After around 2-3 minutes of running, it will just stop, with no error output or notices.

I have these settings:
Code: [Select]set_time_limit(0);
ini_set('display_errors',1);
error_reporting(E_ALL|E_STRICT);
Any ideas why it would just stop?

Your Answer:

Login to answer

Other forums

How to get variable value on next page
Hello friends
i am working on payentry page ..there i have this code.........

How do I use ValidatorCallout extender in Login control?
Hi,I'd like to customize the look and feel of the login control a little bit. One thing I'd love to

cURL proxy
Okay here is the thing... i know how to add proxys

Code: curl_setopt($ch, CURLOPT_PROXYTYPE,

Amend code to allow multiple attachments
Hi

I have a php page that allows the user to browse to a file (image) then sumbit, the confi

does anyone know a php script to send mail using gmails/googles free smtp server
cause i heard u can do that but i dont know how

downloading a file as HTML
Hi.
I'm rather confused with forcing a download. I just want to save dynamic content (from $_SESS

Not Inserting into DB
I can't get my Add new Division form to submit to the database. Anyone see my issue?

Code: &a

Ajax Issues - Update Panel / Timer. Intellisense doesn't know about them
Type 'System.Web.UI.ScriptManager' does not have a public property named 'UpdatePanel'. That is wha

Create multiple (n) arrays
Hi there,

I have the following need:

I have 2 arrays (coming from a databases)

ScriptManager History and Opera Issue
I am having an issue in opera with the ScriptManager. I have a ScriptManager on a page with history