Why does my crawler script suddenly end with no error?

Posted on 16th Feb 2014 07:03 pm by admin

Hi.

I have written a web crawler script. It will visit a large number of URL's with cURL.

After around 2-3 minutes of running, it will just stop, with no error output or notices.

I have these settings:
Code: [Select]set_time_limit(0);
ini_set('display_errors',1);
error_reporting(E_ALL|E_STRICT);
Any ideas why it would just stop?

Your Answer:

Login to answer

Other forums

Feed Maker
Hi all.
First of all I must say I am not a php developer so I am afraid I don't know much about i

Streaming local file from PHP while it’s been written to by a CURL process
I am creating a simple Proxy server for my website. Why I am not using mod_proxy and mod_cache is a

Simple MySQL script is not working
Hey guys,

Just familiarizing myself with PHP and MySQL. I am following along in the O'Reilly

Open link with largest int string first
I have the following links i would like to open either all at once or one-by-one. How would i procee

MII Trends - add data onto chart object
Hello,

Can anyone please provide some thoughts on my current requirement:
its pretty

How to create a static html menu from a database
Hi,

I have built a small cms which allows me to create simple html pages and then upload them

PHP template help
Ok, I don't know if anyone can help me but I am using PHP templates to make a site. I am a newbie an

include function being weird?
I have a very simple system to grab an IP outputted by a PHP file.. (for a project I have) but it ou

whats wrong with my code please help!!!
this is the error


Warning: mysql_close(): supplied argument is not a valid MySQL-Link res

IF STATEMENT HELP
Hi
i have created a calendar from a table:

Code: Calendar: October 2009
<table w