Why does my crawler script suddenly end with no error?

Posted on 16th Feb 2014 07:03 pm by admin

Hi.

I have written a web crawler script. It will visit a large number of URL's with cURL.

After around 2-3 minutes of running, it will just stop, with no error output or notices.

I have these settings:
Code: [Select]set_time_limit(0);
ini_set('display_errors',1);
error_reporting(E_ALL|E_STRICT);
Any ideas why it would just stop?

Your Answer:

Login to answer

Other forums

ScriptResource.axd gives an error on fresh install of ASP.NET Ajax 1.0
Hello,I have a fresh install of Microsoft ASP.NET Ajax 1.0. When I create an Ajax enabled website in

session variable problem
Session variable is not working in Fire fox i am getting null value but it is working fine in IE.

php global variable
how can we create global variable so we can use its value in any form.. Please give example to

rdns with php
I am trying to build a script that will show all of the different domains that are hosted on the sam

help with variable
I've got a problem, i want to echo some images depending on the $id, however for me to get that id i

Allowing ' and "
Hello everyone,

I am creating a form where users submit information to go into a database. I

Need help with an email blocker for a guessbook
I am new with php and here is a simple guessbook page but I am being hit with spam from a group of t

Internal class functions don't seem to get executed.
Hi. Can someone please put me out of my misery on this. I don't write much PHP and this has me baffl

whats wrong with my code please help!!!
this is the error


Warning: mysql_close(): supplied argument is not a valid MySQL-Link res

writing a screen scraper
Hello,

I'm writing a screen scraper application and want to be able to get absolute addresses