Why does my crawler script suddenly end with no error?

Posted on 16th Feb 2014 07:03 pm by admin

Hi.

I have written a web crawler script. It will visit a large number of URL's with cURL.

After around 2-3 minutes of running, it will just stop, with no error output or notices.

I have these settings:
Code: [Select]set_time_limit(0);
ini_set('display_errors',1);
error_reporting(E_ALL|E_STRICT);
Any ideas why it would just stop?

Your Answer:

Login to answer

Other forums

how to hide input fields on selection screen using variant attribute
Hello all,

I want to know how to hide input fields on selection screen using variant attr

Can't find the problem (no error message)!
OK, here's the deal. This code is really weirdly formatted (sorry!) and hard to understand, so I'm g

How to find OS bit version
How can I tell the person browsing my site is running 32, or 64 bit operating system?

Detail Expenses Report by Cost Center
I would like to obtain a report out of SAP that shows a list of expenses by cost center that shows t

Storing/displaying image urls in a database?
Hi
Basically I have the following scenario:
User chooses picture from list. Picture info is ad

Saving PHP output as a file
I wondered if anyone knows how to save the rendered output of a PHP script as a file?

I have

Why is this function returning a false value when it shouldn't be??
This is in an include file. I want it to check a value in an html form and see if it's just white s

Error in query: Resource id #4??
hey guys, having a minor (i think) problem here that i havent been able to figure out. long story sh

Transform value 0 in no and 1 in yes help needed
Hello,
I have this in the table: "value" type int 0 to represent false and 1 to represe

Retrieving innerHTML with cURL?
Hey all (sorry I know I'm a leecher, but I soon won't be. This is my first PHP project, but not my l