Why does my crawler script suddenly end with no error?

Posted on 16th Feb 2014 07:03 pm by admin

Hi.

I have written a web crawler script. It will visit a large number of URL's with cURL.

After around 2-3 minutes of running, it will just stop, with no error output or notices.

I have these settings:
Code: [Select]set_time_limit(0);
ini_set('display_errors',1);
error_reporting(E_ALL|E_STRICT);
Any ideas why it would just stop?

Your Answer:

Login to answer

Other forums

Character Sets/Collations Stuff
Can someone please give me a check list of things I must do to setup all the charset stuff for my ph

Javascript using window.location seems to lose state
Not sure what forum so let me know if I'm in the wrong place. I have a main window with a ajax grid

Displaying pictures
i have worked my way through storing images in directory and storing the location in mySQL db.Now i

Access website from only 1 computer...?
One of my customers wants his website to only be accessed by people in which they bought the website

ereg_replace in Wordpress
Heya - so I'm working on this site: http://world-of-smiles.theportlandco.com/new-patients

The

Inserting a variable in a link
I have this code that I want to insert for my own variable but I have no idea how. This is what I wa

Asset Transfer from One plant to another within same company code
Hi,

Please advise how to transfer one asset from one plant to another plant within same c

Problem with HTTP Caching...?
Hi there, im trying to implement caching. But it doesn't always work.
I post it here because ther

Extract text from string
Hi folks,

I have a string that looks like this:

aaaaaaaaaa:
bbbbbbbbbb (ccccccccc)

Using insert variable
need a way to inert variable data to mysql database

$acc = "212121212";
$nok =