Why does my crawler script suddenly end with no error?

Posted on 16th Feb 2014 07:03 pm by admin

Hi.

I have written a web crawler script. It will visit a large number of URL's with cURL.

After around 2-3 minutes of running, it will just stop, with no error output or notices.

I have these settings:
Code: [Select]set_time_limit(0);
ini_set('display_errors',1);
error_reporting(E_ALL|E_STRICT);
Any ideas why it would just stop?

Your Answer:

Login to answer

Other forums

Serial number of exernal hard disk/Thumbdrive
Hi guys, I am new tio java networking concepts.Please tel me how to get Serial number of exernal

a function to check directory depth
I'm working on a php script to upload files in to a set directory.
the user can select to upload

Exporting new records
Trying to figure out the best way to set this up.

People register my site.

Their prof

LSB (PHP 5.3) problem with static value!
hello,

i'm having a problem. static::$text variable gets lost at some point. can someone plea

Form a inline view based on the results of the previous query?
Hi Experts,
Can we form a inline view on the results of a previous query (another inline view)?

max() problem
I have a while loop to get image names.
Code: $imagequery = mysql_query("SELECT * FROM ad_i

Perplexing problem showing a .jpg
Please disregard..........I figured it out

Parse XML
Good day,

I use oempro software and try to parse XMLReturn but it's not working for me, simpl

Parse error: syntax error, unexpected $end in
hello,
i receive the error :Parse error: syntax error, unexpected $end in C:xamppxampphtdocsh

Problems with array
This code is the last lines in an upload script that uploads 4 images at the same time and writes th