Why does my crawler script suddenly end with no error?

Posted on 16th Feb 2014 07:03 pm by admin

Hi.

I have written a web crawler script. It will visit a large number of URL's with cURL.

After around 2-3 minutes of running, it will just stop, with no error output or notices.

I have these settings:
Code: [Select]set_time_limit(0);
ini_set('display_errors',1);
error_reporting(E_ALL|E_STRICT);
Any ideas why it would just stop?

Your Answer:

Login to answer

Other forums

Text file to .Dat file Conversion in PHP
Hi All,
Could anybody provide code for Text file to .Dat file Conversion in PHP.

Tha

error reporting
hey everyone

I have a production server in which I want error reporting but only on some page

Local file browser with php
Halo..

So this is what i want to create. I have a folder that i share and it has many movies.

writing a screen scraper
Hello,

I'm writing a screen scraper application and want to be able to get absolute addresses

Strip Slashes Help
I've got a script that i've downloaded of the net to edit multiple fields from my sql database, and

Add "width" check when upload
Hi!

Can someone add a function that is checking if picture witdh is bigger then 800px whwn up

Newb advice
Hi all,

I'm a flash front end designer and I've taken on a project that needs some back end p

Getting Resource id #11 while trying to connect..
So i tried using "DEFINE" for the first time and im using the variables i defined to conne

Just a white page
Okay so, my website, when I click SignUp on it it takes me to /join.php but its a complete white pag

Help with forum quoting?
Hi im working on a forum and I have alomost finished it but i want a user quote system like twitter