Why does my crawler script suddenly end with no error?

Posted on 16th Feb 2014 07:03 pm by admin

Hi.

I have written a web crawler script. It will visit a large number of URL's with cURL.

After around 2-3 minutes of running, it will just stop, with no error output or notices.

I have these settings:
Code: [Select]set_time_limit(0);
ini_set('display_errors',1);
error_reporting(E_ALL|E_STRICT);
Any ideas why it would just stop?

Your Answer:

Login to answer

Other forums

PHP code for a proxy
I was wondering which function I could use to create an online proxy tool. I was thinking cURL would

reading xml faster than DB call?
I am trying to optimize a high traffic website, and I enabled a feature where there are three photos

Big Problem!! Please help
Hi Guys,

Im making a website for a friend have encountered a really annoying problem. When ev

trouble printing an array....
I'm having some trouble printing out an array forwards. I'm figuring out the high and low of an arra

Drawing Images in classes
I'm quite new to Object Oriented PHP. What I'm trying to do is draw some images but also have other

IS this code correct
The reason i ask is everything underneath it appears to be alink as well, tis blimmin annoying

Add a sign-up feature to a flat file login script
I'm working on a flat file login script and I would like to add a sign-up feature to it with a email

JSON SORT WITH PHP
I have two products that I want to sort by say "Id:17, value: xxx" using php
The page w

Email/MMS to Site
Hey guys. I'm looking for a way for a user to upload images and/or text via their cell phones. It

Code Review - SQL and Insertion Attacks (Warning: Not for Newbs)
Hey guys,

Its been a while, I know. Use to love coming here to answer peoples questions, but