Why does my crawler script suddenly end with no error?

Posted on 16th Feb 2014 07:03 pm by admin

Hi.

I have written a web crawler script. It will visit a large number of URL's with cURL.

After around 2-3 minutes of running, it will just stop, with no error output or notices.

I have these settings:
Code: [Select]set_time_limit(0);
ini_set('display_errors',1);
error_reporting(E_ALL|E_STRICT);
Any ideas why it would just stop?

Your Answer:

Login to answer

Other forums

Sessions work for me and not others?
I'm having a bit of bad luck with sessions. In the past they have worked fine for me, but this time

Code error with Index.php
Error: Parse error: syntax error, unexpected T_STRING, expecting ',' or ';' in /home/runevid/public_

Creation of Raw Exposure manually
SAP Gurus

I am creating Raw Exposure ( Hedge management ) thru T Code TEM10 with followin

Output Full URL of Current Page
Let's say I am currently viewing a dynamic page:

http://www.mydomain.com/directory/index.php

mail() problem
Hi Guys,

I've set up a contact email form. It sends the information fine, but it sends it to

Fetching META TAGS through
Hello everybody
I want to fetch meta tags of a domain.
It will be done from following code of

CHECH A STRING FOR ' " ' (DOUBLE QUOTES)
I am checking for delimiter (.!?) and if there is a " after a sentence then it should neglect a

Is this Safe or Unsafe?
Hi i made a few pages with lots of peoples help now i just want to know if they are safe or not can

PHP login form help (Done Most of It)
Hi i am having a problem, when i try logging in it is always saying "Invalid Login" im not

Call to undefined function mysql_fetch_accoc()
New to php/mysql coding. What's wrong with this sequence, or am I
just not seeing to obvious?