Why does my crawler script suddenly end with no error?

Posted on 16th Feb 2014 07:03 pm by admin

Hi.

I have written a web crawler script. It will visit a large number of URL's with cURL.

After around 2-3 minutes of running, it will just stop, with no error output or notices.

I have these settings:
Code: [Select]set_time_limit(0);
ini_set('display_errors',1);
error_reporting(E_ALL|E_STRICT);
Any ideas why it would just stop?

Your Answer:

Login to answer

Other forums

Do something every fifth time?
I'm trying to write a loop, but I want it to do something different after every fifth instance. Like

Backflush
I am looking for a report in SAP that would give me a list of all parts that are populated with a ba

Why use interfaces in PHP (OOP)
I am learning OOP in PHP and its been a pretty good learning experience so far. What I didn't unders

help with contest script
I would like to code a contest script to my site but i don't know how. Could someone just give me pu

problem in pagination when processing with selectbox form
Hi,

I have a search form, with select boxes, if am selecting the value from the form, it work

Uploading files/images via forms
Hey all, I am building a database for work that will contain some minor data on plant species along

Variables and Include
Code: [Select]<?php

$header = $_COOKIE['mss']['header'];
$body = $_COOKIE['mss']['b

error checking breaking my code
Hi there, OK first of all, big apologies for what I assume is really fundamental errors in the struc

problem with script manager
Hi iam using Net.2005.I have created ajax enabledwebsite.i have master page in which i have script m

Is it possible to put an entire 500-page book in a database with PHP?
I am working on an intranet and I was wondering if its possible to code php with mysql to enter a fu