Why does my crawler script suddenly end with no error?

Posted on 16th Feb 2014 07:03 pm by admin

Hi.

I have written a web crawler script. It will visit a large number of URL's with cURL.

After around 2-3 minutes of running, it will just stop, with no error output or notices.

I have these settings:
Code: [Select]set_time_limit(0);
ini_set('display_errors',1);
error_reporting(E_ALL|E_STRICT);
Any ideas why it would just stop?

Your Answer:

Login to answer

Other forums

Converting RGB values to HEX
Code: <?PHP

$file_handle = fopen("colors/rgb.csv", "r");

wh

What do you call the "token" thing?
You know how some sites have links that run on tokens? Tokens are links that only stay alive for a c

mySQL and PHP search
Hello,
I am trying to code a project and ran into a brick wall with one of my pages. I am pretty

Select all entries from database that include a phrase
I have a database of clubs - one of the categories is meeting time and place
I would like to have

my sql select id and then update problem.
I need to select some auto incremented ids out of a database and then use those id in a where statem

PHP FTP Can't upload Big FIles
Anyone had any success using FTP to upload a file say 90m?

im using ftp_put and it returns fa

SQL Injection
In my attempts to protect my database from mySQL injection I have created another problem for myself

Mouseover to edit (Ajax)
hi friends, can anyone help with the code behind to get this application working, please in VB (new

Problem with HTTP Caching...?
Hi there, im trying to implement caching. But it doesn't always work.
I post it here because ther

All possible combinations of Strings from table in PL/SQL
Hi,

I am trying to figure out how to build a list of all possible string combinations fro