Why does my crawler script suddenly end with no error?


Posted on 16th Feb 2014 07:03 pm by admin

Hi.

I have written a web crawler script. It will visit a large number of URL's with cURL.

After around 2-3 minutes of running, it will just stop, with no error output or notices.

I have these settings:
Code: [Select]set_time_limit(0);
ini_set('display_errors',1);
error_reporting(E_ALL|E_STRICT);
Any ideas why it would just stop?

No comments posted yet

Your Answer:

Login to answer
121 Like 30 Dislike
Previous forums Next forums
Other forums

Interpret Order
Hello all,

I'm wondering if I have this:

Code: $switch = array(
'one' =>

Reg Ex
Im trying to search for the string: srv_9 (Dead ???)

I thought to use preg_match, however I d

header redirect problems? please help
Ok i am making a site that generates youtube thumbnails and i am generating an image that displays 1

MS Exchange 2007 integration with ECC 6.0
Hello Experts,
We have ECC 6.0 and MS Exchange 2007. We need to integrate ECC 6 and MS Excha

ctype() validation - allowing illegal characters
Hello,
I use ctype() to filter and validate a user form. However, I am trying to allow certain c

Posting to another Site?
I have software that people install.
When they install it, is there a way to make it post their d

do while conditions
i want to display 6 images in a 2x3 table
now what would be the correct logic to
count that th

Passing an array of values From VB6.0 to Oracle PL/SQL Function
Basically we are using VB6.0.

When we try to pass an array from VB6.0 to an Oracle PL/SQL

Problem displaying "scraped" XML data
I'm basicly having trouble displaying XML data scraped from an URL using cURL.

What the code

Material Issuing for receiving batch
Dear All experts in MM/ PP,

Material issuing from main stores to factory is currently usi

Sign up to write
Sign up now if you have flare of writing..
Login   |   Register
Follow Us
Indyaspeak @ Facebook Indyaspeak @ Twitter Indyaspeak @ Pinterest RSS



Play Free Quiz and Win Cash