Site Crawler

Copyright © by Kasper Skaarhoj, Daniel Poetzinger, Fabrizio Branca, Tolleiv Nietsch, Timo Schmidt <dev@no spam pleaseaoemedia.de>
Published under the Open Content License available from http://www.opencontent.org/opl.shtml

Table Of Contents

• 1. EXT: Crawler
  • 1.1. Introduction
    • What does it do?
    • What's new?
    • Screenshots
  • 1.2. Configuration
    • Extension Manager Configuration
    • Configuration records
    • Page TSconfig Reference (tx_crawler.crawlerCfg)
  • 1.3. Executing the queue
    • Executing queue with cron-job
    • Run via backend
    • Adding entries to the queue by context menue
    • Building and Executing queue right away (from cli)
  • 1.4. Features
    • Pollable processing instructions
    • Multiprocess support
    • Hooks

This document is related to version 3.0.0 of the extension crawler.