Site Crawler

Copyright © by Kasper Skaarhoj, Daniel Ptzinger - AOE media <kasper2005@no spam pleasetypo3.com>
Published under the Open Content License available from http://www.opencontent.org/opl.shtml

Table Of Contents

• 1. Crawler
  • 1.1. Introduction
    • What does it do?
    • Screenshots
  • 1.2. Configuration
    • Page TSconfig Reference (tx_crawler.crawlerCfg)
  • 1.3. Executing the queue
    • Executing queue with cron-job
    • Executing queue with backend
    • Building and Executing queue right away
  • 1.4. Tutorial
    • Configuration of crawler
    • Submit URLs to queue
    • Explanation of processing instructions
    • Run via backend
    • Run via cron

This document is related to version 2.1.0 of the extension crawler.