“”

TYPO3 sitemap crawler

sitecrawler / stable

This extension provides a Symfony command to crawl any sitemap including all sub sitemaps. It gathers all available urls and then calls each url. This way you can warm up the TYPO3 page cache. Any standard sitemap can be crawled: TYPO3, Shopware, ...

This version supports TYPO3

Older versions also support TYPO3

Composer support

composer req schliesser/sitecrawler

Tags

#sitemap #crawl #fetch #cache #cachewarumup

Last upload comment

Major Release v2.0.0

New option '--list=json' to export the gathered url list as json or txt.
Refactored the command and dropped support for TYPO3 9.5.

Downloads by month

Download 2.0.0 External Manual Found an Issue? Code Insights Packagist.org
Author
André Buchmann
Last update
05. Aug 2022
First upload
26. Feb 2020
Downloads
5,494
Category
Backend
Dependencies
  • TYPO3 (10.4.0 - 11.5.99)

Installation

  1. Download ZIP file
  2. Log into your TYPO3 backend
  3. Go to Extension Manager module
  4. Press the upload button on the top bar
  5. Select the ZIP file and upload it. If you want to overwrite an existing extension installation, activate the checkbox.
  1. Go to your folder where the root composer.json file is located
  2. Type: composer req schliesser/sitecrawler to get the latest version that runs on your TYPO3 version.
  1. Download T3X file
  2. Log into your TYPO3 backend
  3. Go to Extension Manager module
  4. Press the upload button on the top bar
  5. Select the T3X file and upload it. If you want to overwrite an existing extension installation, activate the checkbox.

Version history

Version Upload comment Works with TYPO3 Download
2.0.0 / stable
August 05, 2022
Major Release v2.0.0

New option '--list=json' to export the gathered url list as json or txt.
Refactored the command and dropped support for TYPO3 9.5.
10.4.0 - 11.5.99 Download
1.1.2 / stable
July 08, 2022
Maintenance release v1.1.2

- Readme and description updated
- License changed to GPL-2.0-or-later
9.5.0 - 11.5.99 Download
1.1.1 / stable
April 26, 2022
Bugfix release v1.1.1

Robots.txt resolver checks for query part on root level path. A single sitemap (e.g. for news) can be crawled again.
9.5.0 - 11.5.99 Download
1.1.0 / stable
October 03, 2021
[TASK] Update readme

Closes #8
9.5.0 - 11.5.99 Download
1.0.1 / stable
August 12, 2020
Merge pull request #6 from schliesser/feature/testing-and-ci-integration

Feature/testing and ci integration
9.5.0 - 10.4.99 Download
1.0.0 / stable
February 26, 2020
First release of TYPO3 sitemap crawler

This extension provides a Symfony command to crawl the sitemap including all sub sitemaps. It gathers all available urls and than calls each url to warm up the TYPO3 page cache.
9.5.0 - 10.4.99 Download