“”

Site Crawler Site Crawler
crawler / stable

Libraries and scripts for crawling the TYPO3 page tree.

  • 7 LTS

Last upload comment

[RELEASE] Release of crawler 5.2.0

* [TASK] add post process signal slot also for use with crawler_im (#211)
* [FEATURE] TypoScript Root Template must be configured in crawler configuration
* [TASK] Add CSS class for AdditionalFieldProviders for TYPO3 7.

Downloads by version

Author
Kasper Skaarhoj, Daniel Poetzinger, Fabrizio Branca, Tolleiv Nietsch, Timo Schmidt, Michael Klapper, Stefan Rotsch, Tomas Norre Mikkelsen
Last update
06. Nov 2017
First upload
22. Dec 2005
Downloads
36,809
Category
Backend Modules
Dependencies
  • PHP (5.3.0 - 7.0.99)
  • TYPO3 (6.2.0 - 7.6.99)
Manual
Extension Manual
External repository
Take a look into the code

Installation

  1. Download ZIP file
  2. Log into your TYPO3 backend
  3. Go to Extension Manager module
  4. Press the upload button on the top bar
  5. Select the ZIP file and upload it. If you want to overwrite an existing extension installation, activate the checkbox.
  1. Download T3X file
  2. Log into your TYPO3 backend
  3. Go to Extension Manager module
  4. Press the upload button on the top bar
  5. Select the T3X file and upload it. If you want to overwrite an existing extension installation, activate the checkbox.

Version history

Version Upload comment Works with TYPO3 Download
5.2.0 / stable
November 06, 2017
[RELEASE] Release of crawler 5.2.0

* [TASK] add post process signal slot also for use with crawler_im (#211)
* [FEATURE] TypoScript Root Template must be configured in crawler configuration
* [TASK] Add CSS class for AdditionalFieldProviders for TYPO3 7.
6.2.0 - 7.6.99 Download ZIP Archive
5.1.4 / stable
October 09, 2017
[RELEASE] Release of crawler 5.1.4

** Release Notes **

* [CLEANUP] Adjust version constraints
* [FEATURE] Use actual page ID for TSFE initialization (#214)
* [TASK] Set more realistic timeouts (30 min should be sufficient) (#213)
* Cleanup/codestyle (#2
6.2.0 - 7.6.99 Download ZIP Archive
5.1.3 / stable
July 21, 2017
[RELEASE] Release of crawler 5.1.3

[TASK] Added Cache to TypoScriptUtility
6.2.0 - 7.99.99 Download ZIP Archive
5.1.2 / stable
November 24, 2016
[RELEASE] Release of crawler 5.1.2

[BUGFIX] Scheduler Tasks broken after upgrade to 5.1.1 (composer) (#154)
[BUGFIX] Fixed incorrect class name in ClassAliasMap (#153)
6.2.0 - 7.99.99 Download ZIP Archive
5.1.1 / stable
November 21, 2016
[RELEASE] Release of crawler 5.1.1

[BUGFIX] It's not possible to "add process" after upgrade to 5.1.0 (#145)
[BUGFIX] #1442236317: MenuItem "" is not valid (More information) (#148)
[BUGFIX] Not possible to add new scheduler tasks (not only Crawler tasks
6.2.0 - 7.99.99 Download ZIP Archive
5.1.0 / stable
November 15, 2016
[RELEASE] Release of crawler 5.1.0

2016-11-08 28961a0 [FEATURE] Remove assignment of queue items to process if process gets killed by cleanup hook
2016-11-13 4d82941 [FEATURE] Adjust namespace of ProcessCleanupTask (#138)
2016-11-08 714a222 [CLEANU
6.2.0 - 7.99.99 Download ZIP Archive
5.0.9 / stable
September 16, 2016
[RELEASE] Release of crawler 5.0.9 6.2.0 - 7.99.99 Download ZIP Archive
5.0.8 / stable
September 16, 2016
[RELEASE] Release of crawler 5.0.8 6.2.0 - 7.99.99 Download ZIP Archive
5.0.7 / stable
September 06, 2016
New TER Release

git-svn-id: https://svn.typo3.org/TYPO3v4/Extensions/crawler/tags/V_3-5-0@67349 735d13b6-9817-0410-8766-e36946ffe9aa
6.2.0 - 7.99.99 Download ZIP Archive
5.0.6 / stable
August 22, 2016
New TER Release

git-svn-id: https://svn.typo3.org/TYPO3v4/Extensions/crawler/tags/V_3-5-0@67349 735d13b6-9817-0410-8766-e36946ffe9aa
6.2.0 - 7.99.99 Download ZIP Archive
5.0.5 / stable
August 15, 2016
New TER Release

git-svn-id: https://svn.typo3.org/TYPO3v4/Extensions/crawler/tags/V_3-5-0@67349 735d13b6-9817-0410-8766-e36946ffe9aa
6.2.0 - 7.99.99 Download ZIP Archive
5.0.4 / stable
March 18, 2016
New TER Release

git-svn-id: https://svn.typo3.org/TYPO3v4/Extensions/crawler/tags/V_3-5-0@67349 735d13b6-9817-0410-8766-e36946ffe9aa
6.2.0 - 7.99.99 Download ZIP Archive
5.0.3 / stable
March 10, 2016
New TER Release

git-svn-id: https://svn.typo3.org/TYPO3v4/Extensions/crawler/tags/V_3-5-0@67349 735d13b6-9817-0410-8766-e36946ffe9aa
6.2.0 - 7.99.99 Download ZIP Archive
5.0.2 / stable
January 26, 2016
New TER Release

git-svn-id: https://svn.typo3.org/TYPO3v4/Extensions/crawler/tags/V_3-5-0@67349 735d13b6-9817-0410-8766-e36946ffe9aa
6.2.0 - 7.99.99 Download ZIP Archive
5.0.1 / stable
December 08, 2015
New TER Release

git-svn-id: https://svn.typo3.org/TYPO3v4/Extensions/crawler/tags/V_3-5-0@67349 735d13b6-9817-0410-8766-e36946ffe9aa
6.2.0 - 7.99.99 Download ZIP Archive
5.0.0 / stable
October 27, 2015
New TER Release

git-svn-id: https://svn.typo3.org/TYPO3v4/Extensions/crawler/tags/V_3-5-0@67349 735d13b6-9817-0410-8766-e36946ffe9aa
6.2.0 - 7.99.99 Download ZIP Archive
4.1.1 / stable
March 10, 2016
New TER Release

git-svn-id: https://svn.typo3.org/TYPO3v4/Extensions/crawler/tags/V_3-5-0@67349 735d13b6-9817-0410-8766-e36946ffe9aa
4.5.0 - 6.2.99 Download ZIP Archive
4.1.0 / stable
October 27, 2015
New TER Release

git-svn-id: https://svn.typo3.org/TYPO3v4/Extensions/crawler/tags/V_3-5-0@67349 735d13b6-9817-0410-8766-e36946ffe9aa
4.5.0 - 6.2.99 Download ZIP Archive
4.0.0 / stable
August 11, 2015
New TER Release

git-svn-id: https://svn.typo3.org/TYPO3v4/Extensions/crawler/tags/V_3-5-0@67349 735d13b6-9817-0410-8766-e36946ffe9aa
4.5.0 - 6.2.99 Download ZIP Archive
3.5.0 / stable
October 22, 2012
- Update Documentation
- Add Multiprocessing CLI
- Add Feature to use sys_domain record for BaseURL
- Added UnitTests
- Enable Crawling with HTTPS
- Improve Scheduler Tasks
- Bug-Fixes
Download ZIP Archive
3.2.0 / stable
June 29, 2010
Improve scheduler integration, minor bug fixes. Download ZIP Archive
3.1.2 / stable
April 19, 2010
Improve handling with "t3lib_FlashMessage", Update documentation, Reimplement "Result Log" column. Download ZIP Archive
3.1.1 / stable
April 01, 2010
Fix documentation Download ZIP Archive
3.1.0 / stable
March 31, 2010
Update documentation
Improved scheduler and TYPO3 4.3 integration
Fixed several bugs
Add more tests
Download ZIP Archive
3.0.5 / beta
February 27, 2010
Release notes:

* TYPO3 4.3 integration
* Scheduler integration
* CLI API cleanup
* Bugfixes
Download ZIP Archive
3.0.0 / alpha
September 10, 2009
- multiprocessing
- new way to configure
- see manual for details
Download ZIP Archive
2.1.0 / alpha
July 24, 2008
diffrent options in em to control sleeptimes / change "keep-alive" header / add debugmode / readurl fixes
Download ZIP Archive
2.0.0 / alpha
March 23, 2007
Updated various features, see manual Download ZIP Archive
1.1.0 / alpha
March 21, 2006
Download ZIP Archive
1.0.0 / alpha
February 16, 2006
Download ZIP Archive
0.0.1 / alpha
December 22, 2005
[Dont download yet - its only for beta testing before 4.0] Download ZIP Archive