-
Notifications
You must be signed in to change notification settings - Fork 417
Closed
Labels
enhancementNew feature or request.New feature or request.t-toolingIssues with this label are in the ownership of the tooling team.Issues with this label are in the ownership of the tooling team.
Description
New feature would allow Crawlers to dynamically archive specific crawled pages as WARC files.
WARC are used for archiving pages and all their resources and can be used for various purposes. For example testing crawlers in development to avoid the need to test on real site, regression testing of crawlers, or just archiving the page for future reference.
Most likely this could be turned on in pre-navigation hook for each page that the user wants to record. This would redirect the request through proxy that handles the recording. For example using wayback in proxy recording mode and store the recorded WARC file in the KVS.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or request.New feature or request.t-toolingIssues with this label are in the ownership of the tooling team.Issues with this label are in the ownership of the tooling team.