增加了大量爬取的功能，修改了爬取逻辑使得它能够直接爬取最新漏洞数据 #5
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
出于大量爬取漏洞数据库数据的需求，添加了大量爬取的功能：
原有数据库在访问大量数据时服务器会拒绝请求，因此使用循环来分批次爬取，实测每次100条，间隔1秒时，服务器不会拦截，成功爬取到了49000条数据
出于实际应用中优先爬取最新数据的需求，添加了爬取最新数据的逻辑：
先进行一次请求获取数据总数，再使用总数来构建url达到倒序爬取的目的
大二学生，第一次提PR，难免会有错漏之处，敬请您批评指正🙏
联系方式：
email: zhaofangming@mail.nwpu.edu.cn
qq: 1983517529