In the previous article, we introduced how to make the ChatGPT bot reply based on the content of the article by adding blog content to the knowledge base. However, the method of manually copying article content is inefficient and very inconvenient to use.
To solve this problem, I developed a project called xlog-Archivist.
xlog-Archivist is a tool for automatically crawling article content and URLs from xlog-based blogs. With xlog-Archivist, we don't need to manually copy article content, it can automatically retrieve article content and export it in JSON format, making it easy for us to migrate blog content to the ChatGPT knowledge base. This greatly simplifies the process of knowledge acquisition, allowing the ChatGPT bot to learn and accumulate knowledge more quickly based on new content.
Usage#
- Clone the project locally or download the zip
git clone https://github.com/endercatone/xlog-Archivist.git - Install dependencies
pip install requests - Run the program
python main.py
Now you should be able to find articles and url.txt in the articles directory
After the first run, your blog URL will be saved in the configuration file, so you don't have to enter the URL next time.