This project extracts cryptocurrency data from Yahoo Finance, processes the data, and stores the results in a SQLite database. It is a simple ETL pipeline implemented using Python and the Bonobo ETL framework.
The script will scrape crypto data from the target website, process the data, and store the results in a SQLite database located at database/crypto.db.
You can view the database using any SQLite database client.
Before running the project, you need to install the required Python libraries. To do this, navigate to the project directory and run the following command:
pip install -r requirements.txtfrom collections import Iterable has been deprecated but is still in use in the bonobo package -- to fix for local usage, modify the import statement in the file bonobo\config\processors.py to from collections.abc import Iterable
To run the ETL pipeline, execute the main script from the project directory:
python scripts/main.pyThis project is licensed under the terms of the MIT license.
Made with ❤️ by Shinno Taguchi