-
Notifications
You must be signed in to change notification settings - Fork 664
Open
Labels
P2Minor bugs or low-priority feature requestsMinor bugs or low-priority feature requestsPerformance 🚀Performance related issues and pull requests.Performance related issues and pull requests.new feature/request 💬Requests and pull requests for new featuresRequests and pull requests for new features
Description
You can provide a list of local files (but not a list of directories or s3 files) to read_parquet, as in #5698. For #5698 I will make a fix to read the files separately and concat the results. I attempted a more general solution but it required too much surgery on modin's read_parquet
code, which assumes in many places that there's just one file or directory to read. One difficulty I had is that ParquetDirectory
can take a list of files but not a list of directories, so we can't convert to a list of directories at the beginning.
Metadata
Metadata
Assignees
Labels
P2Minor bugs or low-priority feature requestsMinor bugs or low-priority feature requestsPerformance 🚀Performance related issues and pull requests.Performance related issues and pull requests.new feature/request 💬Requests and pull requests for new featuresRequests and pull requests for new features