-
-
Notifications
You must be signed in to change notification settings - Fork 18.8k
Open
Labels
BugIO SQLto_sql, read_sql, read_sql_queryto_sql, read_sql, read_sql_queryMultithreadingParallelism in pandasParallelism in pandas
Description
Code Sample, a copy-pastable example if possible
it is hard to generate a reproducible code for this problem as it would need parallel processes reading from a table, and another process that updates that table frequently.
Problem description
Line 646 in d3e5485
self.pd_sql.drop_table(self.name, self.schema) |
Line 647 in d3e5485
self._execute_create() |
The issue is that to_sql with (if_exists='replace') carries the two instructions separately and commits them serially into the database.
This causes 'table not found error' if multiple other processes are reading from the table as well.
The commit needs to happen after the second instruction only and not into the drop query.
Metadata
Metadata
Assignees
Labels
BugIO SQLto_sql, read_sql, read_sql_queryto_sql, read_sql, read_sql_queryMultithreadingParallelism in pandasParallelism in pandas