-
Notifications
You must be signed in to change notification settings - Fork 10
Open
Description
Hi,
I have a big issue when using the QDB python script to save a pd.DataFrame which has the same schema each time, but I don't know what the schema is ahead of time (ie. when I create the table). This is because I use it to save thousands of tables.
I use the below code to save new and old data.
If the table exists - it appends to it. If it does not, it creates it and appends to it.
import pandas as pd
from questdb.ingress import Sender
df = pd.DataFrame({
'id': pd.Categorical(['toronto1', 'paris3']),
'temperature': [20.0, 21.0],
'humidity': [0.5, 0.6],
'timestamp': pd.to_datetime(['2021-01-01', '2021-01-02'])})
conf = f'http::addr=localhost:9000;'
with Sender.from_conf(conf) as sender:
sender.dataframe(df, table_name='sensors', at='timestamp')
This causes a few issue:
- upon creation it always creates a partition by
DAY
Feature request: add a parameter which is by default DAY but can be changed by the user to be MONTH, NONE or other partition frequency
- It silently creates a table by default, even if one does not exist.
Feature request: add a parameter which is by defaultTrue(to create a table) but can be also set toFalsesuch that if a table does not exist, it will not create one.
This is very large pain point to use this py-questdb-client in a more powerful way. For daya which is daily, monthly or even annual, it creates a large burden.
newskooler and sriram-V20
Metadata
Metadata
Assignees
Labels
No labels