8.0.3

Summer time! 🌞 For some, that's sunshine, beaches, lotion, all that fun stuff. Or, well, hiding inside near the air conditioner. It's hot! For QuestDB, it's another set of improvements. This release contains a bundle of smaller improvements, with new features laying down major groundwork. And - of course - the usual motion of performance improvements. Hot indeed. 🔥

Upgrade! Apply liberal amounts of sun screen!

Breaking Changes 💥

• QuestDB no longer supports FreeBSD. We test major browser versions on each release. That means everything we write needs to account for multiple OS paradigms. FreeBSD has very small market share, and more often than not we'd trip over edge cases. To speed things up, we've made the decision to drop support. This means FreeBSD is no longer tested during development.

• NFS is an unsupported file system. NFS has been unsupported for some time, but only warnings were presented to the user. QuestDB will now refuse to start if running on an NFS disk. We recommend using ZFS. For a full list of supported filesystems, check out the Capacity Planning documentation.

New Features 🐣

• JSON Support: The json_extract() function is our first step towards robust JSON support. This function leverages the wonderful simdjson library written by Daniel Lemire. Simply store documents in a VARCHAR column, and then use the function to extract JSON fields into values. There is some performance overhead versus our native format, so we recommend using it to as a way to test a schema before putting the data into native columns. Read more in the documentation.

• New financial SQL Functions: mid price function and spread function for financial calculations. Simple. But effective.

• Smarter Web Console: Vague errors aren't helpful. The Web Console will now help you to troubleshoot common database configuration problems, with clear visual indicators to help you identify and resolve issues.

• More parallel execution: Enhanced GROUP BY operations and parallel execution, particularly benefiting queries with aggregate functions such as avg() and sum().

• HTTP Basic Authentication: QuestDB Open Source now offers Basic Auth to protect its HTTP endpoints. See how to configure this here.

Performance 🚀

• JIT-optimized IN(numeric values) operator for faster query execution.

• Optimized aggregate queries with the LAST function on designated timestamps.

• Speed improvements for self-join queries on symbol columns and more efficient pattern lookups for short ASCII patterns.

Bug Fixes 🐛

• Fixes for various issues including VARCHAR column queries via sqlx::postgres, invalid column errors in GROUP BY with JOIN, and more.

• Improved error handling, such as preventing ugly error messages when the log directory does not exist and ensuring REST interface returns HTTP 408 on timeouts.

• Ensured consistent escaping of VARCHAR fields in CSV exports.

• Improved compatibility with InfluxDB clients and integrations, by supporting Basic Authentication.

Pull requests

• fix(pgwire): Fix for querying VARCHAR columns via sqlx::postgres in Rust by @mtopolnik in #4685
• fix(core): prevent ugly error message when log directory does not exist by @jerrinot in #4669
• fix(pgwire): fix for using vararg functions with bind variables in Go by @glasstiger in #4695
• perf(sql): JIT-optimise IN(numeric values) operator by @zebs97 in #4661
• feat(http): implement HTTP basic auth by @jerrinot in #4672
• fix(sql): fix invalid column error in group by with join by @puzpuzpuz in #4637
• perf(sql): optimise aggregate query with LAST function on designated timestamp by @siddharth0815 in #4676
• fix(http): ensure REST interface consistently returns HTTP 408 on timeouts by @jerrinot in #4711
• fix(core): fix insert default column order after column type change by @ideoma in #4699
• fix(http): missing varchar escaping quotes in CSV export by @puzpuzpuz in #4735
• perf(sql): optimize l2price with up to 5 argument pairs by @puzpuzpuz in #4728
• fix(sql): fix to_date function to accept varchar type by @ideoma in #4740
• feat(sql): add spread function for finance by @sklarsa in #4744
• fix(core): 💥 breaking change - enforce fail-fast on NFS by @jerrinot in #4742
• perf(sql): speed up self-join queries on symbol columns by @puzpuzpuz in #4720
• feat(sql): add mid price SQL function by @sklarsa in #4734
• feat(sql): add variant of timestamp_floor supporting an offset origin by @nwoolmer in #4692
• perf(sql): more efficient pattern lookup for short ASCII patterns by @puzpuzpuz in #4706
• feat(sql): new json_extract function to extract JSON fields from varchars by @amunra in #4633
• feat(sql): increase coverage of SQL queries that use vectorized execution plan by @bluestreak01 in #4690
• fix(ilp): client - avoid spuriously sending duplicate data by @mtopolnik in #4762
• feat(core): table suspension reason in wal_tables() and warnings endpoint by @glasstiger in #4607

New Contributors

• @zebs97 made their first contribution in #4661

Full Changelog: 8.0.1...8.0.3

8.0.1

What's Changed

• fix(core): fix column type change after table truncate can lead to invalid data in the column by @ideoma in #4652
• fix(sql): fix SQL error when bind variables used within the IN() function by @glasstiger in #4657
• fix(ilp): make trailing semicolon in client configuration optional by @jerrinot in #4631
• feat(http): add support for nm=true query parameter in text export (/exp) to exclude metadata by @jerrinot in #4639
• fix(core): fix db crash after enabling dedup on varchar column by @ideoma in #4675
• feat(sql): reintroduce flush_query_cache function by @puzpuzpuz in #4641
• fix(core): fix potentially corrupt storage of STRING column after data inserting by @ideoma in #4681
• perf(sql): support timestamp literals in JIT compiler by @puzpuzpuz in #4658
• perf(sql): support ksum and nsum functions in parallel GROUP BY by @puzpuzpuz in #4682

Full Changelog: 8.0.0...8.0.1

8.0.0

This one's a big one!

In brief, to get you to the goods quicker:

• New VARCHAR data type over STRING default for ++compression & ++performance
• 50% improvement over already fast SQL query performance
• Data compression via system-level ZFS, previously Enterprise-only

And more...

Upgrade as soon as you can. Or start fresh and build out your use case.

If you're still reading, onwards for more detail, breaking changes, plus the PR breakdown.

---

New features 🐣

• VARCHAR: Introduces the VARCHAR data type and a new type driver API. VARCHAR offers UTF8 encoded strings of variable length as opposed to the UTF16 and fixed length encoding found in STRING, for better compression and significantly improved performance.
  • Maximum size of a single VARCHAR field size is 268 MB.
  • A size of a VARCHAR column in a single partition is limited by 218 TB or disk size, whichever is smaller.
  • STRING is now considered a legacy type.
• Data compression: Data compression for OSS! Full support for the ZFS file system and system-level data compression.
  • Read our guide to setup ZFS.
• ALTER COLUMN type: Use ALTER TABLE SQL to convert column types.
  • Migrate between STRING, SYMBOL, and VARCHAR data types.

Breaking changes 💥

• GLIBC upgrade: Upgraded GLIBC to a minimum of 2.28-amd64 and 2.33-aarch64.
• ILP and TextImport changes: ILP and TextImport now create VARCHAR instead of STRING columns.
• Jemalloc allocator: The default allocator is now jemalloc. Later updates include disabling jemalloc as the default allocator and removing jemalloc from no-jre builds.

Performance 🚀

• SQL performance improvement: Average SQL performance improves by 50%. 🔥
• Memory allocation limit: New memory allocation limit prevents OS OOM handler terminations in many cases.

Bug fixes 🥾

• SQL: Fixed issues with SAMPLE BY and GROUP BY not grouping when used with column aliases. Fixed WHERE filters with FLOAT columns null comparison.
• Core: Addressed trailing boundary processing of multipart HTTP requests. Fixed hour parsing and formatting in the 1-12 and 1-24 conventions.

Pull requests

• perf(sql): improve performance of length(VARCHAR) by @mtopolnik in #4442
• perf(sql): reduce query latency when querying tables under heavy writing by @ideoma in #4500
• perf(sql): statistics-based map pre-sizing for parallel group by by @puzpuzpuz in #4517
• perf(sql): speed up GROUP BY queries with ORDER BY and LIMIT by @puzpuzpuz in #4523
• perf(sql): optimize ORDER BY with LIMIT by @jerrinot in #4529
• perf(sql): adaptive work stealing for parallel query execution on multicore machines by @puzpuzpuz in #4542
• feat(core): breaking change💥 - upgrade GLIBC to minimum of 2.28-amd64 and 2.33-aarch64 by @bluestreak01 in #4449
• feat(core): use jemalloc allocator by default by @bluestreak01 in #4451
• feat(sql): implement level 2 order book price function by @nwoolmer in #4462
• feat(core): add ZFS support by @puzpuzpuz in #4507
• feat(sql): support usage of ALTER table SQL to convert column types by @ideoma in #4413
• feat(ilp): breaking change💥 - ILP and TextImport create VARCHAR instead of STRING columns by @jerrinot in #4476
• feat(build): new ami based on amazon linux 2023 💥 - breaking change by @mariusgheorghies in #4519
• feat(core): resume all suspended WAL tables on start up by @ideoma in #4513
• feat(core): add RSS memory limit to prevent OOM kills from the OS by @mtopolnik in #4480
• feat(sql): introduce global select query cache per interface by @puzpuzpuz in #4525
• feat(core): improve SQL error logging by @bluestreak01 in #4532
• fix(sql): fix sample by query compilation error when using as-of join by @amunra in #4452
• fix(core): fix trailing boundary processing of multipart HTTP requests by @glasstiger in #4466
• fix(http): remove ddl:ok response for insert query by @siddharth0815 in #4472
• fix(ui): web console to load regardless of pgwire disabled by @glasstiger in #4484
• fix(sql): fix WHERE filters with FLOAT columns null comparison by @ideoma in #4487
• fix(sql): minor bug fixes in ExpressionParser by @sivukhin in #4441
• fix(sql): IN with NULLs and STRINGs sometimes may return the wrong result by @nwoolmer in #4499
• fix(sql): fix SAMPLE BY and GROUP BY not grouping when used with column aliases by @ideoma in #4503
• fix(core): fix hour parsing and formatting in the 1-12 and 1-24 conventions by @jerrinot in #4506
• fix(sql): fix several bugs in aggregation (avg/sum/first_value) window functions by @sivukhin in #4429
• fix(sql): split_part returning old data when no match is found by @nwoolmer in #4493
• fix(sql): breaking💥 fix coalesce function correctness by @bluestreak01 in #4490
• fix(core): fix potential data loss when non-WAL table dropped and created with same name by @ideoma in #4533
• fix(sql): breaking change💥 - fix expression parser operator precedence by @sivukhin in #4443
• fix(core): fix transient query failure because of missing partition directory by @ideoma in #4611

New Contributors

• @siddharth0815 made their first contribution in #4472

Full Changelog: 7.4.2...8.0.0

7.4.2

Choice is essential. But isn't it nice when a clear, happy path emerges?

To that tune, we're pleased to announce QuestDB 7.4.2, with the usual mix of features, fixes and improvements.

In focus this week is the refinement of our updated ingestion clients. Built atop the InfluxDB Line Protocol (ILP) and leveraging HTTP instead of TCP, our clients and core database now offer a primary ingestion path that is robust, responsive and fast. They're available in popular languages, including Rust, Python, Java, Go, C & C++, and more.

It is quicker, cleaner and clearer to both get started and refine.

New Features 🐣

Fresh goods.

• Added bind variable support for long256 and IPv4 operators, milliseconds and microseconds support to datediff and dateadd.

• Enabled FILL(LINEAR) to support ALIGN TO CALENDAR.

Performance 🚀

Always a category unto its own.

• ILP: Optimized data ingestion for tables with a large number of columns.

• Core: Improved hash table lookup for small string keys and optimized memory usage to reduce OS kills.

• SQL: Faster SQL COPY and REST API CSV import, optimized binary search corner cases, and fixed the stuck parallel GROUP BY queries via better query timeout handling.

Bug Fixes 🥾

Squished, squashed & kiboshed.

• ILP: Addressed 'maximum buffer size exceeded' in ILP client and made config adjustments to disable individual flush types and support auto_flush_bytes. also corrected client processing of data received before TLS close_notification.

• SQL: Fixed incorrect results from lower, upper, substring functions when executed in parallel, erroneous SAMPLE BY behavior mixing asterisks, errors in CASE statements when ELSE branch is missing, incorrect results in GROUP BY queries, and SAMPLE BY queries with functions in the SELECT clause.

  Also fixed a bug for negative upper bound in BETWEEN operator, issues with null string insertions as designated timestamps, and inaccurate double comparisons. Finally, resolved stuck parallel group by queries due to incorrect timeout handling.

• Core: Addressed a rare segmentation fault during o3 commit of a non-WAL table, potential error rebuilding nascent data index, and incorrect partition size after an update.

New Contributors 🎉

@hoangsvit and @sivukhin made their first contributions. Thanks, both of you. 🙏

Pull requests

• fix(core): remove table 'lock' file on drop by @ideoma in #4341
• fix(sql): fix incorrect results returned when executing lower/upper/substring functions in parallel by @puzpuzpuz in #4337
• fix(sql): SAMPLE BY not to mix multiplication asterisk with wildcard column by @glasstiger in #4351
• perf(core): improve hash table lookup performance for small string keys such as column names and symbols by @puzpuzpuz in #4348
• fix(core): rare segfault during o3 commit of a non-wal table by @jerrinot in #4354
• fix(sql): fix error in some CASE statements when ELSE branch is missing by @bluestreak01 in #4367
• feat(core): optimise memory usage to reduce OS kills by @ideoma in #4368
• perf(ilp): optimize ingestion for tables with large number of columns by @puzpuzpuz in #4371
• fix(ilp): fix 'maximum buffer size exceeded' in ILP client by @jerrinot in #4372
• fix(sql): bug for negative upper bound (second argument) in BETWEEN operator by @sivukhin in #4375
• feat(ilp): sender config string can disable individual flush types separately by @jerrinot in #4385
• feat(ilp): client configuration string supports auto_flush_bytes by @jerrinot in #4391
• fix(sql): constant in explicit group by should not remove parallel optimisation by @nwoolmer in #4384
• fix(core): fix potential error rebuilding nascent data index by @bluestreak01 in #4383
• perf(sql): speed up SQL COPY and REST API CSV import by @puzpuzpuz in #4381
• feat(sql): add bind variable support for long256 and IPv4 operators by @puzpuzpuz in #4282
• feat(sql): add milliseconds and microseconds support to datediff by @javier in #4398
• feat(sql): add milliseconds and microseconds support to dateadd by @javier in #4396
• fix(http): client to process data received before TLS close_notification by @jerrinot in #4397
• fix(sql): fix incorrect results returned by GROUP BY queries with int or symbol key by @puzpuzpuz in #4353
• perf(chore): optimize binary search corner cases by @sivukhin in #4402
• fix(sql): fix incorrect results for parallel group by with symbol like filter by @puzpuzpuz in #4411
• fix(sql): internal error when inserting null string as a designated timestamp by @jerrinot in #4408
• fix(sql): fix stuck parallel group by query due to incorrect query timeout handling by @puzpuzpuz in #4420
• fix(core): fix possible issue with incorrect partition size after update by @bluestreak01 in #4412
• fix(sql): fix SAMPLE BY queries with functions in SELECT clause by @ideoma in #4404
• fix(sql): FILL(LINEAR) now supports ALIGN TO CALENDAR by @nwoolmer in #4392
• fix(sql): fix stuck parallel group by query due to incorrect query timeout handling by @puzpuzpuz in #4420
• fix(sql): fix inaccurate double comparison by @bluestreak01 and @eugenels in #4417

New Contributors

• @hoangsvit made their first contribution in #4358
• @sivukhin made their first contribution in #4375

Full Changelog: 7.4.0...7.4.2

7.4.1

What's Changed

We discovered SQL regression in 7.4.1. This regression does not effect any other version. This release is superceeded by 7.4.2

Full Changelog: 7.4.0...7.4.1

7.4.0

🌸 QuestDB 7.4 Release Notes

Ahh, Spring! Sunshine, flowers, all that nice stuff.

As the wheels of the seasons turn, so to the wheels of the applications we're all busy building.

Hopefully good things bloom for us all.

QuestDB is pleased to release version 7.4!

As you've come to expect, this release further improves core database performance. Also as per usual, several bugs and rough edges have been addressed. In important addition, there are note-able breaking changes that will provide a smoother overall experience and set us up well for future changes. Please address them prior to upgrade, and reach out to us if you run into any issues.

⚠️ New Features with breaking changes

• SAMPLE BY queries now use ALIGN TO CALENDAR by default. As such, if you're using SAMPLE BY and expecting ALIGN TO FIRST OBSERVATION, you'll need to update your queries to specify that syntax.

• Java client: The ILP Sender used in our first-party Java client now requires an explicit selection of Transport, (HTTP or TCP) for enhanced reliability and clarity in data ingestion workflows. You will need to pass this value forward. See the PR for more information. If you do not use the Java client, but use another language client, see your respective language client repository and update if needed.

• CREATE TABLE AS and INSERT INTO SELECT statements now operate non-atomically and are batched by default. If you require an atomic table, you can now use the very powerful sounding: CREATE ATOMIC TABLE. For more information, checkout the docs.

🚀 Performance Improvements

This release improves parallel GROUP BY query performance and also makes SAMPLE BY queries more efficient. In addition, we've reduced thread-local allocator overhead in single-threaded GROUP BY & SAMPLE BY queries. The first/last aggregate functions for string columns have also been optimized.

🐛 Bug Fixes

• Resolved issues leading to segmentation faults and incorrect results in certain SQL queries involving GROUP BY and SAMPLE BY clauses.
• Addressed a bug that could cause a table to be omitted from snapshots after significant DDL changes.
• Fixed HTTP request handling to prevent rejections when content-length is set to 0.
• Solved potential TLS handshake stalls on MacOS.
• Corrected the behavior of altering tables to add indexes that could inadvertently remove deduplication keys.
• Eliminated a crash during ingestion and various other stability issues.

What's Changed

• fix(sql): potential incorrect results in non-keyed parallel GROUP BY query by @puzpuzpuz in #4234
• fix(sql): segfault when running SAMPLE BY query with index-based filter and last/first functions by @puzpuzpuz in #4233
• fix(core): fix an issue where a table was not being included in the snapshot due to significant DDL changes being applied. by @ideoma in #4245
• fix(http): fix rejects of HTTP GET requests when content-length set to 0 by @ideoma in #4242
• fix(core): TLS handshake might occasionally get stuck on MacOS by @jerrinot in #4269
• fix(sql): fix incorrect unordered map creation check by @puzpuzpuz in #4255
• perf(sql): parallel GROUP BY support for first/last functions by @puzpuzpuz in #4267
• fix(sql): fix altering table to add index can remove dedup key by @ideoma in #4270
• feat(sql): parallel sample by implementation by @bluestreak01 in #4259
• perf(sql): speed up first/last aggregate functions for string columns by @puzpuzpuz in #4276
• fix(sql): fix a bug in indexed symbol comparison by @mtopolnik in #4273
• fix(core): fix wal table is suspended after invalid drop partition SQL by @ideoma in #4272
• fix(core): fix double-free in text loader by @bluestreak01 in #4278
• perf(sql): avoid redundant parsing of UUID bind variables by @puzpuzpuz in #4281
• feat(sql): breaking change💥 - change default behaviour of SAMPLE BY to ALIGN TO CALENDAR instead of ALIGN TO FIRST OBSERVATION by @nwoolmer in #4277
• fix(core): fix possible crash during ingestion (fuzz) by @mtopolnik in #4286
• fix(sql): qualified names in ORDER BY join queries no longer cause AssertionError by @nwoolmer in #4274
• perf(sql): remove thread-local allocator access overhead in single-threaded GROUP BY/SAMPLE BY queries by @puzpuzpuz in #4271
• feat(ilp): breaking change💥 - ILP Sender requires explicit Transport selection by @jerrinot in #4293
• fix(core): fix ignored JVM arguments in AMI by @petr-tichy in #4258
• fix(http): send http timeout response instead of disconnect on long csv export by @ideoma in #4300
• feat(sql): breaking change💥 - change default behaviour of CREATE TABLE AS/INSERT INTO SELECT statements to be non-atomic and batched by @nwoolmer in #4287
• fix(sql): fix potentially incorrect values produced by LPad/RPad functions by @eugenels in #4327
• fix(sql): fix ILIKE for upper case non-ASCII patterns by @puzpuzpuz in #4321
• fix(sql): fix occasional failures with first()/last()/first_not_null()/last_not_null() direct string function by @jerrinot in #4324

New Contributors

• @petr-tichy made their first contribution in #4258

Full Changelog: 7.3.10...7.4.0

7.3.10

Fresh from QuestDB 🐲

The 2024 Lunar New Year marks the Year of the Dragon. 🐉 When you think of the mighty dragon, what comes to mind? For us, well, we're a little different. Dragons evoke strength, wisdom, and - naturally - wild performance (we're performance obsessed).

Within that theme, the 7.3.10 release offers a 5x-10x speed-up in ASOF and LT JOIN queries, as well as a number of optimizations in parallel filters, GROUP BY, and within the JIT compiler. Further, UUID and LONG256 columns are now supported by parallel GROUP BY for efficient use of all hardware resources. For workloads that feature small, but frequent transactions you'll also see improved performance.

Familiar with InfluxDB? Interested in using InfluxDB Line Protocol (ILP), but with something more robust under-the-hood? This release provides 5x faster ingestion performance when multiple connections are in use, further increasing overall throughput strength compared to InfluxDB, especially in data with high cardinality. Whether you're looking to leverage ILP for high performance streaming, or upgrading an existing InfluxDB workload over to QuestDB, this update makes it easier.

Read more about "drop-in" InfluxDB migration in our recent blog.

Alongside the usual battery of bug fixes, this release also includes a new HyperLogLog-based approx_count_distinct() SQL function that is 3x-5x faster and more memory efficient than the existing count_distinct() function for high cardinality data sets. And finally, Write-Ahead Log (WAL) enabled tables are now the default table type. This means that the benefits of WAL & WAL-required features like deduplication are available to each newly created table.

Whether you're new to QuestDB, working in prod, or migrating existing time series workloads, this release has something for you.

Performance improvements

• perf(sql): support uuid and long256 in parallel GROUP BY by @nwoolmer in #4140
• perf(sql): avoid redundant disk reads in ORDER BY + LIMIT queries with parallel filter by @puzpuzpuz in #4148
• perf(sql): use JIT compiled filter in parallel GROUP BY by @puzpuzpuz in #4138
• perf(sql): speed up GROUP BY and JOIN in case of very small fixed-size keys by @puzpuzpuz in #4134
• perf(sql): add JIT compilation support for string/binary null check by @mtopolnik in #4177
• perf(sql): improve parallel group by detection when aliasing columns by @nwoolmer in #4190
• perf(sql): avoid full table scan in ASOF and LT JOIN with no additional columns by @puzpuzpuz in #4199
• perf(ilp): fix ingress perf issue for small, but frequent transactions by @bluestreak01 in #4221

New features

• feat(ilp): make ILP over HTTP transactional for single table by @ideoma in #4110
• feat(ilp): ILP client supports HTTP as a transport by @jerrinot in #4111
• feat(ilp): Influx Line Protocol client configuration via strings by @jerrinot in #4166
• feat(conf): add new configuration settings to server.conf (including Enterprise settings) by @amunra in #4175
• feat(sql): add approx_count_distinct() SQL function by @piotrrzysko in #4083
• feat(core): enable WAL tables by default by @bluestreak01 in #4223

Fixes and other improvements

• fix(http): fix CSV import for files larger than 2GB by @puzpuzpuz in #4132
• fix(http): fix HTTP protocol errors following non-successful requests by @ideoma in #4125
• fix(core): fix database stuck on startup after some table drops by @ideoma in #4152
• fix(core): fix wal resume exception after table suspended because of an error by @ideoma in #4162
• fix(core): communicate posix_fallocate's return value via errno by @eugenels in #4158
• fix(sql): fix exception when running insert as select by @ideoma in #4157
• fix(core): fix invalid table writer state after ALTER TABLE SQUASH PARTITIONS by @puzpuzpuz in #4153
• fix(core): disable query cancellation and timeout in WAL apply job by @bziobrowski in #4176
• fix(sql): fix NullPointerException in LATEST ON with nested query by @puzpuzpuz in #4192
• fix(core): fix errors / data inconsistencies after restoring from a snapshot by @ideoma in #4205
• fix(sql): enable parallel group-by on servers with fewer than 4 cores by @nwoolmer in #4208
• fix(core): fix snapshot restore to correctly restore tables when snapshot is taken under heavy DDL load by @ideoma in #4212
• fix(pgwire): fix a memory leak when using prepared statement and UUID columns by @jerrinot in #4214

New Contributors

• @mtopolnik made their first contribution in #4177

Full Changelog: 7.3.9...7.3.10

7.3.9

Welcome back from the holidays! (If you had one...)

In the latest QuestDB release, we've focused on enhanced stability and introduced two innovative features. Notably, we've improved GROUP BY performance tenfold through parallel SQL executions and introduced InfluxDB Line Protocol (ILP) support over HTTP, which is compatible with existing InfluxDB drivers. Those familiar with ILP will find a clean entryway into QuestDB.

Thanks to @ssharaev and @pompeiifreckles for their first contributions - we appreciate you.

Dive in to explore the release in detail. 🤿

What's changed

• 10x GROUP BY performance improvement via parallel SQL executions. Should you encounter any problems with this implementation, you can safely disabled [cairo.sql.parallel.groupby.enabled=false] it via configuration without downgrading the database instance.
• InfluxDB Line Protocol (ILP) support over HTTP. This implementation supports existing InfluxDB drivers and eases the transition into QuestDB for existing ILP users. The new client has control over transaction boundaries and will offer helpful feedback on transaction activity. We're also pleased to offer no noticeable performance impact in ingestions performance compared to our TCP implementation.

New features

• feat(sql): add sign() functions by @bziobrowski in #4050
• feat(ilp): ILP over HTTP, InfluxDB native clients writing support, ILP feedback by @ideoma in #4034
• feat(sql): add nullif(double, double) SQL function by @ssharaev in #4063
• perf(sql): parallel group by with optional filtering by @puzpuzpuz in #4032
• feat(sql): add query registry and query cancellation by @bziobrowski in #4066
• perf(sql): support advanced aggregate functions in parallel GROUP BY by @puzpuzpuz in #4097
• feat(http): add createTable parameter to /imp by @nwoolmer in #4108
• perf(sql): speed up GROUP BY queries with small fixed-size keys by @puzpuzpuz in #4128

Fixes and improvements

• fix(sql): fix handling of keyed string aggregation of first(), first_not_null(), last(), last_not_null() functions by @bziobrowski in #4068
• fix(core): fix non-existing partition drop can suspend WAL table by @ideoma in #4064
• fix(ilp): ilp@http small fixes by @glasstiger in #4077
• fix(sql): implement 'case expression when value' for float and double data types by @bziobrowski in #4046
• fix(sql): fix NullPointerException in replace() function with symbol arguments by @bziobrowski in #4073
• fix(core): fix tables() function permanently returning error after some DDLs by @ideoma in #4081
• fix(core): fix potential and unlikely numeric overlflow that could lead to incorrect query results by @pompeiifreckles in #4093
• fix(sql): fix potential races when regexp_replace function is used in where or group by clause by @puzpuzpuz in #4102
• fix(core): fix non-partitioned table corruption caused by truncate & table corruption after restore from snapshot by @bluestreak01 in #4095
• fix(core): fix edge case error when importing a large CSV file @jerrinot in #4103
• fix(sql): unregister query on cursor initialization failure by @bziobrowski in #4113
• fix(core): fix non-partition table truncate not working correctly alter table add column by @ideoma in #4109
• fix(core): reduce unnecessary error logging when table is dropped by @ideoma in #4121

New Contributors

• @ssharaev made their first contribution in #4063
• @pompeiifreckles made their first contribution in #4093

Full Changelog: 7.3.7...7.3.8

7.3.7

What's Changed

The hits keep comin'. Newly released QuestDB 7.3.7 adds supporting functions to pair with our recently released window functions. It also offers brings further improvements to SQL and ingestion performance - this has been a pattern of late!

Additional key features include:

• A 5x speed up of WAL Apply for small transactions
• New built-in functions for covariance, correlation, cumulative sum, and first value window
• Enhanced compatibility with Grafana

Thanks to @AjCH1 and @ksankeerth for their first contributions!

New features

• feat(core): 5x speed up of WAL Apply for small transaction and add SQL function to read WAL transactions metadata by @ideoma in #3992
• perf(sql): micro optimizations SqlKeywords implementation (short-circuit evaluation, inline variable) by @nitram509 in #3978
• feat(sql): information_schema.columns() function to provide list of columns across all tables in the database by @marregui in #3975
• feat(sql): add covariance and corr built-in functions by @charlespnh in #4012
• feat(sql): add cumulative sum and first value window funtions by @bziobrowski in #4001
• fix(core): fix failed SQL update not suspending WAL table potentially leading to a data loss by @ideoma in #4015
• perf(sql): execute complex sum aggregates on short column in parallel by @puzpuzpuz in #4024
• perf(sql): optimize order by designated timestamp and other columns with limit by @bziobrowski in #4025
• perf(sql): speed up select distinct symbol queries by @puzpuzpuz in #4030

Fixes and improvements

• fix(sql): fix symbol != value filter not applied by @bziobrowski in #4035
• fix(sql): fix edge case internal error in case-switch expression by @jerrinot in #4033
• fix(sql): fix column propagation for queries with set operations and distinct by @bziobrowski in #4028
• fix(core): include system tables into database snapshots by @puzpuzpuz in #4026
• fix(sql): improve compatibility with grafana by @bziobrowski in #4039
• fix(sql): occasionally failing queries when under rapid structural changes by @jerrinot in #4000
• fix(sql): fix parsing of single character window frame bounds by @bziobrowski in #4045

New Contributors

• @AjCH1 made their first contribution in #3384
• @ksankeerth made their first contribution in #4008

Full Changelog: 7.3.5...7.3.7

7.3.5

What's Changed

Following up our recent 7.3.4 release which provided Window Functions, Web Console improvements, and more, we're pleased to announce another significant release. Headlining 7.3.5 are the following key additions:

• Percentile function: Find an approximate percentile for a given series with approx_percentile(DD).
• Optimized queries: QuestDB has long stood out for its high performance ingestion. Not be left behind, queries also keep getting faster. In benchmarks, recent performance optimizations have shown 50% improvement to several query types.

New features

• feat(core): config to limit RSS memory used for writing by @ideoma in #3937
• feat(core): print insufficient Linux kernel limits on start-up by @puzpuzpuz in #3980
• feat(sql): implement approx_percentile function by @kevinmingtarja in #3900
• feat(sql): string_agg function support in keyed group by queries by @puzpuzpuz in #3991
• feat(sql): add first_not_null and last_not_null functions for GeoHash and IPv4 data types by @bziobrowski in #3929
• feat(sql) add show server_version by @nitram509 in #3901
• feat(sql): add show parameters SQL by @bluestreak01 in #3972
• perf(sql): optimize count_distinct symbol function by @puzpuzpuz in #3974

Fixes and improvements

• fix(sql): fix segfault in aggregate vectorized functions on zero input length by @puzpuzpuz in #3944
• fix(sql): maintain timestamp metadata in window factories by @bziobrowski in #3940
• fix(core): introduce separate settings for WAL writer pool by @puzpuzpuz in #3948
• fix(sql): improve count(*) and limit x,y performance by @bziobrowski in #3951
• fix(sql): fix create table like with dedup by @brunocalza in #3964
• fix(sql): fix several small sql issues by @bziobrowski in #3970
• fix(core): fix UPDATE timeout can leave non-WAL table in half updated state by @ideoma in #3946
• fix(core): reduce disk and memory footprint of system tables by @puzpuzpuz in #3945
• fix(core): recognize XFS file system as supported by @puzpuzpuz in #3968
• fix(core): handling of unexpected errors in SQL execution by @glasstiger in #3963