Skip to content

Releases: dragonflydb/dragonfly

v0.15.0

05 Feb 08:29
3449ba8
Compare
Choose a tag to compare

Dragonfly v0.15.0

Some prominent changes:

  • Dragonfly is now compatible with redis-sentinel, allowing to support H/A setups.
  • Debian package support for both x86_64 and aarch64.
  • Quite a few lua fixes - now Dragonfly works well with laravel
    framework - thanks to @fannyfan414 !
  • cl.throttle is now part of Dragonfly API - thanks to
    @zetanumbers! See https://github.com/brandur/redis-cell/ for more
    info about the API
  • Added support for ZUNION and BRPOPLPUSH

Huge thanks to all the contributors! ❀️

What's Changed

  • feat(server): report slaves ip and listening port in INFO REPLICATION by @ashotland in #715
  • fix(server): Fix Lua & Zset issue by @dranikpg in #716
  • fix(server): zunion now supports variadic arguments by @romange in #717
  • docs(readme): Add ulimit to docker-compose example by @fannyfan414 in #721
  • test(replica): test journal rewrite with multi shards by @adiholden in #720
  • fix(server): Fix multi tx cleanup by @dranikpg in #723
  • feat(action): build debian package for arm64 arch by @boazsade in #719
  • feat(string family): implement cl.throttle by @zetanumbers in #714
  • fix: a small warning fix. by @romange in #724
  • feat(server): Allow admin commands in multi by @ashotland in #722
  • fix(lua): Fix a deadlock happenning when calling a lua script by @romange in #726
  • Stop ping dev version by @romange in #727
  • chore(ci):Reduce --gtest_repeat to 10 by @ashotland in #730
  • feat(test): Sentinel Integration Test by @ashotland in #728
  • chore(tests): Sentinel test PR code review followup by @ashotland in #731
  • test(replication test): increase sleep time in full sync step by @adiholden in #733
  • test(regression test): run regression tests on github actions every 6… by @adiholden in #734
  • feat(server): Auto expiry + small replica refactor by @dranikpg in #718
  • Add -x to pytest invocation by @ashotland in #735
  • fix(sentinel_test):Remove return type which breaks regression tests by @ashotland in #736
  • feat(action): build debian package for arm64 arch (#719) by @boazsade in #738
  • Add renderer to grafana by @romange in #739
  • feat(list family): support blocking command for replication by @adiholden in #740
  • test(replication test): check data only after replica finished execution by @adiholden in #746
  • feat(server): Print time it took to perform full sync by @dranikpg in #747
  • fix(server): Reset interpreter stack on errors by @dranikpg in #745
  • fix(regression-tests):Skip sentinel regression tests by @ashotland in #748
  • Notify on cancelled too by @ashotland in #749
  • Exclude single threaded test now that expiry is tested by @dranikpg in #750

New Contributors

Full Changelog: v0.14.0...v0.15.0

v0.14.0

20 Jan 05:30
49e53d5
Compare
Choose a tag to compare

Dragonfly v0.14.0

Some prominent features and fixes.

  • Fixed a bug that caused linker incompatibility on some Linux distributions.
  • Release of a Debian package (currently only for x86_64 architectures) - now it's much easier to install Dragonfly as a service on a linux machine.
  • http_admin_console flag is renamed to primary_port_http_enabled. This controls whether --port accepts http requests as well. Also, we added new flags like admin_port and admin_bind to listen on an additional port. Please note that admin_port always exposes the HTTP interface #709
  • Replication is almost ready - one can set up a master/slave setup with a usual "replicaof" command on a secondary replica (FAILOVER is not supported yet)

Huge thanks to all the contributors! ❀️

What's Changed

  • feat(server): Add metrics by @ashotland in #638
  • feat(replica): support multi transaction command by @adiholden in #634
  • feat(chart): add feature to generate/request TLS certificate through cert-manager by @tamcore in #644
  • feat(container): make healthcheck tls compatible by @tamcore in #643
  • Update README.md by @yingshaoxo in #633
  • feat(tools): Add Zipfian cache testing tool by @devangjhabakh in #640
  • fix(lua): fix a bug that truncates the last lua argument. by @romange in #652
  • chore(server): snapshot stats fix by @romange in #653
  • feat(replica): support multi shard commands new flow by @adiholden in #645
  • feat(helio): pull blocking counter fix by @adiholden in #655
  • fix(server): Fix bug in flushing in snapshot by @dranikpg in #654
  • fix(server): Fix db initialization in stable state by @dranikpg in #658
  • fix(server): Flush dbs before full sync by @dranikpg in #659
  • feat(server family): reject flushall and flushdb on multi command by @adiholden in #657
  • feat(replica): enable multi cmd sync on flag by @adiholden in #656
  • bug(replica): multi command on single shard exec all commands by @adiholden in #661
  • Pytests overhaul by @dranikpg in #569
  • fix(build): verify build image for releases by @boazsade in #664
  • feat(server family): impl ROLE command by @adiholden in #660
  • feat(server): implement BRPOPLPUSH command by @romange in #671
  • bug(snapshot): on journal write entry with opcode COMMAND and MULTI_C… by @adiholden in #662
  • fix(server): Fix shutdown order by @dranikpg in #675
  • feat(server): Json rdb support by @dranikpg in #674
  • fix(server): Fix a bug with brpoplpush by @romange in #677
  • fix(tests): Small pytest fixes by @dranikpg in #673
  • fix(server): Fix replication on a single thread by @dranikpg in #676
  • test(replication_test) : add replica flushall test by @adiholden in #669
  • test(rdb): Add json dump & restore test by @dranikpg in #678
  • feat(server): Buffered streamer + use on stable state by @dranikpg in #639
  • fix(core): Fix compact object bug by @dranikpg in #680
  • fix(test): reduce flakiness of tiered_storage_test by @romange in #684
  • chore(facade): speed up request allocation in pipeline mode by @romange in #682
  • fix(server): fix the bug where DF went over maxmemory limit by @romange in #683
  • feat(server): Rewrite journal commands (basic) by @dranikpg in #651
  • fix(server): Convert Heartbeat from a periodic task to a fiber by @ashotland in #690
  • feat(build): debian package building by @boazsade in #689
  • fix(server): Fix connection bug by @dranikpg in #695
  • feat(set family): rewrite set store commands to journal by @adiholden in #691
  • feat(zset family): implement a variant of ZUNION command issue #356 by @adiholden in #686
  • Refactor transaction by @dranikpg in #697
  • feat(actions): add support for debian packaging by @boazsade in #696
  • (generic family): journal support rename command by @adiholden in #698
  • feat(actions): add support for Debian packaging - add missing dependency by @boazsade in #701
  • feat(transaction): simplify calc multi trans unique shard count by @adiholden in #672
  • feat(build): debian package building fix dependency bug by @boazsade in #702
  • feat(server): Journal rewrite p2 + tests by @dranikpg in #699
  • (generic family): journal support move command by @adiholden in #708
  • feat(list family): impl rename journal commands by @adiholden in #704
  • fix(server): fix getex command with persist can get time overflow by @boazsade in #707
  • feat(server): admin port and interface by @boazsade in #709

New Contributors

Full Changelog: v0.13.1...v0.14.0

v0.13.1

02 Jan 11:12
f7e70bc
Compare
Choose a tag to compare

Dragonfly v0.13.1

Minor bugfix

Don't miss the previous v0.13.0 Highlights:
The first release of the year! Yey!

This release contains a few exciting features.

  • Active defrag: any long-running process with non-trivial allocation patterns suffers from excessive memory usage due to external fragmentation. This release uses an improved defragmentation algorithm after we released the initial version in the previous version.
  • Experimental SSD-based data-tiering support. Dragonfly is the only in-memory store that provides data tiering that is faster than other memory-only stores. (Currently only for strings). Try it with flag --spill_file_prefix=<nvme_file_basepath>
  • JSON - this release compliments the already existing JSON API with support for SCAN and TYPE working with JSON data types.
  • Replication (still not for prod use) but can already be tried on simple single key commands like SET, SADD, ZADD, LPUSH etc.
  • Writes and reads of large string values are optimized for x86 architectures with SIMD instructions.
  • Numerous improvements to helm templates, plus docker health checks - thanks to @tamcore !

What's Changed

New Contributors

Full Changelog: v0.13.0...v0.13.1

v0.13.0

01 Jan 20:06
51a78a3
Compare
Choose a tag to compare

Dragonfly v0.13.0

The first release of the year! Yey! πŸ‘―

This release contains a few exciting features.

  • Active defrag: any long-running process with non-trivial allocation patterns suffers from excessive memory usage due to external fragmentation. This release uses an improved defragmentation algorithm after we released the initial version in the previous version.
  • Experimental SSD-based data-tiering support. Dragonfly is the only in-memory store that provides data tiering that is faster than other memory-only stores. (Currently only for strings). Try it with flag --spill_file_prefix=<nvme_file_basepath>
  • JSON - this release compliments the already existing JSON API with support for SCAN and TYPE working with JSON data types.
  • Replication (still not for prod use) but can already be tried on simple single key commands like SET, SADD, ZADD, LPUSH etc.
  • Writes and reads of large string values are optimized for x86 architectures with SIMD instructions.
  • Numerous improvements to helm templates, plus docker health checks - thanks to @tamcore !

Huge thanks to all the contributors! ❀️

What's Changed

  • feat(rdb save): Create compressor interface by @adiholden in #538
  • fix(helm): Chart.yaml - Add home, keywords, sources, kubeVersion by @arukiidou in #542
  • feat: add healtcheck for container by @tamcore in #544
  • Support lz4 compression by @adiholden in #545
  • feat(server): json set type support by @boazsade in #546
  • chore: Update helio dependency by @romange in #553
  • chore: Update helio dependency by @dranikpg in #555
  • feat(server): Implement robust error & cancellation on replica by @dranikpg in #531
  • feat(ci test): add build and test on release mode by @adiholden in #556
  • added example of start script by @Tomato6966 in #557
  • Support offloading of blobs longer than 2KB by @romange in #559
  • feat(server): active memory defrag test application by @boazsade in #552
  • feat(server): JSON family using JSON type by @boazsade in #561
  • feat(server): New auto-journal types/read/write by @dranikpg in #560
  • chore: Refactor RdbLoad by @romange in #564
  • chore: Adding a mpsc intrusive queue based on Vyukov's design by @romange in #562
  • feat: introduce simd algorithm for bitpacking by @romange in #568
  • feat: implement ascii_unpack using SIMD instructions by @romange in #573
  • bug(rdb load): Rdb loader tasks stop running on failure #567 by @adiholden in #576
  • chore: Improve the implementation of simd based packing by @romange in #577
  • fix: choose an appropriate SIMD implementation for aarch64 by @romange in #579
  • fix: handle corner case when comitted memory is negative by @romange in #570
  • feat(server): Use new journal format by @dranikpg in #563
  • fix(chart): probe indentation by @tamcore in #590
  • feat(server): Redesign and simplify tiered storage module. by @romange in #589
  • use dense_set for hashmap family by @romange in #593
  • feat(server): adding version monitoring task by @boazsade in #586
  • feat(replica): Support FlushDB command for replication #580 by @adiholden in #591
  • fix(server): defrag unit test fix by @boazsade in #602
  • chore(server): upgrade mimalloc for v2.0.7 by @boazsade in #597
  • feat: build and publish weekly alpha/development container image by @tamcore in #594
  • life of a transaction by @romange in #278
  • feat(server): track hits and misses stats. by @ashotland in #608
  • feat(server): Improved cancellation by @dranikpg in #599
  • feat(replica): atomicity for multi shard commands by @adiholden in #598
  • fix(server):set of existing object with expiry by @ashotland in #609
  • chore: Add tiered_storage_test. by @romange in #613
  • feat(tools):Cache logs player by @ashotland in #614
  • fix(tools): cache logs player handle append as redis append by @ashotland in #615
  • docs(quickstart): Clarify docker command for macOS by @pjambet in #612
  • feat(tools):cache log player batching all the way optimisation by @ashotland in #617
  • feat(ci test): add testing for helm chart by @tamcore in #622
  • chore: overhaul chart by @tamcore in #620
  • Cache logs player - add one last stats print after completion by @ashotland in #623
  • feat(server): Buffered journal serializers by @dranikpg in #619
  • feat(server): Update helio, optimize and clean up rdb/snapshot by @dranikpg in #625
  • fix(server): only print version if successfully connect. by @boazsade in #628
  • feat(server): using memory defrag with per shard mem info by @boazsade in #616

New Contributors

Full Changelog: v0.12.0...v0.13.0

v0.12.0

07 Dec 04:21
2d83aa9
Compare
Choose a tag to compare

Dragonfly v0.12.0

This release is mainly about bug fixes and extending our codebase to support replication.

  • Dragonfly can now connect as a replica behind Redis master (supports REPLICAOF command)
  • Dragonfly can now connect to another Dragonfly instance to form DF->DF replication. This feature is not fully implemented yet but
    it performs the first part, namely "full sync stage". Now we are working on "stable state replication".
  • Dragonfly now supports the active-defrag feature. See #448 for more details.

Huge thanks to all the contributors! ❀️

What's Changed

  • chore: Rename DoFlush to Drakaris. by @romange in #489
  • feat: Add last_save_duration_sec metric by @romange in #493
  • fix(server): Handle data directory errors by @dranikpg in #494
  • fix: Fix bugs around rdb save and improve memory accounting by @romange in #495
  • A dedicated github actions pipeline that build dragonfly and runs pyt… by @zNNiz in #490
  • fix(test): skip pytest test_simple_full_sync_mutli_crash due to crash by @boazsade in #497
  • fix(server): Fix db_slice insertion bug with callbacks by @dranikpg in #499
  • feat(actions): push helm chart as OCI image by @tamcore in #500
  • feat(server): Switch to stable state replication by @dranikpg in #473
  • feat(io): pull new helio dependency by @adiholden in #506
  • fix(server): all scan commands needs to return cursor as bulk string #503 by @boazsade in #504
  • fix(server): Fix lambda capture in RunInParallel functions by @dranikpg in #507
  • fix: clang warnings. by @romange in #508
  • feat(server): Replication errors & cancellation by @dranikpg in #501
  • fix(server): Increase common abstraction usage by @dranikpg in #511
  • feat(server): implement json.set command (#104) by @iko1 in #514
  • chore: tiered_storage - cleanups and refactorings. by @romange in #515
  • fix(server): Fix replication bug, add gdb opt to pytest by @dranikpg in #513
  • bug(server): exit not crash when port in use by @adiholden in #516
  • feat(rdb save): add blob compression on snapshot by @adiholden in #505
  • feat(rdb save): add unit test to save df with all compression levels by @adiholden in #517
  • feat(rdb save): serializer flush to sink only on flush call by @adiholden in #519
  • fix: Use dev image for running regression test by @romange in #522
  • feat(regression test): add timeout cpu info and inc frequecy by @adiholden in #524
  • bug(rdb loader): When reading from zstd uncompressed buf skip ensure … by @adiholden in #525
  • refactor(server): Refactor snapshot by @dranikpg in #520
  • chore: Add lz4 to the source dependency by @romange in #528
  • feat(server): memory defrag support - unit tests added to verify #448 by @boazsade in #523
  • bug(debug cmd): debug populate support random value population by @adiholden in #527
  • bug(rdb save): snapshot: on push data to channel check serializer len by @adiholden in #532
  • feat: add defrag support for small strings by @romange in #534
  • feat(server): adding matrices for memory defrag by @boazsade in #535
  • feat(rdb save): refactor move zstd serializer under rdb serializer by @adiholden in #533
  • feat(server): defrag memory counter for total memory access by @boazsade in #537
  • feat(rdb save): add Lz4Compressor by @adiholden in #536
  • fix(github): add libzstd-dev dep to aarch64 release pipeline by @romange in #539

New Contributors

Full Changelog: v0.11.0...v0.12.0

v0.11.0

14 Nov 07:38
b395834
Compare
Choose a tag to compare

Dragonfly v0.11.0 - Thanksgiving edition

Some prominent changes:

  • You can now pass --maxmemory=16G flag instead of passing raw bytes.
  • The monitor command is implemented; it's also much faster than in Redis.
  • The missing options in existing commands were added. Amazing work by our contributors!
  • Significant progress was made with the replication. Still, lots needs to be done in this area. Expected release for this feature - Q1 23.

Huge thanks to all the contributors! ❀️

What's Changed

  • feat(server): sending and dispatching of messages using different internals by @boazsade in #424
  • ci(workflow): fixing discord notification link by @ranrib in #419
  • feat(server): Allow sampling arena stats from a specified thread by @romange in #428
  • Monitor command by @boazsade in #427
  • feat(server): Multi rdb load by @dranikpg in #430
  • refactor(server): Refactor and tidy dflycmd by @dranikpg in #432
  • feat(server): monitor command should return OK on creation (#344) by @boazsade in #433
  • refactor(server): Separate start & stop in rdb save by @dranikpg in #437
  • fix: dragonfly compilation failure due to glibc version less than 2.30 by @Super-long in #434
  • feat(server): monitor command - cover by tests by @boazsade in #435
  • fix: "wrong number of arguments" must be in lower case by @lsvmello in #439
  • refactor(tests): Refactor pytest by @dranikpg in #449
  • docs(build): Canonicalize build from source instructions. Add CLA terms by @romange in #454
  • feat(server): implement TOUCH command by @lsvmello in #444
  • feat(server): implement json.mget command (#104) by @iko1 in #455
  • feat(server): Support for LIMIT and REV in ZRANGE #422 by @RedhaL in #456
  • fix(list): Fixes blpop failure. by @romange in #462
  • ci: added clang pre-commit hook by @lsvmello in #461
  • feat(tests): ioredis being able to successfully run tests, make it ready to be part of CI by @boazsade in #459
  • feat(server): Basic capped full sync by @dranikpg in #440
  • feat(server) implement pexpire command by @b0bleet in #464
  • server: implemented json.arrappend command by @b0bleet in #460
  • feat(server): simplify handling of the pub message handling by @boazsade in #465
  • refactor(server): Refactor replica by @dranikpg in #472
  • fix(server): reject eval inside transaction multi blocks #457 by @boazsade in #471
  • feat(server): implement json.debug command (#104) by @iko1 in #463
  • fix(server): issue with invalid command inside multi #468 by @boazsade in #469
  • SSCAN command support count and match parameters by @adiholden in #466
  • bug(dense set): fix scan function to return only home bucket data by @adiholden in #474
  • fix: maxmemory flag support human-readable format by @lsvmello in #476
  • fix(server): Fix bad maxmemory casting by @romange in #477
  • fix(server): Fix snapshot saving bug by @dranikpg in #479
  • feat(server): add support for KEEPTTL options with SET command #389 by @Super-long in #478
  • fix: DCHECK_NOTNULL gets WARNING when return value is not used. by @Super-long in #483
  • fix(server): Fix path concatenation in save by @dranikpg in #481
  • feat(server): implement json.resp command (#104) by @iko1 in #482
  • feat(server family): add connection set name command fixes #458 by @adiholden in #485

New Contributors

Full Changelog: v0.10.0...v0.11.0

v0.10.0

20 Oct 15:04
10c3d9f
Compare
Choose a tag to compare

Dragonfly 0.10.0

Hacktober edition

This release was unquestionably the result of community collaboration.
We are incredibly excited and honored to welcome all first-time contributors.

  • Quite a few commands from 5.x, 6.x APIs were added by the community (see below).
  • Support for dump/restore commands.
  • Not so slowly but quite surely @iko1 added more json commands. This time - array related.
  • bug fixes and general improvements.

Huge thanks to all the contributors! ❀️

What's Changed

New Contributors

Full Changelog: v0.9.1...v0.10.0

v0.9.1

07 Oct 15:34
608a804
Compare
Choose a tag to compare

Dragonfly 0.9.1

A patch release fixing a possible deadlock when running with epoll API.

What's Changed

  • feat(server): Implement MEMORY MALLOC-STATS command by @romange in #363
  • chore(test): Allow running tests with epoll proactor by @romange in #364
  • fix(epoll): pull newest helio dependency by @romange in #366

Full Changelog: v0.9.0...v0.9.1

v0.9.0

04 Oct 15:58
8f5500a
Compare
Choose a tag to compare

Dragonfly 0.9.0

YOM KIPPUR edition

This is quite significant release with lots of interesting features:

  • Epoll support - now we support all Linux kernels !!!
    Dragonfly automatically falls back to epoll API if iouring API is not available for some reason.
    This can also reduce friction when running Dragonfly on k8s (no need to use --privileged flag anymore).
    If you want to force epoll instead of iouring you can do it with --force_epoll flag.
  • New Dragonfly specific command SADDEX key seconds member member ... allows adding to sets members with TTL!
  • CPU usage is greatly reduced when Dragonfly is in idle mode (~10x reduction).
  • Dragonfly can connect to Redis server in SLAVE mode (using replicaof command)
  • MOVE, SORT and PERSIST commands are now supported.

Huge thanks to all the contributors! ❀️

What's Changed

  • feat(server): adding support for bitops #213 by @boazsade in #295
  • Added more useful flags to the flags section by @Tomato6966 in #305
  • feat(server): adding support for EXAT PXAT option at set command by @boazsade in #306
  • feat(server): Implement MOVE command by @dranikpg in #298
  • feat(docs): Update broken link to grafana dashboard by @odedponcz in #314
  • chore(server): update license text by @romange in #312
  • feat(server): bitcount commands support and unit tests #213 by @boazsade in #315
  • chore(server): rdb save can now save into tcp socket directly. by @romange in #317
  • feat(packaging): create deb packaging #249 by @acheevbhagat in #313
  • fix(bug): dashtable split crashes when moving items from the old segment by @romange in #318
  • feat(server): SORT command prototype by @dranikpg in #311
  • fix(server): Fix ContainerEntry union initialization by @dranikpg in #323
  • feat(server): Introduce transaction clock by @romange in #329
  • fix(server): dfly_main.cc now compiled with march=core2. by @romange in #328
  • feat(server): implement json.objkeys command (#104) by @iko1 in #327
  • feat(server): support for bitop command - unit tests added #213 by @boazsade in #319
  • feat(server): Enforce decomitting memory pages upon flushdb call by @romange in #337
  • chore(DenseSet): Some code reorder without change in functionality. by @romange in #338
  • feat(server): zrange family command support INF as case insensitive #326 by @boazsade in #336
  • chore(denseset): Simplify DenseSet, remove empty links. by @romange in #339
  • Add expiry semantics to the set. by @romange in #340
  • chore(hset): Get rid of helper functions in header file. by @romange in #341
  • chore(server): reduce iouring exposure across the codebase by @romange in #342
  • feat(setfamily): Add SADDEX command by @romange in #348
  • fix(server): better support for the --help option by @boazsade in #349
  • fix(server): hscan command better support #347 by @boazsade in #350
  • feat(server): support epoll linux api by @romange in #351
  • feat(server): implement json.strappend & json.clear commands (#104) by @iko1 in #353
  • feat(server): support for persist command #345 by @boazsade in #354

New Contributors

Full Changelog: v0.8.0...v0.9.0

v0.8.0

18 Sep 13:33
Compare
Choose a tag to compare

Dragonfly 0.8.0

A nice release, not very big but things are moving in the right direction.

  1. Transactional support is extended to WATCH/UNWATCH by @dranikpg
  2. A new SET data-structure was introduced by @braydnm . It can potentially provide for up to 50% memory savings when using small records in the set. See dense set doc for more details. You can start using the upgraded set with --use_set2 flag when running dragonfly.
  3. JSON.DEL was added to JSON family by @iko1 .

Huge thanks to all the contributors! ❀️

What's Changed

Full Changelog: v0.7.0...v0.8.0