Skip to content

Commit 28fb51d

Browse files
filinvadimvadim
andauthored
docs: Add pagination explanation to docs (#2134)
Resolves [#2133](#2133). **Summary** This addition to the documentation provides a detailed explanation and implementation of pagination using prefix scans for lexicographically sorted keys. Introduces a practical method for implementing pagination using: - A limit on the number of results returned per query. - A cursor (last processed key) to track the stopping point of the current iteration and resume subsequent queries. Includes a clear example of using cursors to manage iteration over keys in BadgerDB. Provides a complete Go implementation, showcasing how to perform prefix scans, manage pagination, and retrieve key-value pairs sequentially. This addition enhances the documentation by offering both theoretical insights and practical guidance, making it easier for developers to implement efficient, scalable, and sorted data retrieval mechanisms in their applications. Co-authored-by: vadim <filinvadim@pm.me>
1 parent ec1a9f3 commit 28fb51d

File tree

1 file changed

+83
-0
lines changed

1 file changed

+83
-0
lines changed

docs/content/get-started/index.md

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -431,6 +431,89 @@ db.View(func(txn *badger.Txn) error {
431431
})
432432
```
433433

434+
### Possible pagination implementation using Prefix scans
435+
436+
Considering that iteration happens in **byte-wise lexicographical sorting** order,
437+
it's possible to create a sorting-sensitive key. For example, a simple blog post
438+
key might look like:`feed:userUuid:timestamp:postUuid`. Here, the `timestamp` part
439+
of the key is treated as an attribute, and items will be stored in the corresponding order:
440+
441+
| Order ASC | Key |
442+
|:-----------:|:-------------------------------------------------------------|
443+
| 1 | feed:tQpnEDVRoCxTFQDvyQEzdo:1733127889:tQpnEDVRoCxTFQDvyQEzdo |
444+
| 2 | feed:tQpnEDVRoCxTFQDvyQEzdo:1733127533:1Mryrou1xoekEaxzrFiHwL |
445+
| 3 | feed:tQpnEDVRoCxTFQDvyQEzdo:1733127486:pprRrNL2WP4yfVXsSNBSx6 |
446+
447+
It is important to properly configure keys for lexicographical sorting to avoid
448+
incorrect ordering.
449+
450+
A **prefix scan** through the keys above can be achieved using the prefix
451+
`feed:tQpnEDVRoCxTFQDvyQEzdo`. All matching keys will be returned, sorted by `timestamp`.
452+
For the example above, sorting can be done in ascending or descending order based on
453+
`timestamp` or `reversed timestamp` as needed:
454+
455+
```go
456+
reversedTimestamp := math.MaxInt64-time.Now().Unix()
457+
```
458+
459+
This makes it possible to implement simple pagination by using a limit for
460+
the number of keys and a cursor (the last key from the previous iteration) to
461+
identify where to resume.
462+
463+
```go
464+
// startCursor may look like 'feed:tQpnEDVRoCxTFQDvyQEzdo:1733127486'.
465+
// A prefix scan with this cursor will locate the specific key where
466+
// the previous iteration stopped.
467+
err = db.badger.View(func(txn *badger.Txn) error {
468+
it := txn.NewIterator(opts)
469+
defer it.Close()
470+
471+
// Prefix example 'feed:tQpnEDVRoCxTFQDvyQEzdo'
472+
// if no cursor provided prefix scan starts from the beginning
473+
p := prefix
474+
if startCursor != nil {
475+
p = startCursor
476+
}
477+
iterNum := 0 // Tracks the number of iterations to enforce the limit.
478+
for it.Seek(p); it.ValidForPrefix(p); it.Next() {
479+
// The method it.ValidForPrefix ensures that iteration continues
480+
// as long as keys match the prefix.
481+
// For example, if p = 'feed:tQpnEDVRoCxTFQDvyQEzdo:1733127486',
482+
// it matches keys like
483+
// 'feed:tQpnEDVRoCxTFQDvyQEzdo:1733127889:pprRrNL2WP4yfVXsSNBSx6'.
484+
485+
// Once the starting point for iteration is found, revert the prefix
486+
// back to 'feed:tQpnEDVRoCxTFQDvyQEzdo' to continue iterating sequentially.
487+
// Otherwise, iteration would stop after a single prefix-key match.
488+
p = prefix
489+
490+
item := it.Item()
491+
key := string(item.Key())
492+
493+
if iterNum > limit { // Limit reached.
494+
nextCursor = key // Save the next cursor for future iterations.
495+
return nil
496+
}
497+
iterNum++ // Increment iteration count.
498+
499+
err := item.Value(func(v []byte) error {
500+
fmt.Printf("key=%s, value=%s\n", k, v)
501+
return nil
502+
})
503+
if err != nil {
504+
return err
505+
}
506+
}
507+
// If the number of iterations is less than the limit,
508+
// it means there are no more items for the prefix.
509+
if iterNum < limit {
510+
nextCursor = ""
511+
}
512+
return nil
513+
})
514+
return nextCursor, err
515+
```
516+
434517
### Key-only iteration
435518
Badger supports a unique mode of iteration called _key-only_ iteration. It is
436519
several order of magnitudes faster than regular iteration, because it involves

0 commit comments

Comments
 (0)