Skip to content

Commit 5f9306c

Browse files
committed
fix(doc): Improve accurancy of snapshot documentation
Fix various minor errors: - Drop some specifics on the cgroups v1 disclaimer, because all supported host kernel versions are "5.4+" - Do not claim that creating a snapshot has no effect on the running VM, because that's not true. - Cut down on some repeated and confusing information / examples near the end. Signed-off-by: Patrick Roy <roypat@amazon.co.uk>
1 parent cd1a8c3 commit 5f9306c

File tree

1 file changed

+17
-33
lines changed

1 file changed

+17
-33
lines changed

docs/snapshotting/snapshot-support.md

Lines changed: 17 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -122,7 +122,7 @@ the feature can be combined with guest_memfd support in Firecracker.
122122

123123
### Limitations
124124

125-
- High snapshot latency on 5.4+ host kernels due to cgroups V1. We strongly
125+
- High snapshot restoration latency when cgroups V1 are in use. We strongly
126126
recommend to deploy snapshots on cgroups V2 enabled hosts for the implied
127127
kernel versions -
128128
[related issue](https://github.com/firecracker-microvm/firecracker/issues/2129).
@@ -145,10 +145,10 @@ the feature can be combined with guest_memfd support in Firecracker.
145145
resumed from snapshot load memory on-demand from the snapshot and
146146
copy-on-write to anonymous memory.
147147
- Resuming from a snapshot is optimized for speed, while taking a snapshot
148-
involves some extra CPU cycles for synchronously writing dirty memory pages to
149-
the memory snapshot file. Taking a snapshot of a fresh microVM, on which dirty
150-
pages tracking is not enabled, results in the full contents of guest memory
151-
being written to the snapshot.
148+
involves some extra CPU cycles for synchronously writing memory pages to the
149+
memory snapshot file. Taking a full snapshot of a microVM results in the full
150+
contents of guest memory being written to the snapshot, and particularly, in
151+
all guest memory being faulted in.
152152
- The _memory file_ and _microVM state file_ are generated by Firecracker on
153153
snapshot creation. The disk contents are _not_ explicitly flushed to their
154154
backing files.
@@ -353,10 +353,12 @@ Enabling this support enables KVM dirty page tracking, so it comes at a cost
353353
(which consists of CPU cycles spent by KVM accounting for dirtied pages); it
354354
should only be used when needed.
355355

356-
Creating a snapshot will **not** influence state, will **not** stop or end the
357-
microVM, it can be used as before, so the microVM can be resumed if you still
358-
want to use it. At this point, in case you plan to continue using the current
359-
microVM, you should make sure to also copy the disk backing files.
356+
Creating a snapshot has some minor effects on the currently running microVM:
357+
358+
- The vsock device is [reset](#vsock-device-reset), causing the driver to
359+
terminate connection on resumption.
360+
- On x86_64, a notification for KVM-clock is injected to notify the guest about
361+
being paused.
360362

361363
### Resuming the microVM
362364

@@ -381,8 +383,8 @@ ignored (microVM remains in the running state). **Effects**:
381383
### Loading snapshots
382384

383385
If you want to load a snapshot, you can do that only **before** the microVM is
384-
configured (the only resources that can be configured prior are the Logger and
385-
the Metrics systems) by sending the following API command:
386+
configured (the only resources that can be configured prior are the logger and
387+
the metrics systems) by sending the following API command:
386388

387389
```bash
388390
curl --unix-socket /tmp/firecracker.socket -i \
@@ -469,28 +471,10 @@ to the new Firecracker process as they were to the original one.
469471
- _on failure_: A specific error is reported and then the current Firecracker
470472
process is ended (as it might be in an invalid state).
471473

472-
*Notes*: Please, keep in mind that only by setting to true
473-
`enable_diff_snapshots`, when loading a snapshot, or `track_dirty_pages`, when
474-
configuring the machine on a fresh microVM, you can then create a `diff`
475-
snapshot. Also, `track_dirty_pages` is not saved when creating a snapshot, so
476-
you need to explicitly set `enable_diff_snapshots` when sending
477-
`LoadSnapshot`command if you want to be able to do diff snapshots from a loaded
478-
microVM. Another thing that you should be aware of is the following: if a fresh
479-
microVM can create diff snapshots, then if you create a **full** snapshot, the
480-
memory file contains the whole guest memory, while if you create a **diff** one,
481-
that file is sparse and only contains the guest dirtied pages. With these in
482-
mind, some possible snapshotting scenarios are the following:
483-
484-
- `Boot from a fresh microVM` -> `Pause` -> `Create snapshot` -> `Resume` ->
485-
`Pause` -> `Create snapshot` -> ... ;
486-
- `Boot from a fresh microVM` -> `Pause` -> `Create snapshot` -> `Resume` ->
487-
`Pause` -> `Resume` -> ... -> `Pause` -> `Create snapshot` -> ... ;
488-
- `Load snapshot` -> `Resume` -> `Pause` -> `Create snapshot` -> `Resume` ->
489-
`Pause` -> `Create snapshot` -> ... ;
490-
- `Load snapshot` -> `Resume` -> `Pause` -> `Create snapshot` -> `Resume` ->
491-
`Pause` -> `Resume` -> ... -> `Pause` -> `Create snapshot` -> ... ; where
492-
`Create snapshot` can refer to either a full or a diff snapshot for all the
493-
aforementioned flows.
474+
*Notes*: The `track_dirty_pages` configuration is not saved when creating a
475+
snapshot, so you need to explicitly set `track_dirty_pages` again when sending
476+
the `LoadSnapshot` command if you want to be able to do dirty page tracking
477+
based diff snapshots from a loaded microVM.
494478

495479
It is also worth knowing, a microVM that is restored from snapshot will be
496480
resumed with the guest OS wall-clock continuing from the moment of the snapshot

0 commit comments

Comments
 (0)