@@ -122,7 +122,7 @@ the feature can be combined with guest_memfd support in Firecracker.
122
122
123
123
### Limitations
124
124
125
- - High snapshot latency on 5.4+ host kernels due to cgroups V1 . We strongly
125
+ - High snapshot restoration latency when cgroups V1 are in use . We strongly
126
126
recommend to deploy snapshots on cgroups V2 enabled hosts for the implied
127
127
kernel versions -
128
128
[ related issue] ( https://github.com/firecracker-microvm/firecracker/issues/2129 ) .
@@ -145,10 +145,10 @@ the feature can be combined with guest_memfd support in Firecracker.
145
145
resumed from snapshot load memory on-demand from the snapshot and
146
146
copy-on-write to anonymous memory.
147
147
- Resuming from a snapshot is optimized for speed, while taking a snapshot
148
- involves some extra CPU cycles for synchronously writing dirty memory pages to
149
- the memory snapshot file. Taking a snapshot of a fresh microVM, on which dirty
150
- pages tracking is not enabled, results in the full contents of guest memory
151
- being written to the snapshot .
148
+ involves some extra CPU cycles for synchronously writing memory pages to the
149
+ memory snapshot file. Taking a full snapshot of a microVM results in the full
150
+ contents of guest memory being written to the snapshot, and particularly, in
151
+ all guest memory being faulted in .
152
152
- The _ memory file_ and _ microVM state file_ are generated by Firecracker on
153
153
snapshot creation. The disk contents are _ not_ explicitly flushed to their
154
154
backing files.
@@ -353,10 +353,12 @@ Enabling this support enables KVM dirty page tracking, so it comes at a cost
353
353
(which consists of CPU cycles spent by KVM accounting for dirtied pages); it
354
354
should only be used when needed.
355
355
356
- Creating a snapshot will ** not** influence state, will ** not** stop or end the
357
- microVM, it can be used as before, so the microVM can be resumed if you still
358
- want to use it. At this point, in case you plan to continue using the current
359
- microVM, you should make sure to also copy the disk backing files.
356
+ Creating a snapshot has some minor effects on the currently running microVM:
357
+
358
+ - The vsock device is [ reset] ( #vsock-device-reset ) , causing the driver to
359
+ terminate connection on resumption.
360
+ - On x86_64, a notification for KVM-clock is injected to notify the guest about
361
+ being paused.
360
362
361
363
### Resuming the microVM
362
364
@@ -381,8 +383,8 @@ ignored (microVM remains in the running state). **Effects**:
381
383
### Loading snapshots
382
384
383
385
If you want to load a snapshot, you can do that only ** before** the microVM is
384
- configured (the only resources that can be configured prior are the Logger and
385
- the Metrics systems) by sending the following API command:
386
+ configured (the only resources that can be configured prior are the logger and
387
+ the metrics systems) by sending the following API command:
386
388
387
389
``` bash
388
390
curl --unix-socket /tmp/firecracker.socket -i \
@@ -469,28 +471,10 @@ to the new Firecracker process as they were to the original one.
469
471
- _ on failure_ : A specific error is reported and then the current Firecracker
470
472
process is ended (as it might be in an invalid state).
471
473
472
- * Notes* : Please, keep in mind that only by setting to true
473
- ` enable_diff_snapshots ` , when loading a snapshot, or ` track_dirty_pages ` , when
474
- configuring the machine on a fresh microVM, you can then create a ` diff `
475
- snapshot. Also, ` track_dirty_pages ` is not saved when creating a snapshot, so
476
- you need to explicitly set ` enable_diff_snapshots ` when sending
477
- ` LoadSnapshot ` command if you want to be able to do diff snapshots from a loaded
478
- microVM. Another thing that you should be aware of is the following: if a fresh
479
- microVM can create diff snapshots, then if you create a ** full** snapshot, the
480
- memory file contains the whole guest memory, while if you create a ** diff** one,
481
- that file is sparse and only contains the guest dirtied pages. With these in
482
- mind, some possible snapshotting scenarios are the following:
483
-
484
- - ` Boot from a fresh microVM ` -> ` Pause ` -> ` Create snapshot ` -> ` Resume ` ->
485
- ` Pause ` -> ` Create snapshot ` -> ... ;
486
- - ` Boot from a fresh microVM ` -> ` Pause ` -> ` Create snapshot ` -> ` Resume ` ->
487
- ` Pause ` -> ` Resume ` -> ... -> ` Pause ` -> ` Create snapshot ` -> ... ;
488
- - ` Load snapshot ` -> ` Resume ` -> ` Pause ` -> ` Create snapshot ` -> ` Resume ` ->
489
- ` Pause ` -> ` Create snapshot ` -> ... ;
490
- - ` Load snapshot ` -> ` Resume ` -> ` Pause ` -> ` Create snapshot ` -> ` Resume ` ->
491
- ` Pause ` -> ` Resume ` -> ... -> ` Pause ` -> ` Create snapshot ` -> ... ; where
492
- ` Create snapshot ` can refer to either a full or a diff snapshot for all the
493
- aforementioned flows.
474
+ * Notes* : The ` track_dirty_pages ` configuration is not saved when creating a
475
+ snapshot, so you need to explicitly set ` track_dirty_pages ` again when sending
476
+ the ` LoadSnapshot ` command if you want to be able to do dirty page tracking
477
+ based diff snapshots from a loaded microVM.
494
478
495
479
It is also worth knowing, a microVM that is restored from snapshot will be
496
480
resumed with the guest OS wall-clock continuing from the moment of the snapshot
0 commit comments