ZFS supposed independant clones

[B3r3n06]

Hello,

Issue

ZFS provides a dataset cloning feature.
Technically, the procedure is the below:
zfs create MYdataset
zfs snapshot MYdataset@MYmark
zfs clone MYdataset@MYmark MYNEWdataset
zfs promote MYNEWdataset

From this, MYNEWdataset is supposed totally independant from its origin MYdataset.

However, when the clone becomes useless and to be destroyed, it appears a bug:
zfs destroy MYNEWdataset
cannot destroy 'MYNEWdataset': filesystem has children
use '-r' to destroy the following datasets:
MYNEWdataset@MYmark

zfs destroy -r MYNEWdataset
cannot destroy 'MYNEWdataset': filesystem has dependent clones
use '-R' to destroy the following datasets:
MYdataset

So enforcing the action will delete the original dataset, still used.
From this, there is no escape but destroying original dataset for real and rebuild it.

System information

This happens on ZFS on Ubuntu 18.04.5 and FreeBSD 12.

Versions on Ubuntu:
ZFS version: 0.7.5-1ubuntu16.10
SPL version: 0.7.5-1ubuntu2.2

[churchers]

As far as I can see this is expected behaviour.

A clone does not duplicate any data (that's the whole point), and so it is dependent on the original dataset.
In your case, when MYNEWdataset is created, it relies on MYdataset. All the promote command does it reverse the dependency so that MYNEWdataset becomes the "primary". This gives you the ability to destroy MYdataset and keep MYNEWdataset.

Promotes a clone file system to no longer be dependent on its
"origin" snapshot. This makes it possible to destroy the file system
that the clone was created from. The clone parent-child dependency
relationship is reversed, so that the origin file system becomes a
clone of the specified file system.

The first sentence here does read somewhat as if the dependency is gone. However if you continue reading it very clearly states that the original dataset is basically now the clone. You still only have one copy of the data on disk, so your options for deleting come down to a. delete the clone or b. delete the current "primary" dataset and destroy both in the process.

If you plan to use the clone (MYNEWdataset) for something temporarily, then scrap it, you shouldn't promote it. Keep the original dataset as the master.

[B3r3n06]

Hello Ahrens,

Thanks for your answer.

I understand what you say and what is written seems to be apparently the case. To me, this then becomes an issue of terms.
If I clone you, once your clone is independant of you (as the zfs clone command indicates), I can kill it without having to kill you. Else that is not a clone. A clone is an independant copy, even if, to save disk space, it relies on the data block of its original source. If ZFS behaves this way ok, but then a "zfs unpromote" command should exist to avoid a deadend issue upon misuse.

Taking the example of BTRfs, if I create a snapshot of a dataset, this dataset has its own life, can be modified, destroyed, whatever. There is no snapshot hierarchy either.. The clone counts the difference with its "master" as disk space, no more.
Once I finished using it, I just delete the snapshot and the associated directory & data vanish.

With ZFS, creating a real clone seems then not possible but at the price of zfs send|zfs receive.
If I want to have what looks like a clone, ZFS wants a snapshot first (meaning a snapshot is required per clone if there is just a small difference in data)

That is the purpose I want to achieve is the capability for students to work on their own directory tree with the capability of rollback based on versioning (then it should be possible to rollback at any version level without loosing snapshot done later in time), based from an original containing the exercice. Once the student finished, he can keep its clone for later use or the clone is destroyed to free space.

If that is not a bug, maybe this could become a dev request ?

Thanks

Brgrds

[churchers]

[quote]With ZFS, creating a real clone seems then not possible but at the price of zfs send|zfs receive.[/quote]

I'm confused at what you mean by a "real" clone. The point of clone is to create two separate identical datasets without having to duplicate any data. The clone takes no space other than new data written to it. If needed, you can delete the clone with no issue. Now of course, if you try to delete the original dataset, you can't as it has dependent clones. The clone only stores its differences and is relying on the data in that original dataset. If you really want to scrap the original but keep the clone, you need to promote the clone so it becomes the "master".

I can't see how this could possibly work any other way without having two full copies (i.e. send/recv). I know nothing of BTRFS but I can't see how you could snapshot/clone a dataset with zero data copy, then delete that original dataset.

One possible "improvement" here could be the ability to destroy a master and have the data automatically promoted to the oldest clone. I would probably still suggest ZFS having a warning for this though with -f or some specific argument required to actually make it work. This way you could have two datasets (a master and clone) that appear independent, and either could be deleted; The data would just move to whichever one still exists. All you're really doing though is avoiding the promote step, which many would argue is preferred as it gives the administrator complete control over which dataset is currently the master.

For your teaching example you just need to have a base dataset containing the exercise. Clone it for each student but do not run promote on any of them. Each student can work on their own clone and make their own snapshots, and all the clones can be removed if not needed. They will all only take the space required by changes. The only caveat is that rolling back on ZFS removes the intermediate snapshots, so you can't rollback to a month ago, then go forward again. This isn't an issue of cloning, but ZFS in general. (You can of course however look in any snapshot to retrieve data from that point in time)

[B3r3n06]

Hello Matt,

Agree, there is a confusion, probably coming from me.

Moving backwards, I sense this might come because I played with BTRfs which was simpler to achieve my needs and probably because I am far from mastering ZFS.

With BTRfs, a snapshot have its own life and is mountable. It relies on its master of course, but since a snapshot is built, that is call. So I snap the dir to another mount point and then student does whatever. At the end, I destroy snap, no more.
And yes, if on BTRfs you delete the "master", automatically the blocks are assigned to the snapshot.
That is part of BTRfs issue by the way because such an action cost so much that everything's frozen meanwhile and since it is still used by processes, it might generate data corruption...

On ZFS, a snapshot is not a mountpoint but if I deliberately mount it, which has a mountpoint cost. To become a directory, it must be zfs cloned. I understand a 'static' image is required for the clone and so the snapshot is required for this.
But then it means each clone must have its own snapshot, except of course, if at a moment the teacher frozen the dataset. But then, changes he will perform will be lost. The snapshot blocks evolution.
After all from what I understand,, a zfs clone is no more than a RW mounted snapshot.

What if there is no snapshot ? The original could keep evolving, ok. But if there was no snapshot, would it make a difference ? Some block would change, clone will reflect that, ok. If that is not wished, then a snapshot remains possible.

[B3r3n06]

About "They will all only take the space required by changes. The only caveat is that rolling back on ZFS removes the intermediate snapshots, so you can't rollback to a month ago, then go forward again. This isn't an issue of cloning, but ZFS in general. (You can of course however look in any snapshot to retrieve data from that point in time)"

Agree, that is a real concern.. Since I have this strong need and mounting to copy is not acceptable (due to the delay on big amount of data), I may have found a workaround playing on snapshot+clone.

My idea I just got (still to test & check) would then be to have a clone per version.
Because zfs clone is mounted and lives, I think it will be exactly the same as my BTRfs sub snaps :-)
From this moment, versionning without hierarchical enforcement will not be performed with "zfs rollback" but zfs clones that would be "clones of clones" for the requesting student.

To be tested :-)

[IvanVolosyuk]

To 'unpromote' a clone you can just promote the original and you can delete the clone after that. In your example:

zfs promote MYdataset
zfs destroy MYNEWdataset

This will do the trick. You can also keep just MYNEWdataset:

zfs promote MYNEWdataset
zfs destroy MYdataset

[B3r3n06]

Ookkk, thanks Ivan, I am currently moving data from BTRfs back to ZFS, I will then see how far can I go to reach the same situation as before (in terms of flexibiliy I mean), because BTRfs, in terms of performance, is really bad, ZFS is far beyond...

[B3r3n06]

Hello guys,

I am taking the opportunity of skilled ZFS fellows around to be sure.

To reach again the rolling back feature WITH versionning (ZFS destroy earlier snapshots), I wonder if using zfs clone could do.

My idea:
Multiple snapshot exist : T0, T1, T2, from oldest to newest.
With ZFS, if I restore T1, I will loose T2.

So the idea would then be NOT to restore T1 but to zfs clone T1 as new dataset in the previous location/mountpoint.
With this, T1 clone becomes the current dataset and so no interference with T2 and others.

HOWEVER, I wonder if after a while, multiple rollbacks will not finally affect the consistency of the ZFS dataset.
It could generate some "cross" datasets/snapshots...

Feedback welcome.

Thanks

[B3r3n06]

Digging a bit, I realize ZFS might be similar to BTRfs about snapshot and might be able to provide the same service.

ZFS stores its snapshots into MYdataset/.zfs/snapshot/[name].
Files are directly accessible here.

Now, it is possible to snapshot the snapshot ? ;-)
This way, changes would be performed on a copy of the snapshot, saving the original ?

Feedback still welcome :-)

[IvanVolosyuk]

You can zfs set readonly=on and snapshot twice - you will get two copies of snapshot. You cannot snapshot the content of .zfs/snapshot/blah. I used clones quite a bit. You can make a tree of clones if you want. The tricky part with clones is to do incremental sends to preserve the tree structure of clones. What I did is I made clones to be children of the original. This way I could replicate the structure using 'zfs send -R' and after that sent incremental updates to all clones independently. Not sure how I can add more clones in this setup though.

[B3r3n06]

Hello Ivan,

Thanks for your answer.

In fact, I sense the blocking point is precisely the fact snapshots and thus clones are always dependant from a "master".
I got the same situation in BTRfs, but there snapshots were the same type of objects as the master was, and mountable wherever wanted.

To satisfy the need, solution was simple
/mydir -> snapshot S_T0 -> /snaphots/S_T0
/mydir -> snapshot S_T1 -> /snaphots/S_T1
/mydir -> snapshot S_T2 -> /snaphots/S_T2

When I needed to put S_T1 in place of the original /mydir, it was simple:
/snapshots/S_T1 -> snapshot mydir ->/mydir
simple as this because there is no link between mountpoint and volume name.

From this, since /mydir was a snapshot accessible as a directory, it was used for the need.

With this same logic, I was able to have versioning...into clones !
That is S_T1 -> /myCLONEdir

Maybe that is a bit confusing as expressed, I try to focus on how to satisfy the need rather than the way BTRfs or ZFS behaves from my understanding :-(

[richardelling]

The part you are missing is that the clone is a new dataset. So rather than trying to replace /mydir with a child of itself. Just use the child. -- richard

…

On Thu, Dec 17, 2020 at 5:59 AM B3r3n06 ***@***.***> wrote: Hello Ivan, Thanks for your answer. In fact, I sense the blocking point is precisely the fact snapshots and thus clones are always dependant from a "master". I got the same situation in BTRfs, but there snapshots were the same type of objects as the master was, and mountable wherever wanted. To satisfy the need, solution was simple /mydir -> snapshot S_T0 -> /snaphots/S_T0 /mydir -> snapshot S_T1 -> /snaphots/S_T1 /mydir -> snapshot S_T2 -> /snaphots/S_T2 When I needed to put S_T1 in place of the original /mydir, it was simple: /snapshots/S_T1 -> snapshot mydir ->/mydir simple as this because there is no link between mountpoint and volume name. From this, since /mydir was a snapshot accessible as a directory, it was used for the need. With this same logic, I was able to have versioning...into clones ! That is S_T1 -> /myCLONEdir Maybe that is a bit confusing as expressed, I try to focus on how to satisfy the need rather than the way BTRfs or ZFS behaves from my understanding :-( — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <https://github.com/openzfs/zfs/issues/11316#issuecomment-747455031>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGTZTORZDNU7W3ACBILINLSVIFEHANCNFSM4UTP3POQ> .

[B3r3n06]

Hello Richard,

I got you.

However, there are constraints to comply with :-)

Example, mydir must be kept but ok, a mountpoint could do the trick.

Also there is a hierarchical dependancy : /MYpool/MYdataset/MYsubdataset -> MYsubdataset being the clone, must be into the MYsubdataset subdirectory of MYdataset directory :-)

I know, the need is quite complex :-)

Brgrds

[IvanVolosyuk]

I kinda lost the context if you still talking about ZFS or BTRFS. In ZFS a clone can be anywhere in the hierarchy of the pool not necessary a child dataset of a primary dataset and you can reverse child / parent relationship with 'zfs promote' without moving/renaming any datasets.

[B3r3n06]

Hello Ivan,

That is precisely my point : in real life, a clone is an independant living no ? I can understand it starts being dependant of a master, but it should be releasable. For example, zfs promote would make it independant from the original master...

That is the point of the whole example. Promote is not really a promotion since there is still some associated data...

I dont have such issue with BTRfs, I was hoping there would be some method but apparently not.

It means to me, upon a nested zfs dataset project directory tree, I will probably not be able to have multiple clones of a subdataset.
projectA/object_dir1 with multiple versions (without the ZFS hierarchical contraints where rollback T1 (order T0 T1 T2..) means destroy T2, T3 etc
projectA/object_dir2 same way

And capability to have any version of dataset for projectA, object_dir1 or 2...

[IvanVolosyuk]

You can have snapshots in each clone independently. Promote just changes ownership of the shared data between clones. This is needed for accounting purposes - how much space consumed by a dataset. I think accounting part is different in BTRFS, thus the differences. In ZFS you can see exactly how much space you will free deleting a dataset or snapshot.

[bghira]

that's what unionfs/overlayfs are for.

[richardelling]

On Dec 23, 2020, at 11:09 PM, B3r3n06 ***@***.***> wrote:  Hello Ivan, That is precisely my point : in real life, a clone is an independant living no ?

In the computing world, that is known as a copy. There is no biological equivalent to ZFS clones.

…

-- richard

[bghira]

the DSM-V has something called multiple dissociative identity disorder, kinda seems relevant here.

[B3r3n06]

BTRfs sub snap is better than DSM-V then :-)
A snapshot is a living directory, directly usable and independant from its parent.
Upon destruction of the parent, it gets all data :-)

[B3r3n06]

Hello guys,

Merry Xmas & thanks for your answers.

@IvanVolosyuk : agree. Seems BTRfs is behaving internally a different way. This permits to, in fact, use snapshots are a fully operational copy, inpendant from its parent. Really usefull feature but in a awfully slow filesystem :-(.

@misterbigstuff: THANKS ! I know UnionFS to use is in FreeBSD but did not remember it. Seems it exists on Linux. If it behaves the same in such a situation of big files, this might be the answer : instead of a zfs snapshot/clone, I could union mount above the original tree, than do whatever I want to, that is just mount points. Limit will then be the max number of simultaneous mounts...

@richardelling & @misterbigstuff : agree, ZFS clones are not clones in real. Looks more like some sort of aura that can live outside the real body without being independant of it. Cut the spiritual link and aura dies... Maybe same spirit in James Cameron's Avatar :-)

I'll test UnionFS to see if that might solve my need !

[B3r3n06]

Hello all,
Happy New Year for you & yours !

@misterbigstuff : a unionfs does not seem to make it. I tested with aufs, upon a directory renaming or file update, the command is looong because the entire object data (either directory or file) is copied. So it becomes the same as duplicating data...

[IvanVolosyuk]

ZFS and BTRFS have different design and different focus. BTRFS has focus on flexibility, while ZFS focus on stability, and ease of administration.
There are actions in BTRFS which are easier to perform, like writable snapshots. Though, to figure out how much space the clone uses is not a trivial task. You can setup a very flexible and complicated qgroups in BTRFS to figure this out, but it is no working out of the box.
In ZFS clones are similar to writable snapshots from BTRFS, but it is more hierarchical in the space accounting sense:

$ zfs list -o written,space
WRITTEN  NAME                              AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
   259M  ssd/root                           268G  14.4G     1.23G   13.2G             0B         0B
   284M  ssd/temp/root.clone                268G   284M        0B    284M             0B         0B
...

In this example root.clone is a clone of root. I know exactly how much space root uses and how much additional space root.clone uses as well. The clone is not independent, it relies on the availability of the base. I can delete the clone and get 284M back. I can zfs promote the clone and make root dependent on root.clone and delete root if I like:

$ zfs promote ssd/temp/root.clone
$ zfs list -o written,space
WRITTEN  NAME                              AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
   261M  ssd/root                           268G   341M     5.62M    335M             0B         0B
   284M  ssd/temp/root.clone                268G  14.4G      798M   13.6G             0B         0B

I can see that I can get 341M back if I delete the root:

$ zfs destroy -r ssd/root

You can think of ZFS clones as BTRFS writable snapshots where space management is more explicit.
In BTRFS writable snapshots are easier to create, harder to understand how much space they consume.

Bottom line: ZFS clones are not independent from the base dataset, because of the explicit space management. You can delete base dataset, but you have to explicitly promote a clone to sort out the space management questions first. Once you get the grip on the space management questions you will find out that you can create any clone hierarchy and delete any dataset from it if you want. Thus, you can think of ZFS clones as independent from base datasets, just more explicit in terms of space management.

[johnjaylward]

I am running into a similar issue, as I did not understand the terminology of ZFS clones at the time I made some jails.

for example, I setup a jail "jail1" got it to a usable place (i.e. installed packages I wanted common across a couple jails) then cloned to a few jails (jail2, jail3, jail4).

I am using all 4 jails (I do NOT want to delete the original dataset, just the original snapshot), but now the original clone is years old, and the systems have undergone many upgrades, but I can not remove the original snapshot as doing so will apparently delete at least one of my usable jails. Promoting any of the jails does not work, and I just get into a loop of dataset promotions and have no easy recourse to cleanup the ancient snapshot except by turning down my jails, copying the data to new dataset(s), deleting the snapshot, then recreating the deleted dataset(s).

ZFS being COW it seems like it should be "easy" (from an administrator perspective) to split these dataset off into unique trees that point at the same disk blocks, but it looks like that is not possible without physically copying the data. If I had realized this limitation at the time, probably would have simply used the ZFS send/receive commands instead.

[amotin]

Before block cloning implemented recently it was impossible to track reference counts for shared blocks, that's why keeping the snapshot at the clone point was required. It was relatively easy and efficient solution. These days instead of cloning a snapshot you could just copy the data with block cloning to be more flexible. But it also has downsides of being much more expensive and less friendly to replication.