Mz/multi bit decompression

[mayeul-zama]

This change is 

[mayeul-zama]

Not sure if we should ignore thread_count and deterministic_execution

[mayeul-zama]

Removed deterministic_execution
asserted thread_count > 0

[mayeul-zama]

Adds 1.4 GB of data
Feels a bit too much for me

[IceTDrinker]

as discussed we likely can lower the packing ks GlweDimension to reduce on disk size and keep correctness (though not security like other params for data backward)

[IceTDrinker]

check commit is red

[IceTDrinker]

I would prefer we don't do such changes notably for serilization formats for which field orders may matter, it means we have to update the CompressionParametersVersions for no gains

[IceTDrinker]

Some first comments, haven't yet dived in the meat of the PR

@IceTDrinker reviewed 1 of 1 files at r1, 40 of 48 files at r2, all commit messages.
Reviewable status: 41 of 49 files reviewed, 10 unresolved discussions (waiting on @agnesLeroy, @mayeul-zama, and @tmontaigu)

---

tfhe/src/high_level_api/compressed_ciphertext_list.rs line 960 at r2 (raw file):

    use crate::prelude::*;
    use crate::safe_serialization::{safe_deserialize, safe_serialize};
    use crate::shortint::parameters::v1_4::V1_4_COMP_PARAM_MESSAGE_2_CARRY_2_KS_PBS_TUNIFORM_2M128_MB_GPU;

if you rebase we are now on 1.5, also please either create params aliases or test params for those

we do not want to have to update params in a lot of parts of the code base (until we can do better)

---

tfhe/src/integer/gpu/ciphertext/compressed_ciphertext_list.rs line 581 at r2 (raw file):

    use crate::integer::gpu::gen_keys_radix_gpu;
    use crate::integer::{ClientKey, RadixCiphertext, RadixClientKey};
    use crate::shortint::parameters::v1_4::V1_4_COMP_PARAM_MESSAGE_2_CARRY_2_KS_PBS_TUNIFORM_2M128_MB_GPU;

same remark

---

tfhe/src/core_crypto/commons/math/random/tests.rs line 447 at r2 (raw file):

// https://en.wikipedia.org/wiki/Dvoretzky%E2%80%93Kiefer%E2%80%93Wolfowitz_inequality#The_DKW_inequality
#[allow(unused)]

why is this required ?

---

tfhe/src/shortint/backward_compatibility/parameters/list_compression.rs line 46 at r2 (raw file):

            packing_ks_polynomial_size,
            packing_ks_glwe_dimension,
            decompression_grouping_factor: None,

you did not follow the pattern used for the noise squashing ?

---

tfhe/src/integer/ciphertext/compressed_ciphertext_list.rs line 231 at r2 (raw file):

        TEST_PARAM_MESSAGE_2_CARRY_2_KS_PBS_TUNIFORM_2M128,
    };
    use crate::shortint::parameters::v1_4::V1_4_COMP_PARAM_MESSAGE_2_CARRY_2_KS_PBS_TUNIFORM_2M128_MB_GPU;

create an alias and be careful we have updated to 1.5

---

utils/tfhe-backward-compat-data/data/high_level_api.ron line 805 at r2 (raw file):

            test_filename: "client_key",
            parameters: TestClassicParameterSet((
                lwe_dimension: 887,

seems you use a full parameter for the compute params, maybe use a small insecure one ?

---

utils/tfhe-backward-compat-data/data/high_level_api.ron line 822 at r2 (raw file):

                carry_modulus: 4,
                max_noise_level: 5,
                log2_p_fail: -64.138,

2^-64 pfail ?

---

tfhe/src/integer/gpu/list_compression/compressed_server_keys.rs line 48 at r2 (raw file):

            }
            crate::shortint::list_compression::CompressedDecompressionKey::MultiBit { .. } => {
                todo!()

todo for the GPU to get keys on their device ?

[IceTDrinker]

as discussed we likely can lower the packing ks GlweDimension to reduce on disk size and keep correctness (though not security like other params for data backward)

[mayeul-zama]

Reviewable status: 41 of 49 files reviewed, 11 unresolved discussions (waiting on @agnesLeroy, @IceTDrinker, and @tmontaigu)

---

tfhe/src/core_crypto/commons/math/random/tests.rs line 447 at r2 (raw file):

Previously, IceTDrinker (Arthur Meyre) wrote…

why is this required ?

When only the shortint features is activated, this function unused and raises a warning

---

tfhe/src/high_level_api/compressed_ciphertext_list.rs line 960 at r2 (raw file):

Previously, IceTDrinker (Arthur Meyre) wrote…

if you rebase we are now on 1.5, also please either create params aliases or test params for those

we do not want to have to update params in a lot of parts of the code base (until we can do better)

Not sure where these test parameters should be

We may want to replace them by production parameters when we'll have them in the crate

---

tfhe/src/integer/gpu/list_compression/compressed_server_keys.rs line 48 at r2 (raw file):

Previously, IceTDrinker (Arthur Meyre) wrote…

todo for the GPU to get keys on their device ?

Yes

---

tfhe/src/shortint/backward_compatibility/parameters/list_compression.rs line 46 at r2 (raw file):

Previously, IceTDrinker (Arthur Meyre) wrote…

you did not follow the pattern used for the noise squashing ?

I don't see which pattern you're referring to

---

utils/tfhe-backward-compat-data/data/high_level_api.ron line 805 at r2 (raw file):

Previously, IceTDrinker (Arthur Meyre) wrote…

seems you use a full parameter for the compute params, maybe use a small insecure one ?

Yes