Custom Model Inference gives all black images #169
Description
Hello, first of all thank you for your work but i am facing some issues.
I have followed the code given in your github, trained stage 1 and stage 2
for custom model, i had to give my own training configuration file,
model:
cldm:
target: diffbir.model.cldm.ControlLDM
params:
latent_scale_factor: 0.18215
unet_cfg:
use_checkpoint: True
image_size: 32 # unused
in_channels: 4
out_channels: 4
model_channels: 320
attention_resolutions: [ 2, 1 ]
num_res_blocks: 2
channel_mult: [ 1, 2, 4, 4 ]
num_head_channels: 64 # need to fix for flash-attn
use_spatial_transformer: True
use_linear_in_transformer: True
transformer_depth: 1
context_dim: 1024
legacy: False
vae_cfg:
embed_dim: 4
ddconfig:
double_z: true
z_channels: 4
resolution: 256
in_channels: 3
out_ch: 3
ch: 128
ch_mult:
- 1
- 2
- 4
- 4
num_res_blocks: 2
attn_resolutions: []
dropout: 0.0
clip_cfg:
embed_dim: 1024
vision_cfg:
image_size: 224
layers: 32
width: 1280
head_width: 80
patch_size: 14
text_cfg:
context_length: 77
vocab_size: 49408
width: 1024
heads: 16
layers: 24
layer: "penultimate"
controlnet_cfg:
use_checkpoint: True
image_size: 32 # unused
in_channels: 4
hint_channels: 4
model_channels: 320
attention_resolutions: [4, 2, 1 ]
num_res_blocks: 2
channel_mult: [ 1, 2, 4, 4 ]
num_head_channels: 64 # need to fix for flash-attn
use_spatial_transformer: True
use_linear_in_transformer: True
transformer_depth: 1
context_dim: 1024
legacy: False
diffusion:
target: diffbir.model.gaussian_diffusion.Diffusion # Class path to Diffusion
params:
linear_start: 0.00085
linear_end: 0.0120
timesteps: 1000
swinir:
target: diffbir.model.swinir.SwinIR
params:
img_size: 64
patch_size: 1
in_chans: 3
embed_dim: 64
depths: [4,4]
num_heads: [4, 4]
window_size: 4
mlp_ratio: 2
sf: 8
img_range: 1.0
upsampler: "nearest+conv"
resi_connection: "1conv"
unshuffle: True
unshuffle_scale: 8
train:
sd_path: /media/admin1/DL/CapsNetwork/Unsupervised/Diffusion/DiffBIR/weights/v2-1_512-ema-pruned.ckpt # Original SD weights
swinir_path: /media/admin1/DL/CapsNetwork/Unsupervised/Diffusion/DiffBIR/experiments_LR_SWIN/checkpoints/0150000.pt # Original SwinIR weights
i provided the path of trained stage 2 model in python -u inference.py
--upscale 4
--version custom
--train_cfg [path/to/training/config]
--ckpt [path/to/saved/checkpoint]
--captioner llava
--cfg_scale 8
--noise_aug 0
--input inputs/demo/bsr
--output results/custom_demo_bsr
have i done anything wrong?