image_to_image

The Image-to-Image package consists of PyTorch models, run scripts (training and inference), Physgen Dataloader, Testscript and evaluation notebook. It also adds special losses, scheduler and more.

Implemented models:

Pix2Pix (UNet Generator with adversarial loss)
PhysFormer (Transformer)
ResFCN (Simple fully convolutional network - no shrinking)
ResidualDesignModel (Model which consist of 2 of the other models)

Use this model by running:

import sys
sys.path += ["."]

from image_to_image import main
main()

Now set all your arguments/settings through the python argument. Example:

start /B python ./your_py_file_name.py ^
  --mode train ^
  --epochs 50 ^
  --batch_size 1 ^
  --lr 0.001 ^
  --loss weighted_combined ^
  --wc_loss_silog_lambda 0.5 ^
  --wc_loss_weight_silog 0.1 ^
  --wc_loss_weight_grad 5.0 ^
  --wc_loss_weight_ssim 10.0 ^
  --wc_loss_weight_edge_aware 5.0 ^
  --wc_loss_weight_l1 1.0 ^
  --wc_loss_weight_var 0.0 ^
  --wc_loss_weight_range 0.0 ^
  --wc_loss_weight_blur 0.0 ^
  --optimizer adamw ^
  --weight_decay ^
  --weight_decay_rate 0.05 ^
  --gradient_clipping ^
  --gradient_clipping_threshold 2.0 ^
  --scheduler cosine ^
  --use_warm_up ^
  --warm_up_start_lr 0.000005 ^
  --warm_up_step_duration 2000 ^
  --activate_amp ^
  --amp_scaler grad ^
  --checkpoint_save_dir ./checkpoints ^
  --save_only_best_model ^
  --validation_interval 2 ^
  --model physicsformer ^
  --physicsformer_in_channels 1 ^
  --physicsformer_out_channels 1 ^
  --physicsformer_img_size 256 ^
  --physicsformer_patch_size 4 ^
  --physicsformer_embedded_dim 1024 ^
  --physicsformer_num_blocks 8 ^
  --physicsformer_heads 16 ^
  --physicsformer_mlp_dim 2048 ^
  --physicsformer_dropout 0.1 ^
  --data_variation sound_reflection ^
  --input_type osm ^
  --output_type standard ^
  --device cuda ^
  --experiment_name image-to-image ^
  --run_name physicsformer_test ^
  --tensorboard_path ./tensorboard ^
  --save_path ./mlflow_images ^
  --cmap gray ^
  > ./logs/physicsformer_test.log 2>&1

Open the folder with the evaluation notebook:

from image_to_image import open_notebook_folder
open_notebook_folder()

View Source

 1"""
 2The Image-to-Image package consists of 
 3PyTorch models, run scripts (training and inference),
 4Physgen Dataloader, Testscript and evaluation notebook.
 5It also adds special losses, scheduler and more.
 6
 7Implemented models:
 8- Pix2Pix (UNet Generator with adversarial loss)
 9- PhysFormer (Transformer)
10- ResFCN (Simple fully convolutional network - no shrinking)
11- ResidualDesignModel (Model which consist of 2 of the other models)
12
13Use this model by running:
14```python
15import sys
16sys.path += ["."]
17
18from image_to_image import main
19main()
20```
21
22Now set all your arguments/settings through the python argument. Example:
23```
24start /B python ./your_py_file_name.py ^
25  --mode train ^
26  --epochs 50 ^
27  --batch_size 1 ^
28  --lr 0.001 ^
29  --loss weighted_combined ^
30  --wc_loss_silog_lambda 0.5 ^
31  --wc_loss_weight_silog 0.1 ^
32  --wc_loss_weight_grad 5.0 ^
33  --wc_loss_weight_ssim 10.0 ^
34  --wc_loss_weight_edge_aware 5.0 ^
35  --wc_loss_weight_l1 1.0 ^
36  --wc_loss_weight_var 0.0 ^
37  --wc_loss_weight_range 0.0 ^
38  --wc_loss_weight_blur 0.0 ^
39  --optimizer adamw ^
40  --weight_decay ^
41  --weight_decay_rate 0.05 ^
42  --gradient_clipping ^
43  --gradient_clipping_threshold 2.0 ^
44  --scheduler cosine ^
45  --use_warm_up ^
46  --warm_up_start_lr 0.000005 ^
47  --warm_up_step_duration 2000 ^
48  --activate_amp ^
49  --amp_scaler grad ^
50  --checkpoint_save_dir ./checkpoints ^
51  --save_only_best_model ^
52  --validation_interval 2 ^
53  --model physicsformer ^
54  --physicsformer_in_channels 1 ^
55  --physicsformer_out_channels 1 ^
56  --physicsformer_img_size 256 ^
57  --physicsformer_patch_size 4 ^
58  --physicsformer_embedded_dim 1024 ^
59  --physicsformer_num_blocks 8 ^
60  --physicsformer_heads 16 ^
61  --physicsformer_mlp_dim 2048 ^
62  --physicsformer_dropout 0.1 ^
63  --data_variation sound_reflection ^
64  --input_type osm ^
65  --output_type standard ^
66  --device cuda ^
67  --experiment_name image-to-image ^
68  --run_name physicsformer_test ^
69  --tensorboard_path ./tensorboard ^
70  --save_path ./mlflow_images ^
71  --cmap gray ^
72  > ./logs/physicsformer_test.log 2>&1
73```
74
75Open the folder with the evaluation notebook:
76```python
77from image_to_image import open_notebook_folder
78open_notebook_folder()
79```
80"""
81from . import model_interactions
82from . import utils
83from .main import main
84from .utils.package import open_notebook_folder