Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Lumina-image-2.0 #1927

Open
wants to merge 27 commits into
base: sd3
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
d154e76
init
sdbds Feb 12, 2025
c0caf33
update
sdbds Feb 15, 2025
7323ee1
update lora_lumina
sdbds Feb 15, 2025
a00b06b
Lumina 2 and Gemma 2 model loading
rockerBOO Feb 15, 2025
60a76eb
Add caching gemma2, add gradient checkpointing, refactor lumina model…
rockerBOO Feb 16, 2025
1601563
Update metadata.resolution for Lumina 2
rockerBOO Feb 16, 2025
6965a01
Merge pull request #12 from rockerBOO/lumina-model-loading
sdbds Feb 16, 2025
733fdc0
update
sdbds Feb 17, 2025
3ce23b7
Merge branch 'lumina' of https://github.com/sdbds/sd-scripts into lumina
sdbds Feb 17, 2025
bb7bae5
Merge pull request #13 from rockerBOO/lumina-cache-checkpointing
sdbds Feb 17, 2025
aa36c48
update for always use gemma2 mask
sdbds Feb 17, 2025
44782dd
Fix validation epoch divergence
rockerBOO Feb 14, 2025
3365cfa
Fix sizes for validation split
rockerBOO Feb 17, 2025
3ed7606
Clear sizes for validation reg images to be consistent
rockerBOO Feb 17, 2025
1aa2f00
Fix validation epoch loss to check epoch average
rockerBOO Feb 16, 2025
98efbc3
Add documentation to model, use SDPA attention, sample images
rockerBOO Feb 18, 2025
bd16bd1
Remove unused attention, fix typo
rockerBOO Feb 18, 2025
6597631
Merge pull request #14 from rockerBOO/samples-attention
sdbds Feb 19, 2025
025cca6
Fix samples, LoRA training. Add system prompt, use_flash_attn
rockerBOO Feb 23, 2025
6d7bec8
Remove non-used code
rockerBOO Feb 23, 2025
42a8015
Fix system prompt in datasets
rockerBOO Feb 23, 2025
ba725a8
Set default discrete_flow_shift to 6.0. Remove default system prompt.
rockerBOO Feb 23, 2025
48e7da2
Add sample batch size for Lumina
rockerBOO Feb 24, 2025
2c94d17
Fix typo
rockerBOO Feb 24, 2025
653621d
Merge pull request #15 from rockerBOO/samples-training
sdbds Feb 24, 2025
fc772af
1、Implement cfg_trunc calculation directly using timesteps, without i…
sdbds Feb 24, 2025
5f9047c
add truncation when > max_length
sdbds Feb 25, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions library/config_util.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ class BaseSubsetParams:
custom_attributes: Optional[Dict[str, Any]] = None
validation_seed: int = 0
validation_split: float = 0.0
system_prompt: Optional[str] = None


@dataclass
Expand Down Expand Up @@ -106,6 +107,7 @@ class BaseDatasetParams:
debug_dataset: bool = False
validation_seed: Optional[int] = None
validation_split: float = 0.0
system_prompt: Optional[str] = None


@dataclass
Expand Down Expand Up @@ -196,6 +198,7 @@ def __validate_and_convert_scalar_or_twodim(klass, value: Union[float, Sequence]
"caption_prefix": str,
"caption_suffix": str,
"custom_attributes": dict,
"system_prompt": str,
}
# DO means DropOut
DO_SUBSET_ASCENDABLE_SCHEMA = {
Expand Down Expand Up @@ -241,6 +244,7 @@ def __validate_and_convert_scalar_or_twodim(klass, value: Union[float, Sequence]
"validation_split": float,
"resolution": functools.partial(__validate_and_convert_scalar_or_twodim.__func__, int),
"network_multiplier": float,
"system_prompt": str,
}

# options handled by argparse but not handled by user config
Expand Down Expand Up @@ -526,6 +530,7 @@ def print_info(_datasets, dataset_type: str):
batch_size: {dataset.batch_size}
resolution: {(dataset.width, dataset.height)}
enable_bucket: {dataset.enable_bucket}
system_prompt: {dataset.system_prompt}
""")

if dataset.enable_bucket:
Expand Down Expand Up @@ -559,6 +564,7 @@ def print_info(_datasets, dataset_type: str):
token_warmup_step: {subset.token_warmup_step},
alpha_mask: {subset.alpha_mask}
custom_attributes: {subset.custom_attributes}
system_prompt: {subset.system_prompt}
"""), " ")

if is_dreambooth:
Expand Down
Loading