stouputils.data_science.scripts.exhaustive_process module#
- exhaustive_process(datasets_to_process: list[tuple[str, str]], main_script_path: str, default_kfold: int = 0, default_transfer_learning: str = 'imagenet', default_grouping: str = 'none', default_max_workers: int = 1, default_verbose: int = 100) None[source]#
Process all datasets through preprocessing, augmentation, and training.
This script will: 1. Verify if the datasets exist 2. Prepare commands for training models on each dataset 3. Execute the commands with the specified parameters 4. Support multiple grouping strategies and model architectures 5. Allow for k-fold cross-validation and grid search optimization
- Parameters:
datasets_to_process (list[tuple[str, str]]) – List of dataset paths to process. Each tuple contains (dataset_path, based_of_path), e.g. [(“aug_preprocessed_path”, “preprocessed_path”)].
main_script_path (str) – Path to the main script, e.g. “src/main.py”
default_model (str) – Default model architecture to use for training.
default_kfold (int) – Default number of folds for k-fold cross validation.
default_transfer_learning (str) – Default source for transfer learning.
default_grouping_strategy (str) – Default strategy for grouping dataset images.
default_max_workers (int) – Default maximum number of threads for processing.
default_verbose (int) – Default verbosity level for training output.
- Returns:
This function does not return anything.
- Return type:
None