stouputils.data_science.data_processing.technique module#

class ProcessingTechnique(name: Literal['rotation', 'translation', 'shearing', 'noise', 'salt-pepper', 'sharpening', 'contrast', 'axis_flip', 'zoom', 'brightness', 'blur', 'random_erase', 'clahe', 'binary_threshold', 'threshold', 'canny', 'laplacian', 'auto_contrast', 'bias_field_correction', 'curvature_flow_filter', 'resize', 'normalize', 'median_blur', 'nlm_denoise', 'bilateral_denoise', 'tv_denoise', 'wavelet_denoise', 'adaptive_denoise', 'invert', 'custom'], ranges: list[Iterable[Any] | RangeTuple], probability: float = 1.0, custom: Callable[..., NDArray[Any]] | None = None)[source]#

Bases: NamedTuple

A named tuple containing an processing technique.

The following descriptions are the recommendations:

  • rotation: Rotation between -20° and +20° is generally safe

  • translation: Translation up to 15% to avoid position bias

  • shearing: Shearing up to 15° on x and y axes

  • noise: Gaussian noise with fixed values [0.3,0.4,0.5]

  • salt-pepper: Salt-pepper noise with densities [0.01,0.02,0.03]

  • sharpening: Gaussian blur (variance=1) then subtract from original

  • contrast: Linear scaling between min/max intensities [0.7,1.3]

  • axis_flip: Flip along x and/or y axis (0 for x, 1 for y, 2 for both), probability=0.5

  • zoom: Magnification/zoom between 85% and 115%

  • brightness: Adjust overall image brightness [0.7,1.3]

  • blur: Gaussian blur for reduced sharpness [0.5,2.0]

  • random_erase: Randomly erase rectangles (1-10% of image) to learn descriptive features

  • clahe: Contrast Limited Adaptive Histogram Equalization (clip_limit=[1.0,4.0], tile_size=8)

  • binary_threshold: Binary thresholding with threshold [0.1,0.9]

  • threshold: Multi-level thresholding with levels [0.3,0.6]

  • canny: Canny edge detection (thresholds=[50/255,150/255], aperture=3)

  • laplacian: Laplacian edge detection with 3x3 kernel

  • auto_contrast: Auto contrast the image

  • bias_field_correction: Bias field correction

  • curvature_flow_filter: Curvature flow filter (time_step=0.05, iterations=5)

  • median_blur: Median blur (kernel_size=15, iterations=10)

  • resize: Resize the image (width=224, height=224, resample=Image.Resampling.LANCZOS)

  • normalize: Normalize the image (min=0, max=255, method=cv2.NORM_MINMAX)

  • nlm_denoise: Non-local means denoising (h=10, template_window_size=7, search_window_size=21)

  • bilateral_denoise: Bilateral filter denoising (d=9, sigma_color=75, sigma_space=75)

  • tv_denoise: Total variation denoising (weight=0.1, iterations=30)

  • wavelet_denoise: Wavelet denoising (wavelet=’db1’, mode=’soft’, wavelet_levels=3)

  • adaptive_denoise: Adaptive denoising (method=”nlm”, strength=0.5)

  • custom: Custom processing technique (callable) # Ex: ProcessingTechnique(“custom”, custom=f)

name: Literal['rotation', 'translation', 'shearing', 'noise', 'salt-pepper', 'sharpening', 'contrast', 'axis_flip', 'zoom', 'brightness', 'blur', 'random_erase', 'clahe', 'binary_threshold', 'threshold', 'canny', 'laplacian', 'auto_contrast', 'bias_field_correction', 'curvature_flow_filter', 'resize', 'normalize', 'median_blur', 'nlm_denoise', 'bilateral_denoise', 'tv_denoise', 'wavelet_denoise', 'adaptive_denoise', 'invert', 'custom']#

Name of the processing technique

ranges: list[Iterable[Any] | RangeTuple]#

List of ranges for the processing technique. Depending on the technique, multiple ranges might be needed. Ex: Translation (x and y axes) or Shearing (x and y axes).

probability: float#

1.0). Should be used on techniques like “axis_flip” or “random_erase” where the probability of applying the technique is not 100%.

Type:

Probability of applying the processing technique (default

custom: Callable[[...], ndarray[Any, dtype[Any]]] | None#

Custom processing technique (callable), name must be “custom”, e.g. ProcessingTechnique(“custom”, custom=f)

deterministic(use_default: bool = False) → ProcessingTechnique[source]#

Convert the RangeTuple to values by calling the RangeTuple.random() method.

apply(image: ndarray[Any, dtype[Any]], dividers: tuple[float, float] = (1.0, 1.0), times: int = 1) → ndarray[Any, dtype[Any]][source]#

Apply the processing technique to the image.

Parameters:

  • image (NDArray[Any]) – Image to apply the processing technique to

  • dividers (tuple[float, float]) – Dividers used to adjust the processing technique parameters (default: (1.0, 1.0))

  • times (int) – Number of times to apply the processing technique (default: 1)

Returns:

Processed image

Return type:

NDArray[Any]

_asdict()#

Return a new dict which maps field names to their values.

_field_defaults = {'custom': None, 'probability': 1.0}#
_fields = ('name', 'ranges', 'probability', 'custom')#
classmethod _make(iterable)#

Make a new ProcessingTechnique object from a sequence or iterable

_replace(**kwds)#

Return a new ProcessingTechnique object replacing specified fields with new values

class RecommendedProcessingTechnique(value)[source]#

Bases: Enum

A class containing the processing techniques with their recommended ranges based on scientific papers.

ROTATION = ('rotation', [(-20.0, -19.0, -18.0, -17.0, -16.0, -15.0, -14.0, -13.0, -12.0, -11.0, -10.0, -9.0, -8.0, -7.0, -6.0, -5.0, -4.0, -3.0, -2.0, -1.0, 0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.0, 15.0, 16.0, 17.0, 18.0, 19.0)], 1.0, None)#

Rotation between -20° and +20° is generally safe for medical images

TRANSLATION = ('translation', [(-0.15, -0.13999999999999999, -0.12999999999999998, -0.11999999999999997, -0.10999999999999996, -0.09999999999999995, -0.08999999999999994, -0.07999999999999993, -0.06999999999999992, -0.059999999999999915, -0.049999999999999906, -0.0399999999999999, -0.029999999999999888, -0.01999999999999988, -0.00999999999999987, 1.3877787807814457e-16, 0.010000000000000148, 0.020000000000000157, 0.030000000000000165, 0.040000000000000174, 0.05000000000000018, 0.06000000000000019, 0.0700000000000002, 0.08000000000000021, 0.09000000000000022, 0.10000000000000023, 0.11000000000000024, 0.12000000000000025, 0.13000000000000025, 0.14000000000000026), (-0.15, -0.13999999999999999, -0.12999999999999998, -0.11999999999999997, -0.10999999999999996, -0.09999999999999995, -0.08999999999999994, -0.07999999999999993, -0.06999999999999992, -0.059999999999999915, -0.049999999999999906, -0.0399999999999999, -0.029999999999999888, -0.01999999999999988, -0.00999999999999987, 1.3877787807814457e-16, 0.010000000000000148, 0.020000000000000157, 0.030000000000000165, 0.040000000000000174, 0.05000000000000018, 0.06000000000000019, 0.0700000000000002, 0.08000000000000021, 0.09000000000000022, 0.10000000000000023, 0.11000000000000024, 0.12000000000000025, 0.13000000000000025, 0.14000000000000026)], 1.0, None)#

Translation between -15% and 15% of image size

SHEARING = ('shearing', [(-10.0, -9.0, -8.0, -7.0, -6.0, -5.0, -4.0, -3.0, -2.0, -1.0, 0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0), (-10.0, -9.0, -8.0, -7.0, -6.0, -5.0, -4.0, -3.0, -2.0, -1.0, 0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0)], 1.0, None)#

Shearing between -10% and 10% distortion

AXIS_FLIP = ('axis_flip', [['horizontal']], 0.5, None)#

horizontal is much more used than vertical

Type:

Flipping

NOISE = ('noise', [(0.1, 0.15000000000000002, 0.20000000000000004, 0.25000000000000006, 0.30000000000000004, 0.3500000000000001, 0.40000000000000013)], 1.0, None)#

FV values between 0.1 and 0.4 for gaussian noise

Type:

Noise

SALT_PEPPER = ('salt-pepper', [(0.1, 0.15000000000000002, 0.20000000000000004, 0.25000000000000006, 0.30000000000000004, 0.3500000000000001, 0.40000000000000013, 0.45000000000000007)], 1.0, None)#

densities between 0.1 and 0.5

Type:

Salt-pepper

SHARPENING = ('sharpening', [(0.5, 0.6, 0.7, 0.7999999999999999, 0.8999999999999999, 0.9999999999999999, 1.0999999999999999, 1.1999999999999997, 1.2999999999999998, 1.4)], 1.0, None)#

gaussian blur with variance=1 then subtract from original

Type:

Sharpening

CONTRAST = ('contrast', [(0.7, 0.7999999999999999, 0.8999999999999999, 0.9999999999999999, 1.0999999999999999, 1.1999999999999997, 1.2999999999999998)], 1.0, None)#

linear scaling between min and max intensity

Type:

Contrast

ZOOM = ('zoom', [(0.85, 0.9, 0.9500000000000001, 1.0, 1.0500000000000003, 1.1)], 1.0, None)#

between 85% and 115%

Type:

Zoom (Magnification)

BRIGHTNESS = ('brightness', [(0.7, 0.7999999999999999, 0.8999999999999999, 0.9999999999999999, 1.0999999999999999, 1.1999999999999997, 1.2999999999999998)], 1.0, None)#

moderate changes to avoid losing details

Type:

Brightness

BLUR = ('blur', [(0.5, 0.75, 1.0, 1.25, 1.5, 1.75)], 1.0, None)#

gaussian blur preferred, moderate values

Type:

Blur

RANDOM_ERASE = ('random_erase', [(0.01, 0.02, 0.03, 0.04, 0.05, 0.060000000000000005, 0.06999999999999999, 0.08, 0.09)], 1.0, None)#

small regions to force model to learn descriptive features

Type:

Random erasing

CLAHE = ('clahe', [(1.0, 1.5, 2.0, 2.5, 3.0, 3.5), [8]], 1.0, None)#

Contrast Limited Adaptive Histogram Equalization with recommended parameters

Type:

CLAHE

BINARY_THRESHOLD = ('binary_threshold', [(0.1, 0.2, 0.30000000000000004, 0.4, 0.5, 0.6, 0.7000000000000001, 0.8)], 1.0, None)#

threshold between 0.1 and 0.9

Type:

Binary threshold

THRESHOLD = ('threshold', [[0.3], [0.6]], 1.0, None)#

creates X levels of thresholding

Type:

Multi-level threshold

CANNY = ('canny', [[0.19607843137254902], [0.5882352941176471], [3]], 1.0, None)#

Canny edge detection with recommended parameters

LAPLACIAN = ('laplacian', [[3]], 1.0, None)#

Laplacian edge detection with 3x3 kernel

AUTO_CONTRAST = ('auto_contrast', [], 1.0, None)#

Auto contrast the image

CURVATURE_FLOW_FILTER = ('curvature_flow_filter', [[0.05], [5]], 1.0, None)#

Curvature flow filter with recommended parameters

BIAS_FIELD_CORRECTION = ('bias_field_correction', [], 1.0, None)#

Bias field correction with recommended parameters

NORMALIZE = ('normalize', [[0], [255], [32]], 1.0, None)#

Normalize the image to the range 0-255

MEDIAN_BLUR = ('median_blur', [[7], [1]], 1.0, None)#

Median blur with 7x7 kernel and 1 iteration

NLM_DENOISE = ('nlm_denoise', [(5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.0, 15.0, 16.0, 17.0, 18.0, 19.0), [7], [21]], 1.0, None)#

Non-local means denoising with recommended parameters

BILATERAL_DENOISE = ('bilateral_denoise', [[9], (30.0, 40.0, 50.0, 60.0, 70.0, 80.0, 90.0, 100.0, 110.0, 120.0, 130.0, 140.0), (30.0, 40.0, 50.0, 60.0, 70.0, 80.0, 90.0, 100.0, 110.0, 120.0, 130.0, 140.0)], 1.0, None)#

Bilateral filter denoising with recommended parameters

TV_DENOISE = ('tv_denoise', [(0.05, 0.1, 0.15000000000000002, 0.2, 0.25, 0.3, 0.35000000000000003, 0.4, 0.45), [30], ['chambolle']], 1.0, None)#

Total variation denoising with recommended parameters

WAVELET_DENOISE = ('wavelet_denoise', [[3], ['db1'], ['soft']], 1.0, None)#

Wavelet denoising with recommended parameters

ADAPTIVE_DENOISE = ('adaptive_denoise', [['nlm'], (0.1, 0.2, 0.30000000000000004, 0.4, 0.5, 0.6, 0.7000000000000001, 0.8)], 1.0, None)#

Adaptive denoising with recommended parameters

INVERT = ('invert', [], 0.5, None)#

Invert the colors of an image with a 50% probability, hoping the model doesn’t focus on bright parts only

On this page

This Page