stouputils.parallel package#

This module provides utility functions for parallel processing, such as:

I highly encourage you to read the function docstrings to understand when to use each method.

Priority (nice) mapping for multiprocessing():

  • Unix-style values from -20 (highest priority) to 19 (lowest priority)

  • Windows automatic mapping: * -20 to -10: HIGH_PRIORITY_CLASS * -9 to -1: ABOVE_NORMAL_PRIORITY_CLASS * 0: NORMAL_PRIORITY_CLASS * 1 to 9: BELOW_NORMAL_PRIORITY_CLASS * 10 to 19: IDLE_PRIORITY_CLASS

import time
import stouputils as stp

def slow_square(x: int) -> int:
        time.sleep(0.4)  # Pretend this takes a while
        return x * x

# Needed because each new process imports this file again
if __name__ == "__main__":
        # 4 processes at once, with a progress bar (best for heavy computations)
        squares = stp.multiprocessing(slow_square, range(12), desc="Squaring", max_workers=4)
        stp.info("Squares:", squares)

        # 5 threads at once (best for waiting on files or the network)
        stp.multithreading(time.sleep, [0.2] * 20, desc="Waiting on IO", max_workers=5)

        # One call in a separate process, its result sent back
        stp.info("Ran in a subprocess:", stp.run_in_subprocess(slow_square, 7))
Terminal output of the example, with the progress bars filling up

Submodules#