MultiProcessingReadingService

class torchdata.dataloader2.MultiProcessingReadingService(num_workers: int = 0, pin_memory: bool = False, timeout: float = 0, worker_init_fn: Callable[[int], None] | None = None, multiprocessing_context=None, prefetch_factor: int | None = None, persistent_workers: bool = False)

MultiProcessingReadingService that utilizes torch.utils.data.DataLoader to launch subprocesses for DataPipe graph. Please refers to documents of DataLoader in https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader for all arguments.

Note

This ReadingService be replaced by PrototypeMultiProcessingReadingService.

finalize() → None

ReadingService cleans up internal states and fully shuts down the service. Called in DataLoader2’s shutdown and __del__.

initialize(datapipe: IterDataPipe | MapDataPipe) → IterDataPipe | MapDataPipe

ReadingService takes a DataPipe graph, adapts it into a new DataPipe graph based on the custom need. Called once in creating DataLoader2 iterator at first time. Prior to calling this method, the ReadingService object must be picklable.

Parameters:

datapipe – Original DataPipe graph.

Returns:

An adapted or a new DataPipe graph.