Data
Mimics torch.data.Dataset for ray.data integration
RayDataset
¶
map_(self, func, *args, **kwargs)
¶
Inplace Map for ray.data Time complexity: O(dataset size / parallelism)
See https://docs.ray.io/en/latest/data/dataset.html#transforming-datasets
map_batch_(self, func, batch_size=2, **kwargs)
¶
Inplace Map for ray.data Time complexity: O(dataset size / parallelism) See https://docs.ray.io/en/latest/data/dataset.html#transforming-datasets
RayImageFolder
¶
Read image datasets
root/dog/xxx.png
root/dog/xxy.png
root/dog/[...]/xxz.png
root/cat/123.png
root/cat/nsdf3.png
root/cat/[...]/asd932_.png
Data loader for image dataset
image_dataset_from_directory(directory, transform=None, image_size=(224, 224), batch_size=1, shuffle=False, pin_memory=True, num_workers=None, ray_data=False)
¶
Create Dataset and Dataloader for image folder dataset.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
directory |
Union[List[str], pathlib.Path, str] |
required | |
transform |
|
None |
|
image_size |
|
(224, 224) |
|
batch_size |
int |
1 |
|
shuffle |
bool |
False |
|
pin_memory |
bool |
True |
|
num_workers |
Optional[int] |
None |
Returns:
Type | Description |
---|---|
Dict[str, Union[gradsflow.data.ray_dataset.RayDataset, torch.utils.data.dataloader.DataLoader]] |
A dictionary containing dataset and dataloader. |
Last update: September 24, 2021