torch_em.data.datasets.electron_microscopy.isbi2012
The ISBI2012 dataset was the first neuron segmentation challenge, held at the ISBI 2012 competition.
It contains a small annotated EM volume from the fruit-fly brain. If you use this dataset in your research please cite the following publication: https://doi.org/10.3389/fnana.2015.00142.
1"""The ISBI2012 dataset was the first neuron segmentation challenge, held at the ISBI 2012 competition. 2 3It contains a small annotated EM volume from the fruit-fly brain. If you use this dataset in 4your research please cite the following publication: https://doi.org/10.3389/fnana.2015.00142. 5""" 6 7import os 8from typing import List, Optional, Tuple, Union 9 10import torch_em 11from .. import util 12 13ISBI_URL = "https://oc.embl.de/index.php/s/h0TkwqxU0PJDdMd/download" 14CHECKSUM = "0e10fe909a1243084d91773470856993b7d40126a12e85f0f1345a7a9e512f29" 15 16 17def get_isbi_data(path: Union[os.PathLike, str], download: bool) -> str: 18 """Download the ISBI2012 dataset. 19 20 Args: 21 path: Filepath to a folder where the downloaded data will be saved. 22 download: Whether to download the data if it is not present. 23 24 Returns: 25 The path to the downloaded data. 26 """ 27 if path.endswith(".h5"): 28 volume_path = path 29 else: 30 os.makedirs(path, exist_ok=True) 31 volume_path = os.path.join(path, "isbi.h5") 32 util.download_source(volume_path, ISBI_URL, download, CHECKSUM) 33 return volume_path 34 35 36def get_isbi_dataset( 37 path: Union[os.PathLike, str], 38 patch_shape: Tuple[int, int, int], 39 download: bool = False, 40 offsets: Optional[List[List[int]]] = None, 41 boundaries: bool = False, 42 use_original_labels: bool = False, 43 **kwargs 44): 45 """Get the dataset for EM neuron segmentation in ISBI 2012. 46 47 Args: 48 path: Filepath to a folder where the downloaded data will be saved. 49 patch_shape: The patch shape to use for training. 50 download: Whether to download the data if it is not present. 51 offsets: Offset values for affinity computation used as target. 52 boundaries: Whether to compute boundaries as the target. 53 use_original_labels: Whether to use the original annotations or postprocessed 3d annotations. 54 kwargs: Additional keyword arguments for `torch_em.default_segmentation_dataset`. 55 56 Returns: 57 The segmentation dataset. 58 """ 59 assert len(patch_shape) == 3 60 volume_path = get_isbi_data(path, download) 61 62 ndim = 2 if patch_shape[0] == 1 else 3 63 kwargs = util.update_kwargs(kwargs, "ndim", ndim) 64 65 kwargs, _ = util.add_instance_label_transform( 66 kwargs, add_binary_target=False, boundaries=boundaries, offsets=offsets 67 ) 68 69 raw_key = "raw" 70 label_key = "labels/membranes" if use_original_labels else "labels/gt_segmentation" 71 72 return torch_em.default_segmentation_dataset(volume_path, raw_key, volume_path, label_key, patch_shape, **kwargs) 73 74 75def get_isbi_loader( 76 path: Union[os.PathLike, str], 77 patch_shape: Tuple[int, int, int], 78 batch_size: int, 79 download: bool = False, 80 offsets: Optional[List[List[int]]] = None, 81 boundaries: bool = False, 82 use_original_labels: bool = False, 83 **kwargs 84): 85 """Get the DataLoader for EM neuron segmentation in ISBI 2012. 86 87 Args: 88 path: Filepath to a folder where the downloaded data will be saved. 89 patch_shape: The patch shape to use for training. 90 batch_size: The batch size for training. 91 download: Whether to download the data if it is not present. 92 offsets: Offset values for affinity computation used as target. 93 boundaries: Whether to compute boundaries as the target. 94 use_original_labels: Whether to use the original annotations or postprocessed 3d annotations. 95 kwargs: Additional keyword arguments for `torch_em.default_segmentation_dataset` or for the PyTorch DataLoader. 96 97 Returns: 98 The DataLoader. 99 """ 100 ds_kwargs, loader_kwargs = util.split_kwargs( 101 torch_em.default_segmentation_dataset, **kwargs 102 ) 103 dataset = get_isbi_dataset( 104 path, patch_shape, download=download, 105 offsets=offsets, boundaries=boundaries, use_original_labels=use_original_labels, 106 **ds_kwargs 107 ) 108 loader = torch_em.get_data_loader(dataset, batch_size, **loader_kwargs) 109 return loader
ISBI_URL =
'https://oc.embl.de/index.php/s/h0TkwqxU0PJDdMd/download'
CHECKSUM =
'0e10fe909a1243084d91773470856993b7d40126a12e85f0f1345a7a9e512f29'
def
get_isbi_data(path: Union[os.PathLike, str], download: bool) -> str:
18def get_isbi_data(path: Union[os.PathLike, str], download: bool) -> str: 19 """Download the ISBI2012 dataset. 20 21 Args: 22 path: Filepath to a folder where the downloaded data will be saved. 23 download: Whether to download the data if it is not present. 24 25 Returns: 26 The path to the downloaded data. 27 """ 28 if path.endswith(".h5"): 29 volume_path = path 30 else: 31 os.makedirs(path, exist_ok=True) 32 volume_path = os.path.join(path, "isbi.h5") 33 util.download_source(volume_path, ISBI_URL, download, CHECKSUM) 34 return volume_path
Download the ISBI2012 dataset.
Arguments:
- path: Filepath to a folder where the downloaded data will be saved.
- download: Whether to download the data if it is not present.
Returns:
The path to the downloaded data.
def
get_isbi_dataset( path: Union[os.PathLike, str], patch_shape: Tuple[int, int, int], download: bool = False, offsets: Optional[List[List[int]]] = None, boundaries: bool = False, use_original_labels: bool = False, **kwargs):
37def get_isbi_dataset( 38 path: Union[os.PathLike, str], 39 patch_shape: Tuple[int, int, int], 40 download: bool = False, 41 offsets: Optional[List[List[int]]] = None, 42 boundaries: bool = False, 43 use_original_labels: bool = False, 44 **kwargs 45): 46 """Get the dataset for EM neuron segmentation in ISBI 2012. 47 48 Args: 49 path: Filepath to a folder where the downloaded data will be saved. 50 patch_shape: The patch shape to use for training. 51 download: Whether to download the data if it is not present. 52 offsets: Offset values for affinity computation used as target. 53 boundaries: Whether to compute boundaries as the target. 54 use_original_labels: Whether to use the original annotations or postprocessed 3d annotations. 55 kwargs: Additional keyword arguments for `torch_em.default_segmentation_dataset`. 56 57 Returns: 58 The segmentation dataset. 59 """ 60 assert len(patch_shape) == 3 61 volume_path = get_isbi_data(path, download) 62 63 ndim = 2 if patch_shape[0] == 1 else 3 64 kwargs = util.update_kwargs(kwargs, "ndim", ndim) 65 66 kwargs, _ = util.add_instance_label_transform( 67 kwargs, add_binary_target=False, boundaries=boundaries, offsets=offsets 68 ) 69 70 raw_key = "raw" 71 label_key = "labels/membranes" if use_original_labels else "labels/gt_segmentation" 72 73 return torch_em.default_segmentation_dataset(volume_path, raw_key, volume_path, label_key, patch_shape, **kwargs)
Get the dataset for EM neuron segmentation in ISBI 2012.
Arguments:
- path: Filepath to a folder where the downloaded data will be saved.
- patch_shape: The patch shape to use for training.
- download: Whether to download the data if it is not present.
- offsets: Offset values for affinity computation used as target.
- boundaries: Whether to compute boundaries as the target.
- use_original_labels: Whether to use the original annotations or postprocessed 3d annotations.
- kwargs: Additional keyword arguments for
torch_em.default_segmentation_dataset
.
Returns:
The segmentation dataset.
def
get_isbi_loader( path: Union[os.PathLike, str], patch_shape: Tuple[int, int, int], batch_size: int, download: bool = False, offsets: Optional[List[List[int]]] = None, boundaries: bool = False, use_original_labels: bool = False, **kwargs):
76def get_isbi_loader( 77 path: Union[os.PathLike, str], 78 patch_shape: Tuple[int, int, int], 79 batch_size: int, 80 download: bool = False, 81 offsets: Optional[List[List[int]]] = None, 82 boundaries: bool = False, 83 use_original_labels: bool = False, 84 **kwargs 85): 86 """Get the DataLoader for EM neuron segmentation in ISBI 2012. 87 88 Args: 89 path: Filepath to a folder where the downloaded data will be saved. 90 patch_shape: The patch shape to use for training. 91 batch_size: The batch size for training. 92 download: Whether to download the data if it is not present. 93 offsets: Offset values for affinity computation used as target. 94 boundaries: Whether to compute boundaries as the target. 95 use_original_labels: Whether to use the original annotations or postprocessed 3d annotations. 96 kwargs: Additional keyword arguments for `torch_em.default_segmentation_dataset` or for the PyTorch DataLoader. 97 98 Returns: 99 The DataLoader. 100 """ 101 ds_kwargs, loader_kwargs = util.split_kwargs( 102 torch_em.default_segmentation_dataset, **kwargs 103 ) 104 dataset = get_isbi_dataset( 105 path, patch_shape, download=download, 106 offsets=offsets, boundaries=boundaries, use_original_labels=use_original_labels, 107 **ds_kwargs 108 ) 109 loader = torch_em.get_data_loader(dataset, batch_size, **loader_kwargs) 110 return loader
Get the DataLoader for EM neuron segmentation in ISBI 2012.
Arguments:
- path: Filepath to a folder where the downloaded data will be saved.
- patch_shape: The patch shape to use for training.
- batch_size: The batch size for training.
- download: Whether to download the data if it is not present.
- offsets: Offset values for affinity computation used as target.
- boundaries: Whether to compute boundaries as the target.
- use_original_labels: Whether to use the original annotations or postprocessed 3d annotations.
- kwargs: Additional keyword arguments for
torch_em.default_segmentation_dataset
or for the PyTorch DataLoader.
Returns:
The DataLoader.