||The scikit-image toolkit
How are images represented in scientific computing?
How can we detect useful features from an image?
How can we segment an MRI image into different sections?
An interdisciplinary field that deals with gaining high-level understanding from digital images or videos.
Computer vision tasks include methods for acquiring, processing, and analyzing digital images.
For our purposes: multi-dimensional homogeneous data in which spatial relationships matter
- spatial contiguity is meaningful
- usually 3D images or 2D projections
Color deconvolution for immunohistochemical staining
Find the coordinate transform between two independent images
Reconstruct 3D white matter tracts from diffusion tensor imaging (DTI)
Deep learning based segmentation of edema for optical coherence tomography (OCT) images of the retinagithub.com/uw-biomedical-ml/irf-segmenter
scikit-image is a collection of Python algorithms for image processing, developed by the SciPy community.
Installation via conda:
$ conda install scikit-image
More info at http://scikit-image.org.
The main package of
skimage only provides some utility functions - everything else is in submodules:
||test images and example data|
||reading, saving, displaying images and videos|
||color space conversion|
||sharpening, edge finding, rank filters, thresholding|
||measuring image properties (e.g., similarity, contours)|
||feature detection and extraction|
||restoration algorithms (e.g., deconvolution denoising)|
||partitioning an image into multiple regions|
Pixels are array elements:
Values 0 - 255: 0 is black, 255 is white (
import numpy as np image = np.ones((5, 5)) image[0, 0] = 0 image[2, :] = 0
from skimage import io img_rgb = io.imread(path_to_rgb_file) img_gray = io.imread(path_to_rgb_file, as_grey=True)
Note: Since scikit-image operates on NumPy array, any image reader library that provides arrays will do (e.g., imageio, matplotlib, pillow).
img = np.random.rand((500, 500)) io.imsave("noise.jpg", img)
It's easy to draw on images, too.
||Generate line pixel coordinates|
||Generate coordinates of pixels within a circle|
||Generate coordinates of pixels within a polygon|
||Generate an image with random shapes, labeled with bounding boxes|
(adapted from /skimage-tutorials/lectures/00_images_are_arrays.ipynb)
$ git clone https://github.com/mbeyeler/2018-neurohack-skimage.git
One of the most basic and common image operations - used to remove noise or enhance features.
||Multi-dimensional Gaussian filter|
||Find the edge magnitude using the Sobel transform|
||Return local median of an image|
||Returns real and imaginary responses to Gabor filter|
from skimage import data, filters img = data.camera() img_blurred = filters.gaussian_filter(img, sigma=5)
from skimage import data, filters img = data.camera() img_edges = filters.sobel(img)
(can also be used to generate a fun cartoon effect)
from skimage import io, filters, morphology img = io.imread("teddy.jpg") img_blurred = filters.median_filter(img selem=morphology.disk(3) )
Methods for computing abstractions of image information
|Edges||points where there is a boundary between two image regions|
|Corners||point-like features with 2D structure|
|Blobs||regions of interest|
used to detect lines, circles or other parametric curvesThe locus of (a, b) points in parameter space fall on a circle with radius R centered at (x, y). The true center point will be common to all parameter circles, and can be found with a Hough accumulation array.
sum up intensities in small rectangles, calculate the difference between the sums
||Harris corner measure|
||Histogram of oriented gradients (HOG)|
||Oriented FAST and rotated BRIEF feature detector|
Supervised: some prior knowledge is used to guide the algorithm.
Unsupervised: no prior knowledge given.
Thresholding is used to create a binary image from a grayscale image.
Otsu's method calculates an "optimal" threshold that maximizes the variance between two classes of pixels.
Anisotropic diffusion with tracers initiated at the markers' positionLocal diffusivity coefficient is greater if neighboring pixels have similar values, so that diffusion is difficult across high gradients Markers label different phases, like below (too noisy for thresholding).
Starting from user-defined markers, watershed treats pixel values as a local topography, then floods basins from the markers.Markers are often chosen as local minima of the image, from which basins are flooded.
A spatially localized version of k-means clustering.Pixels are represented in 5D (x, y, L, a, b) using the CIELAB color space. k-means: Given a set of k cluster centers, assign each pixel to the nearest one. Update the new cluster centers to be the means of the pixel centroids in the cluster. Repeat until convergence.
||Similar to SLIC: hierarchical segmentation in 5D space|
||Designed to segment objects without clearly defined boundaries|
||Spanning tree based clustering|
||Region adjacency graph (RAG) based graph cuts|