Connected Component Analysis
Last updated on 2023-09-09 | Edit this page
Estimated time 125 minutes
- How to extract separate objects from an image and describe these objects quantitatively.
- Understand the term object in the context of images.
- Learn about pixel connectivity.
- Learn how Connected Component Analysis (CCA) works.
- Use CCA to produce an image that highlights every object in a different colour.
- Characterise each object with numbers that describe its appearance.
In the Thresholding episode we have covered dividing an image into foreground and background pixels. In the shapes example image, we considered the coloured shapes as foreground objects on a white background.
In thresholding we went from the original image to this version:
Here, we created a mask that only highlights the parts of the image
that we find interesting, the objects. All objects have pixel
True while the background pixels are
By looking at the mask image, one can count the objects that are present in the image (7). But how did we actually do that, how did we decide which lump of pixels constitutes a single object?
In order to decide which pixels belong to the same object, one can exploit their neighborhood: pixels that are directly next to each other and belong to the foreground class can be considered to belong to the same object.
Let’s discuss the concept of pixel neighborhoods in more detail.
Consider the following mask “image” with 8 rows, and 8 columns. For the
purpose of illustration, the digit
0 is used to represent
background pixels, and the letter
X is used to represent
object pixels foreground).
0 0 0 0 0 0 0 0 0 X X 0 0 0 0 0 0 X X 0 0 0 0 0 0 0 0 X X X 0 0 0 0 0 X X X X 0 0 0 0 0 0 0 0 0
The pixels are organised in a rectangular grid. In order to
understand pixel neighborhoods we will introduce the concept of “jumps”
between pixels. The jumps follow two rules: First rule is that one jump
is only allowed along the column, or the row. Diagonal jumps are not
allowed. So, from a centre pixel, denoted with
o, only the
pixels indicated with a
1 are reachable:
- 1 - 1 o 1 - 1 -
The pixels on the diagonal (from
o) are not reachable
with a single jump, which is denoted by the
-. The pixels
reachable with a single jump form the 1-jump
The second rule states that in a sequence of jumps, one may only jump
in row and column direction once -> they have to be
orthogonal. An example of a sequence of orthogonal jumps is
shown below. Starting from
o the first jump goes along the
row to the right. The second jump then goes along the column direction
up. After this, the sequence cannot be continued as a jump has already
been made in both row and column direction.
- - 2 - o 1 - - -
All pixels reachable with one, or two jumps form the
2-jump neighborhood. The grid below illustrates the
pixels reachable from the centre pixel
o with a single
jump, highlighted with a
1, and the pixels reachable with 2
jumps with a
2 1 2 1 o 1 2 1 2
We want to revisit our example image mask from above and apply the
two different neighborhood rules. With a single jump connectivity for
each pixel, we get two resulting objects, highlighted in the image with
0 0 0 0 0 0 0 0 0 A A 0 0 0 0 0 0 A A 0 0 0 0 0 0 0 0 B B B 0 0 0 0 0 B B B B 0 0 0 0 0 0 0 0 0
In the 1-jump version, only pixels that have direct neighbors along
rows or columns are considered connected. Diagonal connections are not
included in the 1-jump neighborhood. With two jumps, however, we only
get a single object
A because pixels are also considered
connected along the diagonals.
0 0 0 0 0 0 0 0 0 A A 0 0 0 0 0 0 A A 0 0 0 0 0 0 0 0 A A A 0 0 0 0 0 A A A A 0 0 0 0 0 0 0 0 0
We have just introduced how you can reach different neighboring pixels by performing one or more orthogonal jumps. We have used the terms 1-jump and 2-jump neighborhood. There is also a different way of referring to these neighborhoods: the 4- and 8-neighborhood. With a single jump you can reach four pixels from a given starting pixel. Hence, the 1-jump neighborhood corresponds to the 4-neighborhood. When two orthogonal jumps are allowed, eight pixels can be reached, so the 2-jump neighborhood corresponds to the 8-neighborhood.
In order to find the objects in an image, we want to employ an
operation that is called Connected Component Analysis (CCA). This
operation takes a binary image as an input. Usually, the
False value in this image is associated with background
pixels, and the
True value indicates foreground, or object
pixels. Such an image can be produced, e.g., with thresholding. Given a
thresholded image, the connected component analysis produces a new
labeled image with integer pixel values. Pixels with the same
value, belong to the same object. scikit-image provides connected
component analysis in the function
us add this function to the already familiar steps of thresholding an
First, import the packages needed for this episode:
import imageio.v3 as iio import ipympl import matplotlib.pyplot as plt import numpy as np import skimage as ski %matplotlib widget
In this episode, we will use the
function to perform the CCA.
Next, we define a reusable Python function
def connected_components(filename, sigma=1.0, t=0.5, connectivity=2): # load the image = iio.imread(filename) image # convert the image to grayscale = ski.color.rgb2gray(image) gray_image # denoise the image with a Gaussian filter = ski.filters.gaussian(gray_image, sigma=sigma) blurred_image # mask the image according to threshold = blurred_image < t binary_mask # perform connected component analysis = ski.measure.label(binary_mask, labeled_image, count =connectivity, return_num=True) connectivityreturn labeled_image, count
The first four lines of code are familiar from the Thresholding episode.
Then we call the
ski.measure.label function. This
function has one positional argument where we pass the
binary_mask, i.e., the binary image to work on. With the
connectivity, we specify the neighborhood
in units of orthogonal jumps. For example, by setting
connectivity=2 we will consider the 2-jump neighborhood
introduced above. The function returns a
where each pixel has a unique value corresponding to the object it
belongs to. In addition, we pass the optional parameter
return_num=True to return the maximum label index as
The optional parameter
return_num changes the data type
that is returned by the function
number of labels is only returned if
True. Otherwise, the function only returns the labeled image.
This means that we have to pay attention when assigning the return value
to a variable. If we omit the optional parameter
return_num=False, we can call the function as
If we pass
return_num=True, the function returns a tuple
and we can assign it as
= ski.measure.label(binary_mask, return_num=True)labeled_image, count
If we used the same assignment as in the first case, the variable
labeled_image would become a tuple, in which
labeled_image is the image and
labeled_image is the number of labels. This could cause
confusion if we assume that
labeled_image only contains the
image and pass it to other functions. If you get an
AttributeError: 'tuple' object has no attribute 'shape' or
similar, check if you have assigned the return values consistently with
the optional parameters.
We can call the above function
display the labeled image like so:
= connected_components(filename="data/shapes-01.jpg", sigma=2.0, t=0.9, connectivity=2) labeled_image, count = plt.subplots() fig, ax plt.imshow(labeled_image)"off");plt.axis(
Here you might get a warning
UserWarning: Low image data range; displaying image with stretched contrast.
or just see an all black image (Note: this behavior might change in
future versions or not occur with a different image viewer).
What went wrong? When you hover over the black image, the pixel
values are shown as numbers in the lower corner of the viewer. You can
see that some pixels have values different from
0, so they
are not actually pure black. Let’s find out more by examining
labeled_image. Properties that might be interesting in this
dtype, the minimum and maximum value. We can
print them with the following lines:
print("dtype:", labeled_image.dtype) print("min:", np.min(labeled_image)) print("max:", np.max(labeled_image))
Examining the output can give us a clue why the image appears black.
dtype: int32 min: 0 max: 11
int64. This means that values in this image range from
-2 ** 63 to
2 ** 63 - 1. Those are really big
numbers. From this available space we only use the range from
11. When showing this image in the
viewer, it squeezes the complete range into 256 gray values. Therefore,
the range of our numbers does not produce any visible change.
Fortunately, the scikit-image library has tools to cope with this situation.
We can use the function
ski.color.label2rgb() to convert
the colours in the image (recall that we already used the
ski.color.rgb2gray() function to convert to grayscale).
ski.color.label2rgb(), all objects are coloured
according to a list of colours that can be customised. We can use the
following commands to convert and show the image:
# convert the label image to color image = ski.color.label2rgb(labeled_image, bg_label=0) colored_label_image = plt.subplots() fig, ax plt.imshow(colored_label_image)"off");plt.axis(
Now, it is your turn to practice. Using the function
connected_components, find two ways of printing out the
number of objects found in the image.
What number of objects would you expect to get?
How does changing the
values influence the result?
As you might have guessed, the return value
already contains the number of found images. So it can simply be printed
print("Found", count, "objects in the image.")
But there is also a way to obtain the number of found objects from
the labeled image itself. Recall that all pixels that belong to a single
object are assigned the same integer value. The connected component
algorithm produces consecutive numbers. The background gets the value
0, the first object gets the value
second object the value
2, and so on. This means that by
finding the object with the maximum value, we also know how many objects
there are in the image. We can thus use the
from NumPy to find the maximum value that equals the number of found
= np.max(labeled_image) num_objects print("Found", num_objects, "objects in the image.")
Invoking the function with
threshold=0.9, both methods will print
Found 11 objects in the image.
Lowering the threshold will result in fewer objects. The higher the threshold is set, the more objects are found. More and more background noise gets picked up as objects. Larger sigmas produce binary masks with less noise and hence a smaller number of objects. Setting sigma too high bears the danger of merging objects.
You might wonder why the connected component analysis with
threshold=0.9 finds 11 objects,
whereas we would expect only 7 objects. Where are the four additional
objects? With a bit of detective work, we can spot some small objects in
the image, for example, near the left border.
For us it is clear that these small spots are artifacts and not
objects we are interested in. But how can we tell the computer? One way
to calibrate the algorithm is to adjust the parameters for blurring
sigma) and thresholding (
t), but you may have
noticed during the above exercise that it is quite hard to find a
combination that produces the right output number. In some cases,
background noise gets picked up as an object. And with other parameters,
some of the foreground objects get broken up or disappear completely.
Therefore, we need other criteria to describe desired properties of the
objects that are found.
Morphometrics is concerned with the quantitative analysis of objects
and considers properties such as size and shape. For the example of the
images with the shapes, our intuition tells us that the objects should
be of a certain size or area. So we could use a minimum area as a
criterion for when an object should be detected. To apply such a
criterion, we need a way to calculate the area of objects found by
connected components. Recall how we determined the root mass in the Thresholding episode by
counting the pixels in the binary mask. But here we want to calculate
the area of several objects in the labeled image. The scikit-image
library provides the function
measure the properties of labeled regions. It returns a list of
RegionProperties that describe each connected region in the
images. The properties can be accessed using the attributes of the
RegionProperties data type. Here we will use the properties
"label". You can explore the
scikit-image documentation to learn about other properties
We can get a list of areas of the labeled objects as follows:
# compute object features and extract object areas = ski.measure.regionprops(labeled_image) object_features = [objf["area"] for objf in object_features] object_areas object_areas
This will produce the output
[318542, 1, 523204, 496613, 517331, 143, 256215, 1, 68, 338784, 265755]
Similar to how we determined a “good” threshold in the Thresholding episode, it is often helpful to inspect the histogram of an object property. For example, we want to look at the distribution of the object areas.
- Create and examine a histogram of the object areas
- What does the histogram tell you about the objects?
The histogram can be plotted with
= plt.subplots() fig, ax plt.hist(object_areas)"Area (pixels)") plt.xlabel("Number of objects");plt.ylabel(
The histogram shows the number of objects (vertical axis) whose area is within a certain range (horizontal axis). The height of the bars in the histogram indicates the prevalence of objects with a certain area. The whole histogram tells us about the distribution of object sizes in the image. It is often possible to identify gaps between groups of bars (or peaks if we draw the histogram as a continuous curve) that tell us about certain groups in the image.
In this example, we can see that there are four small objects that
contain less than 50000 pixels. Then there is a group of four (1+1+2)
objects in the range between 200000 and 400000, and three objects with a
size around 500000. For our object count, we might want to disregard the
small objects as artifacts, i.e, we want to ignore the leftmost bar of
the histogram. We could use a threshold of 50000 as the minimum area to
count. In fact, the
object_areas list already tells us that
there are fewer than 200 pixels in these objects. Therefore, it is
reasonable to require a minimum area of at least 200 pixels for a
detected object. In practice, finding the “right” threshold can be
tricky and usually involves an educated guess based on domain
One way to count only objects above a certain area is to first create a list of those objects, and then take the length of that list as the object count. This can be done as follows:
= 200 min_area =  large_objects for objf in object_features: if objf["area"] > min_area: "label"]) large_objects.append(objf[print("Found", len(large_objects), "objects!")
Another option is to use NumPy arrays to create the list of large
objects. We first create an array
the object areas, and an array
object_labels containing the
object labels. The labels of the objects are also returned by
ski.measure.regionprops. We have already seen that we can
create boolean arrays using comparison operators. Here we can use
object_areas > min_area to produce an array that has the
same dimension as
object_labels. It can then used to select
the labels of objects whose area is greater than
= np.array([objf["area"] for objf in object_features]) object_areas = np.array([objf["label"] for objf in object_features]) object_labels = object_labels[object_areas > min_area] large_objects print("Found", len(large_objects), "objects!")
The advantage of using NumPy arrays is that
if statements in Python can be slow, and in practice
the first approach may not be feasible if the image contains a large
number of objects. In that case, NumPy array functions turn out to be
very useful because they are much faster.
In this example, we can also use the
function that we have seen earlier together with the
operator to count the objects whose area is above
= np.count_nonzero(object_areas > min_area) n print("Found", n, "objects!")
For all three alternatives, the output is the same and gives the expected count of 7 objects.
To remove the small objects from the labeled image, we change the value of all pixels that belong to the small objects to the background label 0. One way to do this is to loop over all objects and set the pixels that match the label of the object to 0.
for object_id, objf in enumerate(object_features, start=1): if objf["area"] < min_area: == objf["label"]] = 0 labeled_image[labeled_image
Here NumPy functions can also be used to eliminate
if statements. Like above, we can create an array
of the small object labels with the comparison
object_areas < min_area. We can use another NumPy
np.isin, to set the pixels of all small objects
np.isin takes two arrays and returns a boolean array
True if the entry of the first array is found
in the second array, and
False otherwise. This array can
then be used to index the
labeled_image and set the entries
that belong to small objects to
= np.array([objf["area"] for objf in object_features]) object_areas = np.array([objf["label"] for objf in object_features]) object_labels = object_labels[object_areas < min_area] small_objects = 0labeled_image[np.isin(labeled_image,small_objects)]
An even more elegant way to remove small objects from the image is to
ski.morphology module. It provides a function
ski.morphology.remove_small_objects that does exactly what
we are looking for. It can be applied to a binary image and returns a
mask in which all objects smaller than
excluded, i.e, their pixel values are set to
False. We can
ski.measure.label to the masked image:
= ski.morphology.remove_small_objects(binary_mask,min_area) object_mask = ski.measure.label(object_mask, labeled_image, n =connectivity, return_num=True) connectivity
Using the scikit-image features, we can implement the
enhanced_connected_component as follows:
def enhanced_connected_components(filename, sigma=1.0, t=0.5, connectivity=2, min_area=0): = iio.imread(filename) image = ski.color.rgb2gray(image) gray_image = ski.filters.gaussian(gray_image, sigma=sigma) blurred_image = blurred_image < t binary_mask = ski.morphology.remove_small_objects(binary_mask,min_area) object_mask = ski.measure.label(object_mask, labeled_image, count =connectivity, return_num=True) connectivityreturn labeled_image, count
We can now call the function with a chosen
display the resulting labeled image:
= enhanced_connected_components(filename="data/shapes-01.jpg", sigma=2.0, t=0.9, labeled_image, count =2, min_area=min_area) connectivity= ski.color.label2rgb(labeled_image, bg_label=0) colored_label_image = plt.subplots() fig, ax plt.imshow(colored_label_image)"off"); plt.axis( print("Found", count, "objects in the image.")
Found 7 objects in the image.
Note that the small objects are “gone” and we obtain the correct number of 7 objects in the image.
We already know how to get the areas of the objects from the
regionprops. We just need to insert a zero area value for
the background (to colour it like a zero size object). The background is
0 in the
labeled_image, so we
insert the zero area value in front of the first element of
np.insert. Then we can
colored_area_image where we assign each pixel
value the area by indexing the
object_areas with the label
= np.array([objf["area"] for objf in ski.measure.regionprops(labeled_image)]) object_areas = np.insert(0,1,object_areas) object_areas = object_areas[labeled_image] colored_area_image = plt.subplots() fig, ax = plt.imshow(colored_area_image) im = fig.colorbar(im, ax=ax, shrink=0.85) cbar "Area") cbar.ax.set_title("off");plt.axis(
You may have noticed that in the solution, we have used the
labeled_image to index the array
This is an example of advanced
indexing in NumPy The result is an array of the same shape as the
labeled_image whose pixel values are selected from
object_areas according to the object label. Hence the
objects will be colored by area when the result is displayed. Note that
advanced indexing with an integer array works slightly different than
the indexing with a Boolean array that we have used for masking. While
Boolean array indexing returns only the entries corresponding to the
True values of the index, integer array indexing returns an
array with the same shape as the index. You can read more about advanced
indexing in the NumPy
- We can use
ski.measure.labelto find and label connected objects in an image.
- We can use
ski.measure.regionpropsto measure properties of labeled objects.
- We can use
ski.morphology.remove_small_objectsto mask small objects and remove artifacts from an image.
- We can display the labeled image to view the objects coloured by label.