How to cite:
Daniel Martomanggolo Wonohadidjojo. (2022). Classification of
Bacterial Images Using Transfer Learning, Optimized Training and
Resnet-50. Journal Eduvest. Vol 2(2): 297-305
E-ISSN:
2775-3727
Published by:
https://greenpublisher.id/
Eduvest – Journal of Universal Studies
Volume 2 Number 2, February 2022
p- ISSN 2775-3735 e-ISSN 2775-3727
CLASSIFICATION OF BACTERIAL IMAGES
USING TRANSFER LEARNING, OPTIMIZED
TRAINING AND RESNET-50
ARTICLE INFO ABSTRACT
Received:
January, 26
th
2022
Revised:
February, 17
th
2022
Approved:
February, 18
th
2022
Bacterial image analysis using traditional laboratory
methods encounters bacterial recognition errors and
requires extra experience and long processing time.
Therefore, the automated classification technique of
bacterial images is more useful than traditional visual
observations for biologists because of their accurate
classification, low cost, and fast diagnosis. In this study, a
method to classify bacteria images by implementing the
CNN deep learning method using Transfer Learning is
proposed. This trained ResNet-50 is implemented as the
CNN architecture. In the training of the classification layer,
SGDM optimizer is used. The classification performance for
is evaluated in using confusion matrix and four
performance metrics: Accuracy, Precision, Recall and
Fmeasure. The Confusion Matrix and all the performance
metrics show is successful in classifying bacterial images.
KEYWORDS
Bacteria classification, Convolutional Neural Network,
ResNet-50, Transfer Learning.
This work is licensed under a Creative Commons
Attribution-ShareAlike 4.0 International
INTRODUCTION
The recognition of bacterial samples carried out in culturing process requires
equipment and chemical substance used for staining the samples. Moreover, rigorous
culturing procedures and safety protocols must be met. After that the results are obtained,
Daniel Martomanggolo Wonohadidjojo
Classification of Bacterial Images Using Transfer Learning, Optimized Training and
Resnet-50 298
the samples need to be analysed to find characteristic features and to classify the
particular genera and species of bacteria. Using this classical method in the laboratory,
the bacteria recognition requires knowledge and experience of experts. The problem
encountered using this method are the time-consuming process and the requirement of
comparative analysis of the obtained samples with the reference ones (Zieliński et al.,
2017).
Biologists identify and classify types of bacteria based on different biochemicals
and forms. These are the bacterial attributes which are used for classification. The shape
of cell, the size and structure of the colonies formed by the bacteria are used to
distinguish bacterial species. The bacterial cell can be spiral, cylindrical, and spherical.
Cells of several types of bacteria have different sizes and structures depending on
environmental conditions (Handoyo & Mulyandari, 2021). Several species of bacteria
have very similar shapes. Each bacterial species has its own characteristics, the
biochemical reactions and the metabolic activities carried out by the bacteria help to
classify the species. However, the task to classify bacterial species is not easy even for an
experienced specialist (Talo, 2019).
In general, microbiological image analysis using traditional laboratory methods
encounters bacterial recognition errors and requires extra experience and long processing
time. Therefore, the automated classification technique of bacterial images is more useful
than traditional visual observations for biologists because of its accurate classification,
low cost, and fast diagnosis. Several studies related to bacterial classification have been
conducted and several methods have been proposed. In 2018 a classification method
using CNN with the AlexNET architecture was proposed. It was used to classify 18
categories of bacteria with an accuracy of 73 % (Huang & Wu, 2018).
Then in 2019 another study to classify two types of bacteria using deep learning
method with LeNET architecture was conducted with more than 75 % accuracy of the
results (Treebupachatsakul & Poomrittigul, 2019). Deep learning increases
its remarkable position in computer vision when neural systems began to
outperform other strategies on a few high profile image analysis benchmarks (Lundervold
& Lundervold, 2019). The CNN methods have been found to either outperform or
complement the existing traditional and Machine Learning approaches in terms of
accuracy, sensitivity, Area Under Curve, Dice Similarity Coefficient, and time taken
(Sarvamangala & Kulkarni, 2021).
In this study, a method to classify bacteria by implementing the CNN deep
learning method using Resnet-50 architecture is proposed. The ResNet-50 architecture is
implemented in this study with the consideration of its performance. Previous studies
shows its performance in classifying digital images with high accuracy results such as the
classification of brain tumors (Mohsen, El-Dahshan, El-Horbaty, & Salem, 2018), malaria
infected cells (Sai Bharadwaj Reddy & Sujitha Juliet, 2019), and protein cells (Elnashar,
Abd, & Azim, 2021). Furthermore, in this study the training process in Transfer Learning
is optimized using Stochastic Gradient Descent with Momentum (SGDM) algorithm. The
use of SGDM in machine learning task has been verified in several studies. A stochastic
optimization algorithm performs better in terms of expected error, and hence, makes a
better learning algorithm (Bottou, Curtis, & Nocedal, 2018).
Eduvest – Journal of Universal Studies
Volume 2 Number 2, February 2022
299 http://eduvest.greenvest.co.id
RESEARCH METHOD
The trained network used in this transfer learning is the CNN model. CNN is a
type of neural network which uses special technique called convolution. CNN has 2
major layers, namely the layer for feature extraction and the layer for classification.
Layers for feature extraction consist of convolutional layers, pooling layers, stride, and
padding. Meanwhile, the layers for classification consists of a fully connected layer,
softmax, and an output layer.
ResNet-50 is an architecture from CNN which introduces a new concept, namely
shortcut connections. The emergence of the concept of shortcut connections in the
ResNet-50 architecture is related to the vanishing gradient problem that occurs when
efforts to deepen the structure of a network are carried out. However, deepening a
network with the aim of improving its performance cannot be done simply by stacking
layers. The deeper a network can lead to a vanishing gradient problem that can make the
gradient very small which results in decreased performance or accuracy.
ResNet introduces the concept of shortcut connections and in this concept
features which are inputs from the previous layer are also used as inputs to the outputs of
that layer. This method is implemented as a solution to minimize the loss of important
features during the convolution process.
This trained network is used in the transfer learning method in this study. The
transfer learning stage used in this study includes importing a previously trained network,
replacing the classification layer, training the network on the new dataset whish is the
bacteria cell images, classifying the bacteria and getting accurate performance measures.
Figure 1 shows the transfer learning method used in this study.
Figure 1. The Transfer Learning Method.
Classification of Bacterial Images Using Transfer Learning, Optimized Training and
Resnet-50 300
A. Importing the Trained Network
In this study, the network architecture used is ResNet-50 as the network that has
been trained. The ResNet-50 architecture is a Residual Network consisting of 50 layers.
The layers at the beginning and the middle are called feature extraction layer. These
layers process a range of features from simple ones such as brightness and edges to
uniquely complex ones such as color and shape. The learning outcomes of the feature
extraction layer in the source environment (the environment where the network
originates) can be transferred to the feature extraction layers in the target environment
(the new environment in this study). Feature extraction learning in the target environment
is used to extract the bacterial image data used in the training phase.
B. Transfer learning
In this study, transfer learning method is used for network learning that utilizes
the classification layers. The classification layers are known as the last 3 layers, namely
fully connected, softmax, and output layers. The original classification layers are replaced
with new classification layers that are suitable for the new classification task. These new
layers also include the number of new classes and the learning speed set in the new
network.
The classification layers on the original network are configured for the number
of classes in the previous training. These layers in the network architecture contains
information that is processed by the network into class probabilities, loss values, and
predicted labels. To classify a new image, these layers are replaced with new layers and
adjusted to the new dataset. The fully connected layer is replaced with a new layer that
has the same number of outputs as the number of classes. To make learning process in the
new layer faster compared to the previous layer, the learning speed factor is increased.
The classification layers determine the network output class. The classification layers are
replaced with the new ones without the class label. During training, the network
automatically determines the output class of that layers.
In the input of convolutional layer, the network requires an input image of a
certain size, but the input images have different sizes. The image size is adjusted
automatically such that it can be used for the training network. Furthermore, data
augmentation operations is carried out on the training dataset by applying random affine
geometric transformations, namely horizontal reflection technique, and horizontal and
vertical translation technique. Data augmentation helps prevent the network from
overfitting and remembers precise details from training images.
In the training process, the learning speed parameters are determined for transfer
learning by maintaining the features in the initial network layer that have been trained
previously (the weight of the transferred layer). The learning rate in the transferred layer
needs to be decreased, so the initial learning speed is set to a small value. The learning
rate in the new classification layer needs to be accelerated, so the learning speed factor in
the previous step has been increased. The combination of these learning speed setting
results in fast learning only in the new layer and slower learning in the other layers.
While doing transfer learning, there is no need for training with the same number
of epochs as the previous number. An epoch is a complete training cycle across the entire
training dataset. In this training, the mini batch file size and validation data are also
determined. Network validation is carried out every iteration with a predetermined
frequency during training. Furthermore, training is carried out on the network consisting
Eduvest – Journal of Universal Studies
Volume 2 Number 2, February 2022
301 http://eduvest.greenvest.co.id
of the transferred layer and the new layer using the GPU. Then the image classification is
undertaken using a network that has gone through the training.
C. Optimization of the network training
In this study we apply optimization in the training process of Transfer Learning.
The Gradient Descent (GD) optimization method is applied with learning rate value of
0.0001. GD will update weights after all training data has been processed, or every 1
epoch (iteration). Because the learning rate is small, to avoid the long process to reach the
optimum state, the Stochastic Gradient Descent (SGD) solution is implemented. SGD
updates the weight without waiting for an epoch to finish. SGD uses a concept similar to
batching by dividing the training data into several batches. The weight will be updated in
each batch after processing. To further improve the performance, a momentum weight of
β >0 is applied so that the optimization algorithm used in this study is Stochastic Gradient
Descent with Momentum (SGDM).
D. Dataset and experiment
The dataset for training is bacteria images which is publicly available at
http://misztal.edu.pl/software/databases/dibas/. In this study the dataset used consists of
five types of bacteria that commonly cause infection in humans and one type of bacteria
that causes infection in animal. The type of bacteria that causes infection in animal is also
used as dataset to prove that the proposed method in this study is also able to classify
them among other bacteria that cause infections in humans. The training used 125 digital
bacteria images consisting of 6 classes, namely 23 Clostridium perfringens, 20
Escherichia Coli, 22 Listeria Monocytogenes, 20 Staphylococcus epidermidis, 20
Staphylococcus Saprophiticus and 20 Streptococcus Agalactiae. Clostridium perfringens
bacteria are the type of bacteria of the most common causes of foodborne illness (food
poisoning). Escherichia coli (E. coli) bacteria are bacteria that live in the human intestine
to maintain a healthy digestive system. Listeria monocytogenes is the species of
pathogenic bacteria that can cause serious and fatal infections in infants, children, the sick
and the elderly, and people with weakened immune systems. Staphylococcus epidermidis
is a species of bacteria from the genus Staphylococcus which is known to cause
opportunistic infections (attacking individuals with weak immune systems).
Staphylococcus Saprophiticus can cause urinary tract infections in young women.
Streptococcus agalactiae is a species of bacteria that causes streptococcosis in Nile tilapia
(Oreochromis niloticus).
With 125 images in dataset for training, the training process can take place
faster than using more images in dataset. With this number of dataset, fine-tuning the new
network can still work better, because the new network has learned the new features as
the result of previous training. This is an advantage of the transfer learning method used
in this study.
The size of each digital image is pixels. The experimental setup is as follows: the
dataset is divided into two parts, 70% for training and 30% for testing. The size of all
images is 2048x1532 pixels. For the purposes of this training where image variations are
required, dataset augmentation is also implemented. Figure 2 shows examples of
Clostridium perfringens, Escherichia Coli, Listeria Monocytogenes, Staphylococcus
epidermidis, Staphylococcus Saprophiticus and Streptococcus Agalactiae images
respectively.
Daniel Martomanggolo Wonohadidjojo
Classification of Bacterial Images Using Transfer Learning, Optimized Training and
Resnet-50 302
(1)
(2)
(3)
(4)
(5)
(6)
Figure 2. Examples of Bacteria Digital Images: (1) Clostridium perfringens (2)
Escherichia coli (3) Listeria monocytogenes (4) Staphylococcus epidermidis (5)
Staphylococcus saprophyticus (6) Streptococcus agalactiae
The training parameters used for all networks are as follows: learning rate 1e-4,
minibatch size 10, max epoch 30, validation frequency 3.
The network architecture is implemented with MATLAB on an Intel(R)
Core(TM) i7-8565U CPU @ 1.80GHz 1.99 GHz and 16.0 GB RAM with 1 GB NVIDIA
graphics processing unit (GPU).
RESULT AND DISCUSSION
In the experiments that have been carried out, all network architectures have
succeeded in classifying bacteria images using transfer learning. An example of bacteria
classification results is shown in Figure 3.
Eduvest – Journal of Universal Studies
Volume 2 Number 2, February 2022
303 http://eduvest.greenvest.co.id
A. Confusion matrix
The classification performance for each network architecture is presented in the
form of a confusion matrix as shown in Figure 3. To evaluate the results of the
classification, four performance metrics are used, namely Accuracy, Precision, Recall and
Fmeasure. The results of the performance metrics measurements are presented in Table 1.
Figure 3. Confusion Matrix of the Proposed Method.
Table 1. Performance Metrics of the Proposed Method
Accuracy
Precision
Recall
Fscore
97.37%
0.9761
0.9722
0.9742
Values on the main diagonal of CM represent all correctly classified examples.
The row below each CM shows the level of accuracy achieved for each predicted class
and the class prediction error. The CM shows that almost all images are classified
correctly, only 1 image classified incorrectly. This shows that the proposed method is
successful in the classification task.
B. Evaluation using Performance metrics
Table 1 shows performance metrics of the proposed method measured using four
metrics: Accuracy, Precision, Recall and Fscore. All the metrics show consistent values
that verify the result of Confusion Matrix. This shows that the proposed method is
successful in classifying the bacteria images.
Daniel Martomanggolo Wonohadidjojo
Classification of Bacterial Images Using Transfer Learning, Optimized Training and
Resnet-50 304
CONCLUSION
In this study, a method to classify bacterial images using transfer learning has
been proposed. In this method, transfer learning using ResNet-50 network architecture
that has been trained beforehand is applied to classify the images. In the new environment
the training process is optimized using SGDM algorithm. To increase the number and
variation of dataset and to avoid overfitting problem, data augmentation is implemented.
To evaluate the performance of the network architecture, CM and four performance
metrics namely Accuracy, Precision, Recall and Fmeasure are used.
The results of this study show that almost all the images are classified correctly,
with only one image classified incorrectly. The performance metrics show consistent
result where accuracy is 97.37%, precision is 0.9761, recall is 0.9722 and Fscore is
0,9742. These results show that the proposed method is successful in classifying bacteria
images. They suggest that the proposed method offers a new method to be used in other
target environment to classify bacteria for medical or other same kind of purposes.
For future research direction, it is recommended to increase the values of
performance metrics by applying other optimization technique or algorithm.
REFERENCES
Bottou, L., Curtis, F. E., & Nocedal, J. (2018). Optimization methods for large-scale
machine learning. SIAM Review, 60(2), 223–311.
https://doi.org/10.1137/16M1080173
Elnashar, H., Abd, I., & Azim, E. (2021). Deep Learning : Protein Cells Classifications
using Resnet-50 Model. 10(06), 849–855.
Handoyo, S., & Mulyandari, E. (2021). Analisis Imbangan Air pada Daerah Irigasi Jetu
Kabupaten Karanganyar. Syntax Literate; Jurnal Ilmiah Indonesia, 6(8), 4093–
4106.
Huang, L., & Wu, T. (2018). Novel neural network application for bacterial colony
classification. Theoretical Biology and Medical Modelling, 15(1), 1–16.
https://doi.org/10.1186/s12976-018-0093-x
Lundervold, A. S., & Lundervold, A. (2019). An overview of deep learning in medical
imaging focusing on MRI. Zeitschrift Fur Medizinische Physik, 29(2), 102–127.
https://doi.org/10.1016/j.zemedi.2018.11.002
Mohsen, H., El-Dahshan, E.-S. A., El-Horbaty, E.-S. M., & Salem, A.-B. M. (2018).
Classification using deep learning neural networks for brain tumors. Future
Computing and Informatics Journal, 3(1), 68–71.
https://doi.org/10.1016/j.fcij.2017.12.001
Sai Bharadwaj Reddy, A., & Sujitha Juliet, D. (2019). Transfer learning with RESNET-
50 for malaria cell-image classification. Proceedings of the 2019 IEEE International
Conference on Communication and Signal Processing, ICCSP 2019, 945–949.
https://doi.org/10.1109/ICCSP.2019.8697909
Sarvamangala, D. R., & Kulkarni, R. V. (2021). Convolutional neural networks in
medical image understanding: a survey. Evolutionary Intelligence, (0123456789).
https://doi.org/10.1007/s12065-020-00540-3
Talo, M. (2019). An Automated Deep Learning Approach for Bacterial Image
Classification. 1–5.
Treebupachatsakul, T., & Poomrittigul, S. (2019). Bacteria Classification using Image
Eduvest – Journal of Universal Studies
Volume 2 Number 2, February 2022
305 http://eduvest.greenvest.co.id
Processing and Deep learning. 34th International Technical Conference on
Circuits/Systems, Computers and Communications, ITC-CSCC 2019, 2–4.
https://doi.org/10.1109/ITC-CSCC.2019.8793320
Zieliński, B., Plichta, A., Misztal, K., Spurek, P., Brzychczy-Włoch, M., & Ochońska, D.
(2017). Deep learning approach to bacterial colony classification. PLoS ONE, 12(9).
https://doi.org/10.1371/journal.pone.0184554
