[imageAI]What is the step number displayed in the log messages during custom object detection training.

For most people using imageAI to implement the object detection application, they probably reference the tutorial "imageAI : Custom Detection Model Training" on the official website.

When people begin the training by the sample from the tutorial, the progress bar displayed with some numbers in the log message. Apart from the "Epoch", there are information about time and step but what is the "step" numbers mean?

The step numbers  are counted from 0 in each Epoch. For people who have basic machine learning knowledge, they could think of the batch numbers (all training instances divided by batch size) immediately. However, it seems something different from the batch number.

my case:

-data set: 301 images with 3 types labels

-batch size:4

-Epoch:12

I noticed the difference when I training a data set containing 301 images with batch size 4. I thought the number before the progress bar should be the total of the batch number "73" (301/4) and the current batch number.  The actual number displayed is"480". Well, it is totally different from my expected.

Fortunately, ImageAI is open-source. It allows every programmer to check the code. First, I found the class "Progbar" defined in the "generic_utils.py". The method "update( )" provided by the class generates the string of numbers and progress bar for the log messages displayed on the screen.

 

The total step number("480") in the message is according to the object variable "self.target" in the class "Progbar" and this value is assigned in the method "trainModel()" by the parameter "steps_per_epoch" in the class "DetectionModelTrainer". 

object "progbar": created in 

object variable "self.target" in "progbar" :  assigned by the value "steps_per_epoch" in 

Here's the thing, the result of "len(train_generator)*self.__train_times" is "480". Since "self.__train_times" is 8 in init, the "len(train_generator)" could be 60.  Literally, the value of the statement might be 8 times of something. Besides, the comments about the parameter "steps_per_epoch" is "It should typically be equal to the number of samples of your dataset divided by the batch size.". We could say the "len(train_generator)" is associated with the batch number.

I checked the definition of "trainModel()" method further. Finally, I got the answer. the "trainModel()" had separated the whole dataset (301 images in my case) into two sets by "_create_training_instances()". 80% of the images are categorized into the "train_ints" for training and the remaining 20% are used to evaluate the training result for each batch. 80% of 301 is 240. After dividing it by the batch size 4, we got the number "60". 

Finally, we could also check the  "train_model.fit_generator()" that used by "trainer.trainModel()" in tto veify how imageAI display the steps number.

training_generator.py

In conclusion

In imageAI, the steps number is the number of training batch x 8

-in my case: 60x8=480

The number of training batch is number of training instances(images) divided by batch size.

-in my case:240/4=60

The training data set  is 80% of the total  image as  the default value.

-in my case is "301x0.8=60"