Can I utilize bounding box for image data?

I have an image dataset that has a bounding box for the objects being identified. Some images have multiple bounding boxes with different classifications.

Capture

Perceptilabs does not seem to handle/utilize csv files with bounding boxes. Can the bounding boxes be imported and used for the classification?

Regards,

Cecil

Hi @Kingseso,
Welcome to the forum!

We unfortunately don’t handle bounding boxes yet as you said, we are currently limited to image classification, segmentation and some regression tasks.
Object Detection is something we are looking at adding in sooner rather than later though to complete the Computer Vision trifecta.

Is there any specific Object Detection model/method you were interested in running with?

All the best,
Robert

1 Like

Hello Robert,

I ultimately used Yolov5 for object detection, but I do have a second question.

I was able to use PL for image recognition using some of the examples from the blog.

I am running into an issue of generating probabilities from the exported model. I used the following code in python:

model = keras.models.load_model(path_to_model)

im = Image.open(test_image_path + test_df[‘File Name’][i])
im = np.array(im, dtype=np.float32)
im= np.expand_dims(im, axis=0)

Makes a prediction

prediction1 = model.predict(im)

model.predict only provides the class. I would like the probability of the selected identified class. Maybe you have some recommendations.

Regards,

Cecil

Hello Robert,

Never mind about the probability. I realized I needed the confidence from my bounding box, so I was able to get that else where.

Keep up the good work. PL made model making quick and clean.

Cecil

2 Likes

Hi @Kingseso

If it’s no too much trouble, you would mind explaining how you get the confidence? The statistical bits of ML are - for me - the hardest to get to grips with, so I find new examples very helpful.

Thx :slight_smile:

Hi @JulianSMoore, I had the same issue I wanted to get the confidence of the predictions as well. I managed this by changing the output layer of the model.

The original contents of the model.summary() are:
Model: “model”
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input (InputLayer) [(None, 64, 64, 3)] 0
_________________________________________________________________
image_preprocessing_12 (Imag (None, 64, 64, 3) 0
_________________________________________________________________
training_model (TrainingMode ({‘target’: (None, 4)}, { 1049444
=================================================================
Total params: 1,049,444
Trainable params: 1,049,444
Non-trainable params: 0
_________________________________________________________________

import tensorflow as tf
from tensorflow.keras.models import Model
import cv2
import numpy as np

def prepare(img, img_size=64):
    img = cv2.resize(img, (img_size, img_size))
    img = img.reshape(-1, img_size, img_size, 3)
    return img

model = tf.keras.models.load_model("Model to Load")
model.summary() # Get original model with the postprocessing layer
model = Model(inputs=model.input, outputs=model.get_layer('training_model').output)
model.summary() # Get model without the postprocessing layer

img = cv2.imread('image.png')
img = prepare(img)
prediction = model.predict(img)
idx = np.argmax(prediction[0]['target'])
confidence = prediction[0]['target'][0][idx]

Now the model should refer to the “training_model” layer before the postprocessing layer and you should get your confidence values from the prediction.

Hope that this was of help, I hear the postprocessing step can be turned off in a future version so this move will no longer be necessary.

1 Like

Hi @Taeivas

If I understand that correctly - and thanks for taking the time to share! - you’re saying the confidence is a standard part of the prediction data structure. How did you find that? I guess it must be documented somewhere…?

Anyway - that’s very good to know!

BTW Is it really “confidence”, e.g. a 95% confidence band around some nominal value, or is it the probability of something? All the probabilities would add up to 1, but "confidences* are intervals that could add up to anything. As I recall, @Kingseso said “confidence” in place of “probability” so they’re both sort of involved, but I’m a bit confused :smiley:

No need to stress over that, but if you can clarify sometime that would be even nicer!

But, either way: good work! I’m sure just the “how” alone could be very useful to someone!

I am pretty sure we are referring to the same thing “probability” or “confidence” in this case, it gives a numeric value of which specific classes get a highest activation and run an argmax function on it to get the index of the highest “confidence” class in the predictions, those numbers should range between 0 and 1, but shouldn’t add up to more than 1 when all class “confidences”/“probabilities” are summed up.

With the knowledge I have and from what I understand from the postprocessing layer, it executes the argmax function and looks up which label matches the highest confidence class and outputs that value in it’s model’s end prediction.

The best source for the information would be tensorflow official API documentation, which was the source where I came to this “hack”, just removed the postprocessing layer.

I am by no means an expert, I just dabble in trying to learn as much as I can and apply it in practical applications to solve real-world problems.

Hi @Taeivas

Thanks for clarifying! I think I can help a bit here: in statistics etc. probability and confidence mean different things, even though deeper down they are both probabilities.

The softmax/softargmax produces a “probability distribution” i.e. a set of probabilities (e.g. for a list of classes) that add up to 1, meaning one of them MUST happen.

Confidence - hang on to your hat! - refers to the probability that some value is actually close to the “real” value, so you can think of it as “probability of probability” in this case :slight_smile:

For example, at the LHC the criterion for a “discovery” is 5 sigma - 5 standard deviations, which means that the probability the measurement is wrong is < 1 in 3.5 million and the confidence is ~99.9999%.

So one can have both a probability measure for a classification and a confidence in that probability.

If you know which is which you will get better answers if you need help creating or interpreting statistics.

(I recently used confidence intervals to help me calculate whether the disagreement between a regression and the true value could occur by chance - every experiment produces e.g. a measure of probability, but confidence tells you the probability the probability is “good” :laughing:)

Hope that helps!

Thanks @JulianSMoore for the clarification! I’ll look more into it when I have the time available, definitely sounds great to know the differences in more detail.

1 Like