Create a Bounding Box Project#

When your project depends on bounding boxes, the uploaded dataset needs to contain the required bounding box information alongside each image as part of a single datapoint.

First we will create a dataset, and later on, a dummy model that returns bounding box information.

Create dataset#

A convenient approach for creating a bounding box dataset is by ensuring that the local format is in COCO-compatible format.

import efemarai as ef

# Create a project
project = ef.Session().create_project(
    name="Example Bounding Box Project (COCO)",
    description="Example project using the COCO dataset format.",

dataset = project.create_dataset(
    name="Bounding Box dataset",

If your dataset is remote or part of an existing database with custom formats, you can easily upload it to the system by (1) iterating over the dataset and (2) creating datapoints containing the images and required targets.

In add the following script to perform the above steps.

import numpy as np
import efemarai as ef

def upload_dataset():
    images = [
        np.random.randint(0, 256, size=(480, 640, 3), dtype=np.uint8) for _ in range(10)

    project = ef.Session().create_project(
        name="Example Bounding Box Project",
        description="Example project with custom datapoint upload.",

    dataset = project.create_dataset(
        name="Bounding Box dataset",

    class_count = 3
    for class_id in range(class_count):
        dataset.add_annotation_class(id=class_id, name=f"class_{class_id}_name")

    for image_data in images:
        # Create the inputs to the model
        # image = ef.Image(file_path="path/to/data") # if image is stored as a file
        image = ef.Image(data=image_data)  # if we have direct access to the image

        # Let's create random boxes with random classes and store the boxes
        boxes = []
        for i in range(3):
            image_height, image_width = 480, 640
            box_height, box_width = np.random.randint(40, 200, size=2)
            box_y1, box_x1 = np.random.randint(0, image_height - box_height), np.random.randint(0, image_width - box_width)
            box_y2, box_x2 = box_y1 + box_height, box_x1 + box_width

            # Get the label of the box from the dataset.
            # You can do it from either the id or the name
            # label = dataset.get_annotation_class(name=f"class_{np.random.randint(0, class_count)}_name")
            label = dataset.get_annotation_class(id=np.random.randint(0, class_count))

            bbox = ef.BoundingBox(
                xyxy=[box_x1, box_y1, box_x2, box_y2],  # top-left, bottom-right of the box
                label=label,  # the class label of the box
                ref_field=image,  # the image that this bounding box corresponds to (can also be a list of images)
                instance_id=i,  # Optional to say that this is a different instance in the same image

        # A sample, or a datapoint, contains reference to the dataset,
        # the inputs and target outputs
        datapoint = ef.Datapoint(
            inputs={"image": image},  # inputs and targets can be dicts or lists

        # Upload the datapoint

    # Let us know that these were all the datapoints for this dataset, so we
    # can calculate additional metadata.

if __name__ == "__main__":

Run python and wait for it to be completed.

After wrapping up any processing, you can confirm the status in the UI and explore the inputs and annotations.

Create a model#

A model that works with bounding boxes dataset will need to return a list of ef.BoundingBox objects that will be matched to the ones stored in the dataset. In a file save the following code:

import efemarai as ef
import numpy as np

class DummyModel:
    """A DummyModel returning a random bbox"""

    def __init__(self, device):
        self.device = device # Move model to device

    def __call__(self, image):
        return {
            "class_id": np.random.randint(0, 3),
            "bbox": [100, 150, 250, 350],
            "score": np.random.random(),

def predict_images(datapoints, model, device):
    outputs = []
    for datapoint in datapoints:
        image = datapoint.get_input("image") # This corresponds to the key from the datapoint input creation dict

        image_post_process = / 255 - 0.5 # perform any pre-processing

        output = model(image_post_process)

        # Here again the label can be referenced by name or class
        # label = ef.AnnotationClass(name=output["class_name"])
        label = ef.AnnotationClass(id=output["class_id"])

                    confidence=output["score"], # Confidence of detection
                    ref_field=image, # Say which image this output refers to
                    label=label,     # And what label it has
    return outputs

def load_model(device):
    model = DummyModel(device)
    return model

def test():
    # Load the model
    device = "cpu"
    model = load_model(device=device)

    image_RGB = np.random.randint(0, 256, size=(480, 640, 3), dtype=np.uint8)

    datapoint = ef.Datapoint(
            "image": ef.Image(data=image_RGB),

    # Pass a list of datapoints to the predictor function
    output = predict_images([datapoint], model, device)

    assert isinstance(output, list)
    assert isinstance(output[0][0], ef.BoundingBox)

if __name__ == "__main__":

If you run it with python you’ll be able to confirm that the output of the model is a list of detections per input datapoint.

efemarai.yaml file#

To run the model, you need to have defined the loading and inference capabilities in the efemarai.yaml file.

Here’s the one corresponding to the dummy model.

  name: "Example Bounding Box Project"

  - name: Dummy Model
    description: This is a dummy model to show consuming inputs and outputs

      image: python:3.10-slim-buster
      device: "gpu"
        max_size: 10
        entrypoint: dummy_model:load_model
          - name: device
            value: ${model.runtime.device}
          name: model

        entrypoint: dummy_model:predict_images
          - name: datapoints
            value: ${datapoints}
          - name: model
            value: ${model.runtime.load.output.model}
          - name: device
            value: ${model.runtime.device}
          name: predictions
            - bbox

Register the model#

To register the model, use the CLI to upload it by going into the root of the file directory, next to the efemarai.yaml.

ef model create .

Now you should be able to see the model uploaded and active with this project.