Create Dual Image Regression Project#

This example shows a case where the model consumes two images and regresses a value, thus the uploaded dataset needs to contain the required dual images information alongside the regressed ground truth in a single datapoint.

First we will create a dataset, and later on, a dummy model that returns a regressed value.

Create dataset#

Let’s create the dataset with the custom format by iterating over the images in pairs and (2) creating datapoints containing the images and required value.

In upload_dataset.py add the following script to perform the above steps.

import os
import random

import efemarai as ef

def get_dataset():
    images = [
        "https://this-person-does-not-exist.com/img/avatar-11c3c1714641dccb5e0cdd221a4fa6aa.jpg",
        "https://this-person-does-not-exist.com/img/avatar-110c31e334d64b1598a07d19bb374fec.jpg",
        "https://this-person-does-not-exist.com/img/avatar-112eedb7d5026b58f4762607498137e0.jpg",
        "https://this-person-does-not-exist.com/img/avatar-11c85c80da827fd91f7b6a4ad4cd8c1d.jpg",
        "https://this-person-does-not-exist.com/img/avatar-1186b1e698b5d848305a606cddeadd10.jpg",
        "https://this-person-does-not-exist.com/img/avatar-116a7a1444014822a3715e009e882105.jpg",
        "https://this-person-does-not-exist.com/img/avatar-11035e8e377de99688e853971caccf2c.jpg",
        "https://this-person-does-not-exist.com/img/avatar-111dd492cd4c87c5ed2673c2bfcf38bc.jpg",
        "https://this-person-does-not-exist.com/img/avatar-1175bc9c09349a5275f3f2ff81821c8b.jpg",
        "https://this-person-does-not-exist.com/img/avatar-115b84f284677e28b0e464fcb49a0497.jpg",
    ]

    similarities = [0.0, 0.25, 0.5, 0.75, 1.0]

    images_a, images_b = [], []

    # Download images
    os.makedirs("images", exist_ok=True)
    for i, url in enumerate(images):
        path = os.path.basename(url)
        if not os.path.exists(path):
            os.system(f"wget -nv -O {path} {url}")

        if i % 2:
            images_a.append(path)
        else:
            images_b.append(path)

    return zip(images_a, images_b, similarities)


def upload_dataset():
    project = ef.Session().create_project(
        name="Dual Image Regression",
        description="Example project showing dual image regression.",
        exists_ok=True,
    )

    dataset = project.create_dataset(
        name="Dual image dataset",
        stage=ef.DatasetStage.Test,
        format=ef.DatasetFormat.Custom,
        exists_ok=True,
    )

    for a_url, b_url, similarity in get_dataset():
        # Create the inputs to the model
        image_a = ef.Image(file_path=os.path.basename(a_url))
        image_b = ef.Image(file_path=os.path.basename(b_url))

        # Create target outputs for the model
        similarity = ef.Value(value=similarity, ref_field=[image_a, image_b])

        # A sample, or a datapoint, contains reference to the dataset,
        # the inputs and target outputs
        datapoint = ef.Datapoint(
            dataset=dataset,
            inputs={"image_a": image_a, "image_b": image_b},
            targets={
                "similarity_score": similarity,
            },
        )

        # Upload the datapoint
        datapoint.upload()

    # Let us know that these were all the datapoints for this dataset, so we
    # can calculate additional metadata.
    dataset.finalize()


if __name__ == "__main__":
    upload_dataset()

Run python upload_dataset.py and wait for it to be completed.

After wrapping up any processing, you can confirm the status in the UI and explore the inputs and annotations.

Create a model#

A model that works with value dataset will need to return a list of ef.Value objects that will be matched to the ones stored in the dataset. In a file dummy_model.py save the following code:

import efemarai as ef
import numpy as np


class DummyModel:
    """A DummyModel returning random value"""

    def __init__(self, device):
        # Move model to device
        self.device = device

    def __call__(self, image_a, image_b):
        return {
            "similarity_score": np.random.random(),
        }


def predict_images(datapoints, model, device):
    outputs = []
    for datapoint in datapoints:
        image_a = datapoint.get_input("image_a")
        image_b = datapoint.get_input("image_b")

        output = model(image_a.data, image_b.data)
        outputs.append(
            [
                ef.Value(
                    value=output["similarity_score"], ref_field=[image_a, image_b]
                ),
            ]
        )
    return outputs


def load_model(device):
    model = DummyModel(device)
    return model


def test():
    device = "cpu"
    model = load_model(device=device)

    image_RGB = np.random.randint(0, 256, size=(480, 640, 3), dtype=np.uint8)

    ef_datapoint = ef.Datapoint(
        dataset=None,
        inputs={
            "image_a": ef.Image(data=image_RGB),
            "image_b": ef.Image(data=image_RGB),
        },
    )

    output = predict_images([ef_datapoint], model, device)
    print(output)

    assert isinstance(output, list)
    assert isinstance(output[0][0], ef.Value)


if __name__ == "__main__":
    test()

If you run it with python dummy_model.py you’ll be able to confirm that the output of the model is a list of detections per input datapoint.

`efemarai.yaml` file#

To run the model, you need to have defined the loading and inference capabilities in the efemarai.yaml file.

Here’s the one corresponding to the dummy model.

project:
  name: "Dual Image Regression"
  description: "Example project showing dual image regression."

models:
  - name: Dummy Model
    description: This is a dummy model to show consuming inputs and outputs

    runtime:
      image: python:3.10-slim-buster
      device: "gpu"
      batch:
        max_size: 10
      load:
        entrypoint: dummy_model:load_model
        inputs:
          - name: device
            value: ${model.runtime.device}
        output:
          name: model

      predict:
        entrypoint: dummy_model:predict_images
        inputs:
          - name: datapoints
            value: ${datapoints}
          - name: model
            value: ${model.runtime.load.output.model}
          - name: device
            value: ${model.runtime.device}
        output:
          name: predictions
          keys:
            - similarityScore

Register the model#

To register the model, use the CLI to upload it by going into the root of the file directory, next to the efemarai.yaml.

ef model create .

Now you should be able to see the model uploaded and active with this project.

Create Dual Image Regression Project#

Create dataset#

Create a model#

efemarai.yaml file#

Register the model#

`efemarai.yaml` file#