3. Create Model

An integral part of improving your ML system is iterating on your models. You can define multiple different models (by creating them with ef model create), as well as versions of the same model (by pushing a new version with ef model push), such that you can track key performance indicators and see the direction of development.

Models definition

We need to define several parts in order to integrate the model with the Efemarai system.

Name and files

We should extend the previous project file with the following data.

models:
  - name: Our First Model
    files:
      - name: params
        url: weights/model.pt

This specifies that we would like to call our model Our First Model and has a single file associated with it that we can further internally refer to as params. It points to the file in the filesystem under the url - weights/model.pt.

To upload multiple files, or ones in remote repositories one can extend it as:

    files:
      - name: performance-graph
        url: weights/loss.png
      - name: performance-log
        url: weights/loss-log.json
      - name: remote-params
        url: https://some-remote-bucket.example.com
        upload: False

If you upload images, json, or any other configuration files, you’ll have the opportunity to use them when loading the model, inspect them and see exactly what is associated with this particular model in the UI.

Sometimes weights can be quite large and stored in a separate repository/bucket. To accommodate this workflow, you can specify a direct path to them and you can avoid uploading the weights at the current time with upload: False.

Runtime

An important part of connecting your model to the Efemarai platform is specifying two python functions under the runtime definitions - load - for loading the model; and predict - running a forward pass on it.

    runtime:
      image: pytorch/pytorch:1.9.1-cuda11.1-cudnn8-runtime
      load:
        entrypoint: inference:get_model
        inputs:
          - name: params_url
            value: ${model.files.params.url}
        output:
          name: model
      predict:
        entrypoint: inference:predict
        inputs:
          - name: model
            value: ${model.runtime.load.output.model}
          - name: datapoints
            value: ${datapoints}
        output:
          name: predictions
          keys:
            - boxes
            - classes
            - scores

Image

We rely on Docker containerization to create the right working environment for your model. You can specify a base image under the image directive with any publicly accessible image or Dockefile.

After the image has been pulled, we install any dependencies in requirements.txt, as well as some additional artifacts to interface the container with our system.

load function

The function definition logic is shared between the load, predict functions and has the following structure:

      function-name:
        entrypoint: module.entry-function
        inputs:
          - name: param_name
            value: "absolute" <or refered value> ${model.files.params.url}
        output:
          name: function-output

For the load call we can decompose

      load:
        entrypoint: inference:get_model
        inputs:
          - name: params_url
            value: ${model.files.params.url}
        output:
          name: model

To load the model, we would find inference.py in the root directory of the project, and call the get_model function with a single argument named params_url that will have the value, which we defined in the above step (weights/model.pt).

The output of the load function is a single object, named model.

predict function

      predict:
        entrypoint: inference:predict
        inputs:
          - name: model
            value: ${model.runtime.load.output.model}
          - name: datapoints
            value: ${datapoints}
        output:
          name: predictions

A key difference to note here is the use of ${model.runtime.load.output.model}, which is exactly what the load function returns.

A second element is the ${datapoints} variable, which is the list of datapoints passed to the function. Each datapoint can have different images, video, text associated with it and your model can prepare the input in any standard in order to consume and make a forward pass.

The method returns a single output predictions that contains the list of predictions for each datapoint (a list of lists).

Full example

Here is a full example defining a local model.

models:
  - name: YOLOR-P6-Local
    files:
      - name: params
        url: results/yolor_p6.pt
        upload: True
    runtime:
      image: pytorch/pytorch:1.9.1-cuda11.1-cudnn8-runtime
      load:
        entrypoint: inference:get_model
        inputs:
          - name: params_url
            value: ${model.files.params.url}
        output:
          name: model
      predict:
        entrypoint: inference:predict
        inputs:
          - name: model
            value: ${model.runtime.load.output.model}
          - name: datapoints
            value: ${datapoints}
        output:
          name: predictions
          keys:
            - boxes
            - classes
            - scores

Up until now, your efemarai.yaml should contain a project name name and the two datasets. Now add the model to your file. After you have added the above code snippet to your efemarai.yaml file, you should be able to add a model via our CLI. Note that the following commands should be executed in the directory of your local code such that it is correctly uploaded.

efemarai model create efemarai.yaml

or if you’re using the standard name as above, you can simply type

efemarai model create .

When ready, you will be able to see the loaded model in the Models page and in the CLI

ef model list

Next, let’s create an Operational Domain to test this model.