LEIP Models

class models.Model(options: ModelOptions, framework: Framework, source_file: Optional[Union[str, PathLike]] = None)

Abstract class representing an ML model in a supported framework.

Parameters
  • options (ModelOptions) –

  • framework (Framework) –

  • source_file (Union[str, PathLike]) –

dict() dict

Gives a serializable dict version of the model. This includes:

options

The options used to load and configure the model.

source_file

The name of the source file from which it was loaded, if any.

source_file_md5_hash

An MD5 hash of the source file.

source_file_size

Source file size in bytes.

framework

The framework of the model.

inputs

A list of input objects with names, shapes and types.

outputs

A list of output objects with names, shapes and types.

weights_count

The amount of tensors in the model.

weights_size

The sum of sizes of all tensors in the model.

weights_data_type

The data type(s) of the tensors in the model.

Returns

Options and metrics as a dict.

Return type

dict

abstract infer(single_input: ndarray) Tuple[List[ndarray], float]

Runs inference from a numpy array.

Parameters

single_input (ndarray) – A numpy array that will be sent to the model. The array must be of the right type and shape.

Returns

  • list – A list of all the outputs of the model.

  • float – Number of seconds inference took.

Return type

Tuple[List[ndarray], float]

abstract property inputs: List[models.tensors.TensorSpec]

Determines the inputs to the model.

Returns

A list of input objects with names, shapes and types.

Return type

list

abstract property outputs: List[models.tensors.TensorSpec]

Determines the outputs of the model.

Returns

A list of output objects with names, shapes and types.

Return type

list

abstract property weights: List[models.tensors.Tensor]

Determines the tensors of the model.

Returns

A list of all the tensors in the model.

Return type

list

class models.ModelOptions(*, inference: InferenceOptions = InferenceOptions(context='cpu', host='localhost', port='50051'), input_names: List[str] = [], output_names: List[str] = [], input_shapes: List[Tuple[Optional[int], ...]] = [], remove_nodes: List[str] = [], dataset: DatasetOptions = DatasetOptions(public_dataset=None, type='leip'), custom_objects: dict = None, crc: int = None, metadata: dict = {}, preprocessor: str = None, postprocessor: str = None, preprocessor_config: dict = None, output_format: OutputFormat = OutputFormat.CLASSIFIER)

Options and metadata related to a model.

Parameters
  • inference (InferenceOptions) –

  • input_names (List[str]) –

  • output_names (List[str]) –

  • input_shapes (List[Tuple[Optional[int], ...]]) –

  • remove_nodes (List[str]) –

  • dataset (DatasetOptions) –

  • custom_objects (Optional[dict]) –

  • crc (Optional[int]) –

  • metadata (dict) –

  • preprocessor (Optional[str]) –

  • postprocessor (Optional[str]) –

  • preprocessor_config (Optional[dict]) –

  • output_format (OutputFormat) –

Return type

None

crc: Optional[int]

CRC of a TVM model binary before saving, used to verify data corruption.

custom_objects: Optional[dict]

Value passed to keras’ load_model as custom_objects. Ignored for other frameworks.

dataset: models.model_options.DatasetOptions

Options describing the kind of dataset that would be used to run inference.

inference: models.model_options.InferenceOptions

Options describing the context in which inference will be run.

input_names: List[str]

Optional input names, required for some framework formats, like TF-GraphProto and version 1 of TF-SavedModel.

input_shapes: List[Tuple[Optional[int], ...]]

Optional output shapes, required for some framework formats, like Pytorch.

metadata: dict

Metadata used for data analysis.

output_format: models.enums.OutputFormat

Format of output, used to determine how to score when evaluating the model.

output_names: List[str]

Optional output names, required for some framework formats, like TF-GraphProto and version 1 of TF-SavedModel.

postprocessor: Optional[str]

Name of a known or custom function used to optionally postprocess data after running inference.

preprocessor: Optional[str]

Name of a known or custom function used to optionally preprocess data before running inference.

preprocessor_config: Optional[dict]

Optional configuration passed to the preprocessor.

remove_nodes: List[str]

List of node names in a TF model that should be removed before running inference.

models.load_model(path: Union[str, PathLike], options: ModelOptions = ModelOptions(inference=InferenceOptions(context='cpu', host='localhost', port='50051'), input_names=[], output_names=[], input_shapes=[], remove_nodes=[], dataset=DatasetOptions(public_dataset=None, type='leip'), custom_objects=None, crc=None, metadata={}, preprocessor=None, postprocessor=None, preprocessor_config=None, output_format='classifier'), load_baseline_options_from_json: bool = True) Model

Load a supported model from disk.

Parameters
  • path (Union[str, PathLike]) – Directory or file path to the model files(s).

  • options (ModelOptions) – Options to configure the model. These may also come from a model_schema.json file in the same folder as the model files. Any options defined here override those in that file.

  • load_baseline_options_from_json (bool) – Indicates if model options should also be loaded from model_schema.json file.

Returns

Loaded model object.

Return type

Model

Raises

NotRecognizedModel – If a supported model was not found at the given path, or if the framework needed for it is not installed.

leip.load_compilable_model(path: Union[str, PathLike], options: ModelOptions = ModelOptions(inference=InferenceOptions(context='cpu', host='localhost', port='50051'), input_names=[], output_names=[], input_shapes=[], remove_nodes=[], dataset=DatasetOptions(public_dataset=None, type='leip'), custom_objects=None, crc=None, metadata={}, preprocessor=None, postprocessor=None, preprocessor_config=None, output_format='classifier'), load_baseline_options_from_json: bool = True) Union[CompilableModel, QuantizableModel]

Load a compilable model from disk.

Parameters
  • path (Union[str, PathLike]) – Directory or file path to the model files(s).

  • options (ModelOptions) – Options to configure the model. These may also come from a model_schema.json file in the same folder as the model files. Any options defined here override those in that file.

  • load_baseline_options_from_json (bool) – Indicates if model options should also be loaded from model_schema.json file.

Returns

Loaded model object.

Return type

CompilableModel

Raises

NotRecognizedModel – If a supported model was not found at the given path, or if the framework needed for it is not installed.

class leip.CompilableModel(options: ModelOptions, framework: Framework, source_file: Optional[Union[str, PathLike]] = None)

Abstract class representing a compilable ML model in a supported framework. This class adds additional methods on top of Model that are used when compiling or optimizing a model.

Parameters
  • options (ModelOptions) –

  • framework (Framework) –

  • source_file (Union[str, PathLike]) –