CLI Commands

The LEIP command line interface allows you to easily perform common operations without having to interact with the python library directly.

leip

Latent AI develops core technologies and platform tools to enable efficient, adaptive AI optimized for compute, energy, and memory with seamless integration to existing AI/ML infrastructure and frameworks. The Latent AI (LEIP) software development kit enables developer and data scientist access to these tools

leip [OPTIONS] COMMAND [ARGS]...

analyze

Analyzes a model and prints out/saves its layer names.

leip analyze [OPTIONS]

Options

--loglevel <loglevel>

Log output level

Default

WARNING

Options

DEBUG | INFO | WARNING | ERROR | CRITICAL

--input_path <input_path>

Required The directory or file path to the model

--output_path <output_path>

The root output directory path for the saved model and/or summaries

Default

./analyze_output

--include_layers <LIST>

The comma separated partial or full name of layers that you DO want to include in analysis

--exclude_layers <LIST>

The comma separated partial or full name of layers that you DON’T want to include in analysis

--config <config>

Read configuration from FILE.

--tags <LIST>

User defined tags for LEIP Enterprise events.

Default

compile

Compiles the model to an executable for a particular target.

leip compile [OPTIONS]

Options

--loglevel <loglevel>

Log output level

Default

WARNING

Options

DEBUG | INFO | WARNING | ERROR | CRITICAL

--input_path <input_path>

Required The directory or file path to the model

--output_path <output_path>

The root output directory path for the saved model and/or summaries

Default

./compile_output

--input_names <LIST>

The comma-separated names of the input layers of the model

--output_names <LIST>

The comma-separated names of the output layers of the model

--input_shapes <LISTS>

The shapes of the input layers of the model, hyphen-separated for each input, comma separated for each dimension. eg “1, 299, 299, 3 - 1, 400, 400, 3”

--remove_nodes <LIST>

Comma-separated list of nodes to remove from the model before processing

--layout <NCHW|NHWC>

The desired channel layout (For CPU targets only) [default: NCHW]

Options

Layout.NCHW | Layout.NHWC

--target <target>

The target device to compile the model to [default: llvm]

--target_host <target_host>

Host compilation target, if target is cuda [default: llvm]

--crc_check <BOOLEAN>

Specify this argument if you want to save a crc check value when storing the parameters [default: False]

--force_int8 <BOOLEAN>

Enforce the storage of parameters as int8 in the compiled output [default: False]

--legacy_artifacts <BOOLEAN>

Generate three separate artifacts (lib, graph, params) [default: False]

--optimization <optimization>

An optimization configuration. You may use this argument more than once to specify all desired optimizations. Current supported optimization formats: [‘category:kernel,level:<1-4>’, ‘category:cuda,enabled:true|false’, ‘category:graph,iterations:<0-30000>’]. Cuda optimization is enabled by default when target is cuda too, but may be overridden [default: [‘category:kernel,level:3’, ‘category:cuda,enabled:False’]]

--config <config>

Read configuration from FILE.

--tags <LIST>

User defined tags for LEIP Enterprise events.

Default

compress

Quantizes a pre-trained model.

If –data_type is float32, only weights are quantized, and the output model remains in the same format with dequantized weights. If –data_type is an integer type, both weights and operations are quantized, and the output model is either TFLite or (quantized) TorchScript.

leip compress [OPTIONS]

Options

--loglevel <loglevel>

Log output level

Default

WARNING

Options

DEBUG | INFO | WARNING | ERROR | CRITICAL

--input_path <input_path>

Required The directory or file path to the model

--output_path <output_path>

The root output directory path for the saved model and/or summaries

Default

./compress_output

--input_names <LIST>

The comma-separated names of the input layers of the model

--output_names <LIST>

The comma-separated names of the output layers of the model

--input_shapes <LISTS>

The shapes of the input layers of the model, hyphen-separated for each input, comma separated for each dimension. eg “1, 299, 299, 3 - 1, 400, 400, 3”

--preprocessor <preprocessor>

 The callback method used for preprocessing input data when running inference. It has three possible forms: 1) A name from [bgrtorgb|bgrtorgb2|bgrtorgb3|bgrtorgbcaffe |imagenet|imagenet_caffe|imagenet_torch_nchw |mnist|mnist_int|rgbtogray|rgbtogray_int8 |rgbtogray_symm|float32|uint8|symm|norm] 2) A python function as ‘package.module.func’ 3) A python function as ‘path/to/module.py::func’

--preprocessor_config_path <preprocessor_config_path>

The preprocessor configuration JSON, if any

--quantizer <asymmetric|symmetric|symmetricpc|power_of_two>

Which quantizer to use [default: asymmetric]

Options

QuantizerType.ASYMMETRIC | QuantizerType.SYMMETRIC | QuantizerType.SYMMETRICPC | QuantizerType.POWER_OF_TWO

--bits <bits>

The number of bits to quantize to [default: 8]

--include_layers <LIST>

The comma separated partial or full name of layers that you DO want to include for compression. eg: “foo,bar” will include layers with names like “foo1” or “my_bar”. The include_layers are applied before the exclude_layers. By default all layers are included. When used, ONLY layer names matched by include_layers will be compressed.

--exclude_layers <LIST>

The comma separated partial or full name of layers that you DON’T want to include for compression. eg: “foo,bar” will exclude layers with names like “foo1” or “my_bar”. The include_layers are applied before the exclude_layers. By default all layers are included.

--data_type <int8|uint8|float32>

The data type to use when quantizing and casting during compression [default: float32]

Options

DataType.INT8 | DataType.UINT8 | DataType.FLOAT32

--optimization <LIST from [tensor_splitting|bias_correction>

Which optimization passes to apply to the conversion [default: ]

--rep_dataset <rep_dataset>

The path to the file used as representative dataset input, used during calibration. This file should contain a newline separated list of path names to the input instances

--tflite_flags <tflite_flags>

Path to a JSON file with flags to be passed to the TFLite builder

--config <config>

Read configuration from FILE.

--tags <LIST>

User defined tags for LEIP Enterprise events.

Default

config

Generates a LEIP config.json file to use as a baseline config template.

leip config [OPTIONS]

Options

--output_path <output_path>

Where to save generated config file

Default

config.json

diff

Prints out a textual diff of 2 models

leip diff [OPTIONS]

Options

--loglevel <loglevel>

Log output level

Default

WARNING

Options

DEBUG | INFO | WARNING | ERROR | CRITICAL

--left_path <left_path>

Required The path to the first model to compare

--right_path <right_path>

Required The path to the second model to compare

--names <BOOLEAN>

Whether to include op names, names of input to each op [default: False]

--config <config>

Read configuration from FILE.

enterprise

Configure LEIP Enterprise.

leip enterprise [OPTIONS] COMMAND [ARGS]...

init

Initialize LEIP Enterprise Connection (host, user, project).

leip enterprise init [OPTIONS]

Options

--loglevel <loglevel>

Log output level

Default

WARNING

Options

DEBUG | INFO | WARNING | ERROR | CRITICAL

--host <host>

LEIP Enterprise host. Default should work for local docker deployment. [default: http://backend:8080]

--username <username>

Username in LEIP Enterprise

--password <password>

Password in LEIP Enterprise

--project_id <project_id>

Project UUID

--tags <LIST>

Predefined tags for events

Default

login

Login to LEIP Enterprise.

leip enterprise login [OPTIONS]

Options

--loglevel <loglevel>

Log output level

Default

WARNING

Options

DEBUG | INFO | WARNING | ERROR | CRITICAL

--username <username>

Username in LEIP Enterprise

--password <password>

Password in LEIP Enterprise

set_host

Set host of LEIP Enterprise.

leip enterprise set_host [OPTIONS]

Options

--loglevel <loglevel>

Log output level

Default

WARNING

Options

DEBUG | INFO | WARNING | ERROR | CRITICAL

--host <host>

LEIP Enterprise host. Default should work for local docker deployment. [default: http://backend:8080]

set_project

Set project for LEIP experiments.

leip enterprise set_project [OPTIONS]

Options

--loglevel <loglevel>

Log output level

Default

WARNING

Options

DEBUG | INFO | WARNING | ERROR | CRITICAL

--project_id <project_id>

Project UUID

set_tags

Set predefined tags for events.

leip enterprise set_tags [OPTIONS]

Options

--loglevel <loglevel>

Log output level

Default

WARNING

Options

DEBUG | INFO | WARNING | ERROR | CRITICAL

--tags <LIST>

Predefined tags for events

Default

evaluate

Performs inference of test data on the model and collects accuracy information.

leip evaluate [OPTIONS]

Options

--loglevel <loglevel>

Log output level

Default

WARNING

Options

DEBUG | INFO | WARNING | ERROR | CRITICAL

--input_path <input_path>

Required The directory or file path to the model

--output_path <output_path>

The output directory path for the evaluation summaries

Default

./evaluate_output

--input_names <LIST>

The comma-separated names of the input layers of the model

--output_names <LIST>

The comma-separated names of the output layers of the model

--input_shapes <LISTS>

The shapes of the input layers of the model, hyphen-separated for each input, comma separated for each dimension. eg “1, 299, 299, 3 - 1, 400, 400, 3”

--preprocessor <preprocessor>

 The callback method used for preprocessing input data when running inference. It has three possible forms: 1) A name from [bgrtorgb|bgrtorgb2|bgrtorgb3|bgrtorgbcaffe |imagenet|imagenet_caffe|imagenet_torch_nchw |mnist|mnist_int|rgbtogray|rgbtogray_int8 |rgbtogray_symm|float32|uint8|symm|norm] 2) A python function as ‘package.module.func’ 3) A python function as ‘path/to/module.py::func’

--preprocessor_config_path <preprocessor_config_path>

The preprocessor configuration JSON, if any

--postprocessor <postprocessor>

 The callback method used for postprocessing output data after running inference. It has three possible forms: 1) A name from [top1|top5] 2) A python function as ‘package.module.func’ 3) A python function as ‘path/to/module.py::func’

--output_format <classifier|yolov5|ssd>

The output format/architecture corresponding to the model [default: classifier]

Options

OutputFormat.CLASSIFIER | OutputFormat.YOLOV5 | OutputFormat.SSD

--inference_context <cpu|cuda>

Context under which to run inference [default: cpu]

Options

InferenceContext.CPU | InferenceContext.CUDA

--dataset_type <leip|detection|coco|voc>

The dataset type corresponding to the model [default: leip]

Options

DatasetType.LEIP | DatasetType.DETECTION | DatasetType.COCO | DatasetType.VOC

--dataset <cifar10|cifar100|voc|custom>

The test data to use. The value ‘custom’ is used when it’s not a dataset that is part of the TensorFlow library

Options

PublicDataSet.CIFAR10 | PublicDataSet.CIFAR100 | PublicDataSet.VOC | PublicDataSet.CUSTOM

--host <host>

Host on which to deploy model [default: localhost]

--port <port>

Port to access at host [default: 50051]

--test_path <test_path>

Required The path to the file containing a list of test examples and their output classification

--batch_size <batch_size>

The number of test images to load into one batch for inference [default: 1]

--config <config>

Read configuration from FILE.

--tags <LIST>

User defined tags for LEIP Enterprise events.

Default

health

Checks the health of the SDK environment

leip health [OPTIONS]

Options

--loglevel <loglevel>

Log output level

Default

WARNING

Options

DEBUG | INFO | WARNING | ERROR | CRITICAL

--config <config>

Read configuration from FILE.

optimize

Quantizes and compiles a pre-trained model.

leip optimize [OPTIONS]

Options

--loglevel <loglevel>

Log output level

Default

WARNING

Options

DEBUG | INFO | WARNING | ERROR | CRITICAL

--input_path <input_path>

Required The directory or file path to the model

--output_path <output_path>

The root output directory path for the saved model and/or summaries

Default

./optimize_output

--input_names <LIST>

The comma-separated names of the input layers of the model

--output_names <LIST>

The comma-separated names of the output layers of the model

--input_shapes <LISTS>

The shapes of the input layers of the model, hyphen-separated for each input, comma separated for each dimension. eg “1, 299, 299, 3 - 1, 400, 400, 3”

--remove_nodes <LIST>

Comma-separated list of nodes to remove from the model before processing

--preprocessor <preprocessor>

 The callback method used for preprocessing input data when running inference. It has three possible forms: 1) A name from [bgrtorgb|bgrtorgb2|bgrtorgb3|bgrtorgbcaffe |imagenet|imagenet_caffe|imagenet_torch_nchw |mnist|mnist_int|rgbtogray|rgbtogray_int8 |rgbtogray_symm|float32|uint8|symm|norm] 2) A python function as ‘package.module.func’ 3) A python function as ‘path/to/module.py::func’

--preprocessor_config_path <preprocessor_config_path>

The preprocessor configuration JSON, if any

--quantizer <asymmetric|symmetric|symmetricpc|power_of_two>

Which quantizer to use [default: asymmetric when cuda compile optimization is not set, otherwise symmetric]

Options

QuantizerType.ASYMMETRIC | QuantizerType.SYMMETRIC | QuantizerType.SYMMETRICPC | QuantizerType.POWER_OF_TWO

--bits <bits>

The number of bits to quantize to [default: 8]

--data_type <int8|uint8|float32>

The data type to use when quantizing and casting during compression [default: uint8 when cuda compile optimization is not set, otherwise int8]

Options

DataType.INT8 | DataType.UINT8 | DataType.FLOAT32

--rep_dataset <rep_dataset>

The path to the file used as representative dataset input, used during calibration. This file should contain a newline separated list of path names to the input instances

--compress_optimization <LIST from [tensor_splitting|bias_correction>

Which optimization passes to apply during the compression [default: ]

--use_legacy_quantizer <BOOLEAN>

Use LEIP 1.x quantizer method [default: False]

--quantize_input <BOOLEAN>

Whether the model input layers will be (u)int8 [default: True]

--quantize_output <BOOLEAN>

Whether the model output layers will be (u)int8 [default: True]

--wrap_quantizable_ops <BOOLEAN>

Whether sequences of unquantizable ops (like exp and stack) should have dequantize ops inserted before them and quantize ops inserted after them [default: True]

--layout <NCHW|NHWC>

The desired channel layout (For CPU targets only) [default: NCHW]

Options

Layout.NCHW | Layout.NHWC

--target <target>

The target device to compile the model to [default: llvm]

--target_host <target_host>

Host compilation target, if target is cuda [default: llvm]

--force_int8 <BOOLEAN>

Enforce the storage of parameters as int8 in the compiled output [default: False]

--legacy_artifacts <BOOLEAN>

Generate three separate artifacts (lib, graph, params) [default: False]

--compile_optimization <compile_optimization>

A compile optimization configuration. You may use this argument more than once to specify all desired optimizations. Current supported optimization formats: [‘category:kernel,level:<1-4>’, ‘category:cuda,enabled:true|false’, ‘category:graph,iterations:<0-30000>’]. Cuda optimization is enabled by default when target is cuda too, but may be overridden [default: [‘category:kernel,level:3’, ‘category:cuda,enabled:False’]]

--config <config>

Read configuration from FILE.

--tags <LIST>

User defined tags for LEIP Enterprise events.

Default

package

Generates a directory with all the required files needed to generate an executable on the target device.

leip package [OPTIONS]

Options

--loglevel <loglevel>

Log output level

Default

WARNING

Options

DEBUG | INFO | WARNING | ERROR | CRITICAL

--input_path <input_path>

Required The directory or file path to the model

--output_path <output_path>

The root output directory path for the compiling artifacts

Default

./package_output

--input_names <LIST>

The comma-separated names of the input layers of the model

--preprocessor <preprocessor>

 The callback method used for preprocessing input data when running inference. It has three possible forms: 1) A name from [bgrtorgb|bgrtorgb2|bgrtorgb3|bgrtorgbcaffe |imagenet|imagenet_caffe|imagenet_torch_nchw |mnist|mnist_int|rgbtogray|rgbtogray_int8 |rgbtogray_symm|float32|uint8|symm|norm] 2) A python function as ‘package.module.func’ 3) A python function as ‘path/to/module.py::func’

--postprocessor <postprocessor>

 The callback method used for postprocessing output data after running inference. It has three possible forms: 1) A name from [top1|top5] 2) A python function as ‘package.module.func’ 3) A python function as ‘path/to/module.py::func’

--metrics <LIST from [inferences_count|latency|most_common_class>

Metrics to include in runtime library

--format <python3.6|python3.8|python3.9|docker|cc>

Library’s output format

Options

PackageFormat.PYTHON36 | PackageFormat.PYTHON38 | PackageFormat.PYTHON39 | PackageFormat.DOCKER | PackageFormat.CC

--config <config>

Read configuration from FILE.

--tags <LIST>

User defined tags for LEIP Enterprise events.

Default

pipeline

Runs a pipeline of commands on a model from a configuration file.

leip pipeline [OPTIONS]

Options

--loglevel <loglevel>

Log output level

Default

WARNING

Options

DEBUG | INFO | WARNING | ERROR | CRITICAL

--input_path <input_path>

A directory or file path to be used as an input

--output_path <output_path>

The root output directory path for all task results

Default

./pipeline_output

--config_path <config_path>

Required Path to a YAML/JSON file describing the pipeline

--tags <LIST>

User defined tags for LEIP Enterprise events.

Default

run

Run an inference through the specified model.

leip run [OPTIONS]

Options

--loglevel <loglevel>

Log output level

Default

WARNING

Options

DEBUG | INFO | WARNING | ERROR | CRITICAL

--input_path <input_path>

Required The directory or file path to the model

--output_path <output_path>

The output directory path for the run results

Default

./run_output

--input_names <LIST>

The comma-separated names of the input layers of the model

--output_names <LIST>

The comma-separated names of the output layers of the model

--input_shapes <LISTS>

The shapes of the input layers of the model, hyphen-separated for each input, comma separated for each dimension. eg “1, 299, 299, 3 - 1, 400, 400, 3”

--preprocessor <preprocessor>

 The callback method used for preprocessing input data when running inference. It has three possible forms: 1) A name from [bgrtorgb|bgrtorgb2|bgrtorgb3|bgrtorgbcaffe |imagenet|imagenet_caffe|imagenet_torch_nchw |mnist|mnist_int|rgbtogray|rgbtogray_int8 |rgbtogray_symm|float32|uint8|symm|norm] 2) A python function as ‘package.module.func’ 3) A python function as ‘path/to/module.py::func’

--preprocessor_config_path <preprocessor_config_path>

The preprocessor configuration JSON, if any

--postprocessor <postprocessor>

 The callback method used for postprocessing output data after running inference. It has three possible forms: 1) A name from [top1|top5] 2) A python function as ‘package.module.func’ 3) A python function as ‘path/to/module.py::func’

--output_format <classifier|yolov5|ssd>

The output format/architecture corresponding to the model [default: classifier]

Options

OutputFormat.CLASSIFIER | OutputFormat.YOLOV5 | OutputFormat.SSD

--inference_context <cpu|cuda>

Context under which to run inference [default: cpu]

Options

InferenceContext.CPU | InferenceContext.CUDA

--test_path <test_path>

Required The path to the file to use as input to the inference

--class_names <class_names>

Required The path of the file specifying the class names

--items <items>

The number of top confidence classes to show [default: 5]

--config <config>

Read configuration from FILE.

--tags <LIST>

User defined tags for LEIP Enterprise events.

Default

train

Train a model using Quantization Guided Training. Note that the Keras .h5 format is currently supported.

leip train [OPTIONS]

Options

--loglevel <loglevel>

Log output level

Default

WARNING

Options

DEBUG | INFO | WARNING | ERROR | CRITICAL

--input_path <input_path>

Required The directory or file path to the model

--output_path <output_path>

Directory to store resulting trained model

--training_config_file <training_config_file>

Path to training config JSON file. the fields supported are: -dataset: please see /latentai/docs/leip_train.html#the-dataset-field -regularizer_attachment_scheme: please see /latentai/docs/leip_train.html#the-regularizer-attachment-scheme-field -batchsize: (int) batchsize used during training -epochs: (int) number of epochs used during training -epochs_pre_quantization: (int) the number of pre-quantization epochs to train on the model -bits: (int) number of bits to quantize to -batchnorm_folding: (bool) whether to fold batchnorm params into a layer’s weights -remove_bias_regularizers_from_all: whether bias regularizers should be removed from all layers -remove_all_regularizers_for_batch_normalization: whether any regularization should be done on batch norm layers -change_conv2d_kernel_regularizers_to_channelwise: whether to change kernel_regularizers to channelwise on Conv2d layers -change_depthwiseconv2d_depthwise_regularizers_to_channelwise: whether to change depthwise_regularizers to channelwise on DepthwiseConv2d layers -quantizer: quantization algorithm to use

--training_script_path <training_script_path>

destination location of the generated python training script

--invoke_training <BOOLEAN>

whether to automatically run the generated training script [default: True]

--step <config|train|list|dump_model_json|validate>

Which training step to perform. “list” will dump all layers, “dump_model_json” will dump the entire model as json, “config” will create a training config JSON file, “validate” will check that the config JSON file is valid [default: train]

Options

TrainSteps.CONFIG | TrainSteps.TRAIN | TrainSteps.LISTLAYERS | TrainSteps.DUMPJSON | TrainSteps.VALIDATE

--config <config>

Read configuration from FILE.

--tags <LIST>

User defined tags for LEIP Enterprise events.

Default

version

The LEIP sdk version.

leip version [OPTIONS]

Options

--loglevel <loglevel>

Log output level

Default

WARNING

Options

DEBUG | INFO | WARNING | ERROR | CRITICAL

zoo

Access models from the LEIP Zoo.

leip zoo [OPTIONS] COMMAND [ARGS]...

download

Download a specified model or dataset from the Latent AI Model Zoo

leip zoo download [OPTIONS]

Options

--models_index_json_path <models_index_json_path>

Path to a local json file that contains zoo metadata

--model_id <model_id>

Model id to download

--dataset_id <dataset_id>

Dataset id to download

--variant_id <variant_id>

Required Variant id to download

license

Show the license for the zoo

leip zoo license [OPTIONS]

list

List the available models in the zoo as an ASCII formatted table

leip zoo list [OPTIONS]

Options

--models_index_json_path <models_index_json_path>

Path to a local json file that contains zoo metadata