LEIP Optimize

leip.optimize(model: CompilableModel, output_path: Path, options: Optional[OptimizeOptions]) → CompressResults

Optimizes a pre-trained model by quantizing and compiling it.

Parameters

  • model (CompilableModel) – Model object.

  • output_path (Path) – Directory path for generated artifacts.

  • options (Optional[OptimizeOptions]) – Options that configure the model optimization.

Returns

Results of the quantization, if any.

Return type

CompressResults

class leip.OptimizeOptions(*, compress: CompressOptions, compile: CompileOptions = CompileOptions(layout='NCHW', target='llvm -keys=cpu ', target_host=None, model_id=None, crc_check=False, force_int8=False, legacy_artifacts=False, optimization=CompileOptimizations(kernel=KernelOptimization(level=3), cuda=CudaOptimization(enabled=False), graph=None)))

Options for model optimization.

Parameters
Return type

None

compile: leip.core.operations.compile.options.CompileOptions

Options for compilation stage of the optimization.

compress: leip.core.operations.optimize.options.CompressOptions

Options for compression (quantization) stage of the optimization.

class leip.CompressOptions(*, quantizer: QuantizerType = QuantizerType.ASYMMETRIC, rep_dataset: Optional[FilePath] = None, optimization: List[CompressOptimization] = [], quantize_input: bool = False, quantize_output: bool = False, calibration_method: CalibrationMethod = CalibrationMethod.MINMAX, standard_deviations: PositiveFloat = 4.5)

Options for the compression stage of a model optimization.

Parameters

  • quantizer (QuantizerType) –

  • rep_dataset (Optional[FilePath]) –

  • optimization (List[CompressOptimization]) –

  • quantize_input (bool) –

  • quantize_output (bool) –

  • calibration_method (CalibrationMethod) –

  • standard_deviations (PositiveFloat) –

Return type

None

calibration_method: leip.core.models.enums.CalibrationMethod

The method to use for setting ranges from calibration data. One of [minmax, average, normal].

optimization: List[leip.core.models.enums.CompressOptimization]

Options to configure optimizations. Choose from [tensor_splitting, bias_correction].

quantize_input: bool

Whether the model input layers will be quantized.

quantize_output: bool

Whether the model output layers will be quantized.

quantizer: leip.core.models.enums.QuantizerType

Type of quantizer. One of [asymmetric, symmetric, symmetricpc, none].

rep_dataset: Optional[pydantic.types.FilePath]

The path to a text file that specifies a representative dataset input, used during calibration. This file should contain a newline separated list of path names to the input instances

standard_deviations: pydantic.types.PositiveFloat

The number of standard deviations to use when clipping a normal distribution of calibration range. The default is 4.5. This option is only valid for the “normal” calibration_method.