LEIP Compile

leip.compile_(model: CompilableModel, output_path: Path, options: Optional[CompileOptions] = CompileOptions(layout='NCHW', target='llvm -keys=cpu ', target_host=None, model_id=None, crc_check=False, force_int8=False, legacy_artifacts=False, optimization=CompileOptimizations(kernel=KernelOptimization(level=3), cuda=CudaOptimization(enabled=False), graph=None))) → None

Compiles a model to an executable for a particular target.

Parameters

  • model (CompilableModel) – Model object.

  • output_path (Path) – Directory path for generated artifacts.

  • options (Optional[CompileOptions]) – Options that configure the compilation.

Return type

None

class leip.CompileOptions(*, layout: Layout = Layout.NCHW, target: str = 'llvm', target_host: Optional[str] = None, model_id: Optional[str] = None, crc_check: bool = False, force_int8: bool = False, legacy_artifacts: bool = False, optimization: CompileOptimizations = CompileOptimizations(kernel=KernelOptimization(level=3), cuda=CudaOptimization(enabled=False), graph=None))

Options for model compilation.

Parameters

  • layout (Layout) –

  • target (str) –

  • target_host (Optional[str]) –

  • model_id (Optional[str]) –

  • crc_check (bool) –

  • force_int8 (bool) –

  • legacy_artifacts (bool) –

  • optimization (CompileOptimizations) –

Return type

None

crc_check: bool

If True, a CRC value is stored in model_schema.json, which is used to verify data corruption when loading.

force_int8: bool

Enforces the storage of parameters as int8 in the compiled output.

layout: leip.core.models.enums.Layout

Desired target layout for classification models. One of [NCHW, NHWC].

legacy_artifacts: bool

If True, three separate artifacts will be saved (lib, graph, params).

model_id: Optional[str]

Vendor specific model ID. If not present, a UUID will be generated.

optimization: leip.core.operations.compile.options.CompileOptimizations

Options to configure optimizations.

target: str

Target description and codgen module in the format <target_kind> [-option=value], where option may be:

-device=<device name>

The device name.

-mtriple=<target triple>

Specify the target triple, which is useful for cross compilation.

-mcpu=<cpuname>

Specify a specific chip in the current architecture to generate code for. By default this is infered from the target triple and autodetected to the current architecture.

-arch=<target architecture>

The target architecture. Examples: sm_20, sm_75.

-mattr=a1,+a2,-a3,…

Override or control specific attributes of the target, such as whether SIMD operations are enabled or not. The default set of attributes is set by the current CPU.

-mabi=<abi>

Generate code for the specified ABI, for example “lp64d”.

target_host: Optional[str]

Host compilation target, if target is cuda. Format is the same as target.

Optimizations

class leip.CompileOptimizations(*, kernel: KernelOptimization = KernelOptimization(level=3), cuda: CudaOptimization = CudaOptimization(enabled=False), graph: Optional[GraphOptimization] = None)

Optimizations for compilation, available in LEIP Compile and LEIP Optimize.

Parameters
Return type

None

cuda: leip.core.operations.compile.options.CudaOptimization

Optimization specific for CUDA targets.

graph: Optional[leip.core.operations.compile.options.GraphOptimization]

Optimization at the graph level.

kernel: leip.core.operations.compile.options.KernelOptimization

Optimization at the kernel level.

class leip.KernelOptimization(*, level: OptLevel = 3)

Optimization for compilation at the kernel level.

Parameters

level (OptLevel) –

Return type

None

level: leip.core.operations.compile.options.OptLevel

Level between 1 and 4. Higher numbers provide greater optimization but longer compilation time.

class leip.GraphOptimization(*, iterations: GraphOptimizationIterations = 0)

Optimization for compilation at the graph level.

Parameters

iterations (GraphOptimizationIterations) –

Return type

None

iterations: leip.core.operations.compile.options.GraphOptimizationIterations

Number of iterations to run the optimization.

class leip.CudaOptimization(*, enabled: bool = False)

Optimization specific for CUDA targets.

Parameters

enabled (bool) –

Return type

None

enabled: bool

Indicates if it’s enabled or not. The default is True in GPU environments, False otherwise.