Core

Modules

qonnx.core.data_layout

qonnx.core.data_layout.get_channels_first_layout_for_ndims(ndims)

qonnx.core.data_layout.get_channels_last_layout_for_ndims(ndims)

qonnx.core.data_layout.is_channels_last(layout)

qonnx.core.datatype

class qonnx.core.datatype.BaseDataType

Bases: ABC

Base class for QONNX data types.

abstract allowed(value)

Check whether given value is allowed for this DataType.

value (float32): value to be checked

abstract bitwidth(): Returns the number of bits required for this DataType.

abstract get_canonical_name(): Return a canonical string representation of this QONNX DataType.

abstract get_hls_datatype_str(): Returns the corresponding Vivado HLS datatype name.

abstract get_num_possible_values(): Returns the number of possible values this DataType can take. Only implemented for integer types for now.

abstract is_fixed_point(): Returns whether this DataType represent fixed-point values only.

abstract is_integer(): Returns whether this DataType represents integer values only.

abstract max(): Returns the largest possible value allowed by this DataType.

abstract min(): Returns the smallest possible value allowed by this DataType.

property name

signed(): Returns whether this DataType can represent negative numbers.

abstract to_numpy_dt(): Return an appropriate numpy datatype that can represent this QONNX DataType.

class qonnx.core.datatype.BipolarType

Bases: BaseDataType

allowed(value)

Check whether given value is allowed for this DataType.

value (float32): value to be checked

bitwidth(): Returns the number of bits required for this DataType.

get_canonical_name(): Return a canonical string representation of this QONNX DataType.

get_hls_datatype_str(): Returns the corresponding Vivado HLS datatype name.

get_num_possible_values(): Returns the number of possible values this DataType can take. Only implemented for integer types for now.

is_fixed_point(): Returns whether this DataType represent fixed-point values only.

is_integer(): Returns whether this DataType represents integer values only.

max(): Returns the largest possible value allowed by this DataType.

min(): Returns the smallest possible value allowed by this DataType.

to_numpy_dt(): Return an appropriate numpy datatype that can represent this QONNX DataType.

class qonnx.core.datatype.DataType(value)

Bases: Enum

Enum class that contains QONNX data types to set the quantization annotation. ONNX does not support data types smaller than 8-bit integers, whereas in QONNX we are interested in smaller integers down to ternary and bipolar.

static get_accumulator_dt_cands()

static get_smallest_possible(value): Returns smallest (fewest bits) possible DataType that can represent value. Prefers unsigned integers where possible.

class qonnx.core.datatype.DataTypeMeta(cls, bases, classdict): Bases: EnumMeta

class qonnx.core.datatype.FixedPointType(bitwidth, intwidth)

Bases: IntType

allowed(value)

Check whether given value is allowed for this DataType.

value (float32): value to be checked

frac_bits()

get_canonical_name(): Return a canonical string representation of this QONNX DataType.

get_hls_datatype_str(): Returns the corresponding Vivado HLS datatype name.

int_bits()

is_fixed_point(): Returns whether this DataType represent fixed-point values only.

is_integer(): Returns whether this DataType represents integer values only.

max(): Returns the largest possible value allowed by this DataType.

min(): Returns the smallest possible value allowed by this DataType.

scale_factor()

to_numpy_dt(): Return an appropriate numpy datatype that can represent this QONNX DataType.

class qonnx.core.datatype.Float16Type

Bases: BaseDataType

allowed(value)

Check whether given value is allowed for this DataType.

value (float32): value to be checked

bitwidth(): Returns the number of bits required for this DataType.

get_canonical_name(): Return a canonical string representation of this QONNX DataType.

get_hls_datatype_str(): Returns the corresponding Vivado HLS datatype name.

get_num_possible_values(): Returns the number of possible values this DataType can take. Only implemented for integer types for now.

is_fixed_point(): Returns whether this DataType represent fixed-point values only.

is_integer(): Returns whether this DataType represents integer values only.

max(): Returns the largest possible value allowed by this DataType.

min(): Returns the smallest possible value allowed by this DataType.

to_numpy_dt(): Return an appropriate numpy datatype that can represent this QONNX DataType.

class qonnx.core.datatype.FloatType

Bases: BaseDataType

allowed(value)

Check whether given value is allowed for this DataType.

value (float32): value to be checked

bitwidth(): Returns the number of bits required for this DataType.

get_canonical_name(): Return a canonical string representation of this QONNX DataType.

get_hls_datatype_str(): Returns the corresponding Vivado HLS datatype name.

get_num_possible_values(): Returns the number of possible values this DataType can take. Only implemented for integer types for now.

is_fixed_point(): Returns whether this DataType represent fixed-point values only.

is_integer(): Returns whether this DataType represents integer values only.

max(): Returns the largest possible value allowed by this DataType.

min(): Returns the smallest possible value allowed by this DataType.

to_numpy_dt(): Return an appropriate numpy datatype that can represent this QONNX DataType.

class qonnx.core.datatype.IntType(bitwidth, signed)

Bases: BaseDataType

allowed(value)

Check whether given value is allowed for this DataType.

value (float32): value to be checked

bitwidth(): Returns the number of bits required for this DataType.

get_canonical_name(): Return a canonical string representation of this QONNX DataType.

get_hls_datatype_str(): Returns the corresponding Vivado HLS datatype name.

get_num_possible_values(): Returns the number of possible values this DataType can take. Only implemented for integer types for now.

is_fixed_point(): Returns whether this DataType represent fixed-point values only.

is_integer(): Returns whether this DataType represents integer values only.

max(): Returns the largest possible value allowed by this DataType.

min(): Returns the smallest possible value allowed by this DataType.

to_numpy_dt(): Return an appropriate numpy datatype that can represent this QONNX DataType.

class qonnx.core.datatype.ScaledIntType(bitwidth)

Bases: IntType

allowed(value)

Check whether given value is allowed for this DataType.

value (float32): value to be checked

get_canonical_name(): Return a canonical string representation of this QONNX DataType.

get_hls_datatype_str(): Returns the corresponding Vivado HLS datatype name.

is_fixed_point(): Returns whether this DataType represent fixed-point values only.

is_integer(): Returns whether this DataType represents integer values only.

max(): Returns the largest possible value allowed by this DataType.

min(): Returns the smallest possible value allowed by this DataType.

signed(): Returns whether this DataType can represent negative numbers.

to_numpy_dt(): Return an appropriate numpy datatype that can represent this QONNX DataType.

class qonnx.core.datatype.TernaryType

Bases: BaseDataType

allowed(value)

Check whether given value is allowed for this DataType.

value (float32): value to be checked

bitwidth(): Returns the number of bits required for this DataType.

get_canonical_name(): Return a canonical string representation of this QONNX DataType.

get_hls_datatype_str(): Returns the corresponding Vivado HLS datatype name.

get_num_possible_values(): Returns the number of possible values this DataType can take. Only implemented for integer types for now.

is_fixed_point(): Returns whether this DataType represent fixed-point values only.

is_integer(): Returns whether this DataType represents integer values only.

max(): Returns the largest possible value allowed by this DataType.

min(): Returns the smallest possible value allowed by this DataType.

to_numpy_dt(): Return an appropriate numpy datatype that can represent this QONNX DataType.

qonnx.core.datatype.resolve_datatype(name)

qonnx.core.execute_custom_node

qonnx.core.execute_custom_node.execute_custom_node(node, context, graph, onnx_opset_version=11): Call custom implementation to execute a single custom node. Input/output provided via context.

qonnx.core.modelwrapper

class qonnx.core.modelwrapper.ModelWrapper(onnx_model_proto, make_deepcopy=False, fix_float64=False, fix_missing_initializer_valueinfo=True)

Bases: object

A wrapper around ONNX ModelProto that exposes some useful utility functions for graph manipulation and exploration.

analysis(analysis_fxn): Runs given anaylsis_fxn on this model and return resulting dict.

check_all_tensor_shapes_specified(fix_missing_init_shape=False): Checks whether all tensors have a specified shape (ValueInfo). The ONNX standard allows for intermediate activations to have no associated ValueInfo, but QONNX expects this. If fix_missing_init_shape is specified, it will add a ValueInfoProto for initializers that are missing theirs.

check_compatibility()

Checks this model for QONNX compatibility:

no embedded subgraphs
all tensor shapes are specified, including activations
all constants are initializers

cleanup(): Run cleanup transformations on the model.

find_consumer(tensor_name): Finds and returns the node that consumes the tensor with given name. If there are multiple consumers, only the first one is returned. If there are no consumers, returns None.

find_consumers(tensor_name): Finds and returns a list of the nodes that consume tensor with given name.

find_direct_predecessors(node): Finds and returns a list of the nodes that are predecessors of given node.

find_direct_successors(node): Finds and returns a list of the nodes that are successors of given node.

find_producer(tensor_name): Finds and returns the node that produces the tensor with given name.

find_upstream(tensor_name, finder_fxn, keep_if_not_found=False): Follow the producer chain upstream, calling finder_fxn on each upstream node until it returns True or there are no nodes left. Returns the list of nodes visited, or None if finder_fxn did not return True. If keep_if_not_found is specified, returns the list of nodes visited, even if finder_fxn never returned True, i.e., if the search terminated at an input or initializer.

get_all_tensor_names(): Returns a list of all (input, output and value_info) tensor names in the graph.

get_finn_nodes(): Returns a list of nodes where domain == ‘qonnx.*’.

get_initializer(tensor_name, return_dtype=False): Gets the initializer value for tensor with given name, if any. ret_dtype can be set to True to retrieve the TensorProto.DataType of the initializer by returning it as a second element of a tuple.

get_metadata_prop(key): Returns the value associated with metadata_prop with given key, or None otherwise.

get_node_from_name(node_name): Returns the node with the specified name, or None if not found.

get_node_index(node): Returns current index of given node, or None if not found.

get_nodes_by_op_type(op_type): Returns a list of nodes with specified op_type.

get_non_finn_nodes(): Returns a list of nodes where domain != ‘qonnx.*’.

get_tensor_datatype(tensor_name): Returns the QONNX DataType of tensor with given name.

get_tensor_fanout(tensor_name): Returns the number of nodes for which the tensor with given name is as input.

get_tensor_layout(tensor_name): Returns the data layout annotation of tensor with given name. The data layout is expressed as a list of strings with as many elements as the number of dimensions in the tensor shape. Each string annotates what is contained in that dimension. If there is no data layout annotation, None will be returned. Examples of data layout annotations: [“N”, “C”] is tensor[batch][channel] [“N”, “C”, “H”, “W”] is tensor[batch][channel][height][width] [“N”, “H”, “W”, “C”] is tensor[batch][height][width][channel]

get_tensor_shape(tensor_name, fix_missing_init_shape=False): Returns the shape of tensor with given name, if it has ValueInfoProto. If fix_missing_init_shape is specified, it will add a ValueInfoProto for initializers that are missing theirs.

get_tensor_sparsity(tensor_name): Returns the sparsity of a given tensor as dictionary.

get_tensor_valueinfo(tensor_name): Returns ValueInfoProto of tensor with given name, if it has one.

property graph: Returns the graph of the model.

is_fork_node(node): Checks if the given node is a fork, that is, the node has multiple direct successors

is_join_node(node): Checks if the given node is a join, that is, the node has multiple direct predecessors

make_empty_exec_context()

Creates an empty execution context for this model.

The execution context is a dictionary of all tensors used for the inference computation. Any initializer values will be taken into account, all other tensors will be zero.

make_new_valueinfo_name(): Returns a name that can be used for a new value_info.

property model: Returns the model.

rename_tensor(old_name, new_name): Renames a tensor from old_name to new_name.

save(filename): Saves the wrapper ONNX ModelProto into a file with given name.

set_initializer(tensor_name, tensor_value): Sets the initializer value for tensor with given name.

set_metadata_prop(key, value): Sets metadata property with given key to the given value.

set_tensor_datatype(tensor_name, datatype): Sets the QONNX DataType of tensor with given name.

set_tensor_layout(tensor_name, data_layout): Sets the data layout annotation of tensor with given name. See get_tensor_layout for examples.

set_tensor_shape(tensor_name, tensor_shape, dtype=None): Assigns shape in ValueInfoProto for tensor with given name. If override_dtype is None, it will try to preserve the existing datatype, otherwise defaults to single-precision float.

set_tensor_sparsity(tensor_name, sparsity_dict): Sets the sparsity annotation of a tensor with given name.

temporary_fix_oldstyle_domain()

transform(transformation, make_deepcopy=True, cleanup=True)

Applies given Transformation repeatedly until no more changes can be made and returns a transformed ModelWrapper instance.

make_deepcopy : operates on a new (deep)copy of model.
cleanup : execute cleanup transformations before returning

qonnx.core.onnx_exec

qonnx.core.onnx_exec.compare_execution(model_a, model_b, input_dict, compare_fxn=<function <lambda>>)

Executes two ONNX models and compare their outputs using given function.

compare_fxn should take in two tensors and return a Boolean

qonnx.core.onnx_exec.execute_node(node, context, graph, return_full_exec_context=False, opset_version=11)

Executes a single node by using onnxruntime or with a custom function.

Input/output provided via context.

qonnx.core.onnx_exec.execute_onnx(model, input_dict, return_full_exec_context=False, start_node=None, end_node=None)

Executes given ONNX ModelWrapper with given named inputs.

If return_full_exec_context is False, a dict of named outputs is returned as indicated by the model.graph.output.

If return return_full_exec_context is True, the full set of tensors used by the execution (including inputs, weights, activations and final outputs) will be returned as a dict.

When start_node and end_node are set to None, the whole graph is executed. If they are set to particular ONNX nodes, only the subgraph between (and including) those nodes is executed.

qonnx.core.onnx_exec.execute_onnx_and_make_model(model, input_dict): Executes given ONNX ModelWrapper with given named inputs and return a new ModelWrapper where an initializer is provided for each tensor as taken from the execution. This new model is useful for debugging, since it contains all the intermediate activation values.

finn.core.onnx_exec

finn.core.onnx_exec.compare_execution(model_a, model_b, input_dict, compare_fxn=<function <lambda>>)

Executes two ONNX models and compare their outputs using given function.

compare_fxn should take in two tensors and return a Boolean

finn.core.onnx_exec.execute_onnx(model, input_dict, return_full_exec_context=False, start_node=None, end_node=None): Executes given ONNX ModelWrapper with given named inputs. If return_full_exec_context is False, a dict of named outputs is returned as indicated by the model.graph.output. If return return_full_exec_context is True, the full set of tensors used by the execution (including inputs, weights, activations and final outputs) will be returned as a dict. When start_node and end_node are set to None, the whole graph is executed. If they are set to particular ONNX nodes, only the subgraph between (and including) those nodes is executed.

finn.core.onnx_exec.execute_onnx_and_make_model(model, input_dict): Executes given ONNX ModelWrapper with given named inputs and return a new ModelWrapper where an initializer is provided for each tensor as taken from the execution. This new model is useful for debugging, since it contains all the intermediate activation values.

finn.core.rtlsim_exec

finn.core.rtlsim_exec.rtlsim_exec(model, execution_context, pre_hook=None, post_hook=None): Use PyVerilator to execute given model with stitched IP. The execution context contains the input values. Hook functions can be optionally specified to observe/alter the state of the circuit, receiving the PyVerilator sim object as their first argument: - pre_hook : hook function to be called before sim start (after reset) - post_hook : hook function to be called after sim end

finn.core.throughput_test

finn.core.throughput_test.throughput_test_rtlsim(model, batchsize=100): Runs a throughput test for the given IP-stitched model. When combined with tracing, useful to determine bottlenecks and required FIFO sizes.