0.0.0+develop

flytekitplugins.onnxscikitlearn.schema

Directory

Classes

Class Description
ScikitLearn2ONNX
ScikitLearn2ONNXConfig ScikitLearn2ONNXConfig is the config used during the scikitlearn to ONNX conversion.
ScikitLearn2ONNXTransformer Base transformer type that should be implemented for every python native type that can be handled by flytekit.

Methods

Method Description
extract_config()
to_onnx()

Methods

extract_config()

def extract_config(
    t: Type[ScikitLearn2ONNX],
) -> Tuple[Type[ScikitLearn2ONNX], ScikitLearn2ONNXConfig]
Parameter Type
t Type[ScikitLearn2ONNX]

to_onnx()

def to_onnx(
    ctx,
    model,
    config,
)
Parameter Type
ctx
model
config

flytekitplugins.onnxscikitlearn.schema.ScikitLearn2ONNX

class ScikitLearn2ONNX(
    model: sklearn.base.BaseEstimator,
)
Parameter Type
model sklearn.base.BaseEstimator

Methods

Method Description
from_dict()
from_json()
schema()
to_dict()
to_json()

from_dict()

def from_dict(
    kvs: typing.Union[dict, list, str, int, float, bool, NoneType],
    infer_missing,
) -> ~A
Parameter Type
kvs typing.Union[dict, list, str, int, float, bool, NoneType]
infer_missing

from_json()

def from_json(
    s: typing.Union[str, bytes, bytearray],
    parse_float,
    parse_int,
    parse_constant,
    infer_missing,
    kw,
) -> ~A
Parameter Type
s typing.Union[str, bytes, bytearray]
parse_float
parse_int
parse_constant
infer_missing
kw

schema()

def schema(
    infer_missing: bool,
    only,
    exclude,
    many: bool,
    context,
    load_only,
    dump_only,
    partial: bool,
    unknown,
) -> SchemaType[A]
Parameter Type
infer_missing bool
only
exclude
many bool
context
load_only
dump_only
partial bool
unknown

to_dict()

def to_dict(
    encode_json,
) -> typing.Dict[str, typing.Union[dict, list, str, int, float, bool, NoneType]]
Parameter Type
encode_json

to_json()

def to_json(
    skipkeys: bool,
    ensure_ascii: bool,
    check_circular: bool,
    allow_nan: bool,
    indent: typing.Union[int, str, NoneType],
    separators: typing.Tuple[str, str],
    default: typing.Callable,
    sort_keys: bool,
    kw,
) -> str
Parameter Type
skipkeys bool
ensure_ascii bool
check_circular bool
allow_nan bool
indent typing.Union[int, str, NoneType]
separators typing.Tuple[str, str]
default typing.Callable
sort_keys bool
kw

flytekitplugins.onnxscikitlearn.schema.ScikitLearn2ONNXConfig

ScikitLearn2ONNXConfig is the config used during the scikitlearn to ONNX conversion.

class ScikitLearn2ONNXConfig(
    initial_types: List[Tuple[str, Type]],
    name: Optional[str],
    doc_string: str,
    target_opset: Optional[int],
    custom_conversion_functions: Dict[Callable[..., Any], Callable[..., None]],
    custom_shape_calculators: Dict[Callable[..., Any], Callable[..., None]],
    custom_parsers: Dict[Callable[..., Any], Callable[..., None]],
    options: Dict[Any, Any],
    intermediate: bool,
    naming: Optional[Union[str, Callable[..., Any]]],
    white_op: Optional[Set[str]],
    black_op: Optional[Set[str]],
    verbose: int,
    final_types: Optional[List[Tuple[str, Type]]],
)
Parameter Type
initial_types List[Tuple[str, Type]]
name Optional[str]
doc_string str
target_opset Optional[int]
custom_conversion_functions Dict[Callable[..., Any], Callable[..., None]]
custom_shape_calculators Dict[Callable[..., Any], Callable[..., None]]
custom_parsers Dict[Callable[..., Any], Callable[..., None]]
options Dict[Any, Any]
intermediate bool
naming Optional[Union[str, Callable[..., Any]]]
white_op Optional[Set[str]]
black_op Optional[Set[str]]
verbose int
final_types Optional[List[Tuple[str, Type]]]

Methods

Method Description
from_dict()
from_json()
schema()
to_dict()
to_json()

from_dict()

def from_dict(
    kvs: typing.Union[dict, list, str, int, float, bool, NoneType],
    infer_missing,
) -> ~A
Parameter Type
kvs typing.Union[dict, list, str, int, float, bool, NoneType]
infer_missing

from_json()

def from_json(
    s: typing.Union[str, bytes, bytearray],
    parse_float,
    parse_int,
    parse_constant,
    infer_missing,
    kw,
) -> ~A
Parameter Type
s typing.Union[str, bytes, bytearray]
parse_float
parse_int
parse_constant
infer_missing
kw

schema()

def schema(
    infer_missing: bool,
    only,
    exclude,
    many: bool,
    context,
    load_only,
    dump_only,
    partial: bool,
    unknown,
) -> SchemaType[A]
Parameter Type
infer_missing bool
only
exclude
many bool
context
load_only
dump_only
partial bool
unknown

to_dict()

def to_dict(
    encode_json,
) -> typing.Dict[str, typing.Union[dict, list, str, int, float, bool, NoneType]]
Parameter Type
encode_json

to_json()

def to_json(
    skipkeys: bool,
    ensure_ascii: bool,
    check_circular: bool,
    allow_nan: bool,
    indent: typing.Union[int, str, NoneType],
    separators: typing.Tuple[str, str],
    default: typing.Callable,
    sort_keys: bool,
    kw,
) -> str
Parameter Type
skipkeys bool
ensure_ascii bool
check_circular bool
allow_nan bool
indent typing.Union[int, str, NoneType]
separators typing.Tuple[str, str]
default typing.Callable
sort_keys bool
kw

flytekitplugins.onnxscikitlearn.schema.ScikitLearn2ONNXTransformer

Base transformer type that should be implemented for every python native type that can be handled by flytekit

def ScikitLearn2ONNXTransformer()

Methods

Method Description
assert_type()
from_binary_idl() This function primarily handles deserialization for untyped dicts, dataclasses, Pydantic BaseModels, and attribute access.
from_generic_idl() TODO: Support all Flyte Types.
get_literal_type() Converts the python type to a Flyte LiteralType.
guess_python_type() Converts the Flyte LiteralType to a python object type.
isinstance_generic()
to_html() Converts any python val (dataframe, int, float) to a html string, and it will be wrapped in the HTML div.
to_literal() Converts a given python_val to a Flyte Literal, assuming the given python_val matches the declared python_type.
to_python_value() Converts the given Literal to a Python Type.

assert_type()

def assert_type(
    t: Type[T],
    v: T,
)
Parameter Type
t Type[T]
v T

from_binary_idl()

def from_binary_idl(
    binary_idl_object: Binary,
    expected_python_type: Type[T],
) -> Optional[T]

This function primarily handles deserialization for untyped dicts, dataclasses, Pydantic BaseModels, and attribute access.｀

For untyped dict, dataclass, and pydantic basemodel: Life Cycle (Untyped Dict as example): python val -> msgpack bytes -> binary literal scalar -> msgpack bytes -> python val (to_literal) (from_binary_idl)

For attribute access: Life Cycle: python val -> msgpack bytes -> binary literal scalar -> resolved golang value -> binary literal scalar -> msgpack bytes -> python val (to_literal) (propeller attribute access) (from_binary_idl)

Parameter Type
binary_idl_object Binary
expected_python_type Type[T]

from_generic_idl()

def from_generic_idl(
    generic: Struct,
    expected_python_type: Type[T],
) -> Optional[T]

TODO: Support all Flyte Types. This is for dataclass attribute access from input created from the Flyte Console.

Note:

  • This can be removed in the future when the Flyte Console support generate Binary IDL Scalar as input.
Parameter Type
generic Struct
expected_python_type Type[T]

get_literal_type()

def get_literal_type(
    t: Type[ScikitLearn2ONNX],
) -> LiteralType

Converts the python type to a Flyte LiteralType

Parameter Type
t Type[ScikitLearn2ONNX]

guess_python_type()

def guess_python_type(
    literal_type: LiteralType,
) -> Type[ScikitLearn2ONNX]

Converts the Flyte LiteralType to a python object type.

Parameter Type
literal_type LiteralType

isinstance_generic()

def isinstance_generic(
    obj,
    generic_alias,
)
Parameter Type
obj
generic_alias

to_html()

def to_html(
    ctx: FlyteContext,
    python_val: T,
    expected_python_type: Type[T],
) -> str

Converts any python val (dataframe, int, float) to a html string, and it will be wrapped in the HTML div

Parameter Type
ctx FlyteContext
python_val T
expected_python_type Type[T]

to_literal()

def to_literal(
    ctx: FlyteContext,
    python_val: ScikitLearn2ONNX,
    python_type: Type[ScikitLearn2ONNX],
    expected: LiteralType,
) -> Literal

Converts a given python_val to a Flyte Literal, assuming the given python_val matches the declared python_type. Implementers should refrain from using type(python_val) instead rely on the passed in python_type. If these do not match (or are not allowed) the Transformer implementer should raise an AssertionError, clearly stating what was the mismatch

Parameter Type
ctx FlyteContext
python_val ScikitLearn2ONNX
python_type Type[ScikitLearn2ONNX]
expected LiteralType

to_python_value()

def to_python_value(
    ctx: FlyteContext,
    lv: Literal,
    expected_python_type: Type[ONNXFile],
) -> ONNXFile

Converts the given Literal to a Python Type. If the conversion cannot be done an AssertionError should be raised

Parameter Type
ctx FlyteContext
lv Literal
expected_python_type Type[ONNXFile]

Properties

Property Type Description
is_async
name
python_type
This returns the python type
type_assertions_enabled
Indicates if the transformer wants type assertions to be enabled at the core type engine layer