Encoders¶
Warning
The current page still doesn't have a translation for this language.
But you can help translating it: Contributing.
Encoders are basically converters between datastructures and json. Its main use is in response were different return values are automatically parsed into json objects.
For understanding what they are doing, it is helpful to go through the manual usage
Manual usage¶
Encoders provide two public methods:
- json_encode
- apply_structure
json_encode¶
json_encode
is basically an enhanced json.dumps. It provides via the encoder converters which parse additional
types to json.
By default only a simplification is done, therefore the json string is deserialized again so a simple datastructure is returned.
from lilya.encoders import json_encode
json_string = json_encode({"hello": "world"}, post_transform_fn=None)
# or
json_string = json_encode({"hello": "world"}, post_transform_fn=lambda x: x)
apply_structure¶
apply_structure
is basically the inverse of json_encode
.
It assumes it is known from which structure a value was serlialized.
If one encoder matches via is_type_structure
(for Encoder by default isclass(structure) and issubclass(structure, self.__type__)
) the encoder is used.
First is checked via is_type
if the value is already converted and if yes simply returned.
Otherwise the value is molded via the structure in an instance and returned.
from typing import Any
from dataclasses import dataclass
from lilya.encoders import apply_structure, json_encode
@dataclass
class Foo:
a: int
b: int
simplified = json_encode(Foo(a=3, b=5))
# dict {"a": 3, "b": 5}
foo = apply_structure(Foo, simplified)
# now a Foo object again
assert foo == Foo(a=3, b=5)
Default Encoders¶
In order to understand how to serialize a specific object into json
, Lilya has some default
encoders that evaluates when tries to guess the response type.
DataclassEncoder
- Serialisesdataclass
objects.NamedTupleEncoder
- SerialisesNamedTuple
objects.ModelDumpEncoder
- Serialises objects by calling its model_dump method. This allows serializing pydantic objects out of the box.EnumEncoder
- SerialisesEnum
objects.PurePathEncoder
- SerializesPurePath
objects.DateEncoder
- Serializes date and datetime objects.StructureEncoder
- Serializes more complex data types which implementIterable
.
What if a brand new encoder is needed and it is not natively supported by Lilya? Well, building a custom encoder is extremly easy and possible.
All default encoders are also implementing the molding protocol.
Build a custom encoder¶
As mentioned before, Lilya has default encoders that are used to transform a response
into a json
serializable response.
To build a custom encoder you must implement the EncoderProtocol
or MoldingProtocol
.
You can use the Encoder
helper class from Lilya for that and provide the serialize()
and/or encode
function
where it applies the serialisation process of the encoder type.
If the encoder should also be able to deserialize a value in an provided object, you need additionally the method:
encode
and maybe the method is_type_structure
.
Then you must register the encoder for Lilya to use it.
When defining an encoder the def is_type(self, value: Any) -> bool:
or (Encoder helper class only)__type__
must be declared or overridden.
When in an Encoder subclass the __type__
is properly declared, the default is_type
and is_type_structure
(MoldingProtocol) will evaluate the object against the
type and return True
or False
. __type__
can be a single type or a tuple of types.
This is used internally to understand the type of encoder that will be applied to a given object.
Warning
If you are not able to provide the __type__
for any reason and you just want to override the
default evaluation process, simple use EncoderProtocol, override the is_type()
and apply your custom logic there.
E.g.: In Python 3.8, for a Pydantic BaseModel
if passed in the __type__
, it will throw an
error due to Pydantic internals, so to workaround this issue, you can simply override the is_type()
and apply the logic that validates the type of the object and returns a boolean.
Example
Create and register an encoder that handles msgspec.Struct
types.
from typing import Any
import msgspec
from msgspec import Struct
from lilya.encoders import Encoder, register_encoder
class MsgSpecEncoder(Encoder):
__type__ = Struct
def serialize(self, obj: Any) -> Any:
"""
When a `msgspec.Struct` is serialised,
it will call this function.
"""
return msgspec.json.decode(msgspec.json.encode(obj))
def encode(
self,
structure: Any,
obj: Any,
) -> Any:
return msgspec.json.decode(obj, type=structure)
# A normal way
register_encoder(MsgSpecEncoder())
# As alternative
register_encoder(MsgSpecEncoder)
Simple right? Because now the MsgSpecEncoder
is registered, you can simply do this in your handlers
and return directly the msgspec.Struct
object type.
from msgspec import Struct
from lilya.routing import Path
class User(Struct):
name: str
email: str
def msgspec_struct():
return User(name="lilya", url="example@lilya.dev")
Design specific custom encoders¶
Lilya being 100% pure python and not tight to any particular validation library allows you to design custom encoders that are later used by Lilya responses.
Ok, this sounds a bit confusing right? I bet it does so let us go slowly.
Imagine you want to use a particular validation library such as Pydantic, msgspec or even attrs or something else at your choice.
You want to make sure that if you return a pydantic model or a msgspec Struct or even a define
attr class.
Let us see how it would look like for all of them.
For pydantic
Nothing required anymore. Works out of the box thanks to the ModelDumpEncoder. But we can do an instance check instead:
from __future__ import annotations
from typing import Any
from pydantic import BaseModel
from lilya.encoders import Encoder, register_encoder
class PydanticEncoder(Encoder):
__type__ = BaseModel
# optional a name can be provided, so same named encoders are removed
name = "ModelDumpEncoder"
# is_type and is_type_structure are provided by Encoder.
# checked is the type provided by __type__.
def serialize(self, obj: BaseModel) -> dict[str, Any]:
return obj.model_dump()
def encode(self, structure: type[BaseModel], value: Any) -> Any:
return structure(**value)
# A normal way
register_encoder(PydanticEncoder())
# As alternative
register_encoder(PydanticEncoder)
For msgspec Struct
from typing import Any
import msgspec
from msgspec import Struct
from lilya.encoders import Encoder, register_encoder
class MsgSpecEncoder(Encoder):
__type__ = Struct
def serialize(self, obj: Any) -> Any:
"""
When a `msgspec.Struct` is serialised,
it will call this function.
"""
return msgspec.json.decode(msgspec.json.encode(obj))
def encode(
self,
structure: Any,
obj: Any,
) -> Any:
return msgspec.json.decode(obj, type=structure)
# A normal way
register_encoder(MsgSpecEncoder())
# As alternative
register_encoder(MsgSpecEncoder)
For attrs
from typing import Any
from attrs import asdict, has
from lilya.encoders import Encoder, register_encoder
class AttrsEncoder(Encoder):
def is_type(self, value: Any) -> bool:
"""
You can use this function instead of declaring
the `__type__`.
"""
return has(value)
def serialize(self, obj: Any) -> Any:
return asdict(obj)
def encode(self, structure: type[Any], obj: Any) -> Any:
if isinstance(obj, dict):
return structure(**obj)
return structure(*obj)
# A normal way
register_encoder(AttrsEncoder())
# As alternative
register_encoder(AttrsEncoder)
Easy and powerful, right? Yes.
Do you understand what does this mean? Means you can design any encoder at your choice using also any library of your choice as well.
The flexibility of Lilya allows you to be free and for Lilya not to be tight to any particular library.
Tip
You can replace other Encoders by providing a name attribute. By default all encoders use their class-name as name.
Custom encoders and responses¶
After the custom encoders in the examples are created, this allows to do something like this directly.
from attrs import define
from msgspec import Struct
from pydantic import BaseModel
from lilya.apps import Lilya
from lilya.routing import Path
class User(BaseModel):
name: str
age: int
class Item(Struct):
name: str
age: int
@define
class AttrItem:
name: str
age: int
def pydantic_response():
return User(name="lilya", age=24)
def pydantic_response_list():
return [User(name="lilya", age=24)]
def msgspec_struct():
return Item(name="lilya", age=24)
def msgspec_struct_list():
return [Item(name="lilya", age=24)]
def attrs_response():
return AttrItem(name="lilya", age=24)
def attrs_response_list():
return [AttrItem(name="lilya", age=24)]
app = Lilya(
routes=[
Path("/pydantic", pydantic_response),
Path("/pydantic-list", pydantic_response_list),
Path("/msgspec", msgspec_struct),
Path("/msgspec-list", pydantic_response_list),
Path("/attrs", attrs_response),
Path("/attrs-list", attrs_response_list),
]
)
Custom encoders and the make_response
¶
Well, here its where the make_response
helps you. The make_response
will generate a JSONResponse
by default and when you return a custom encoder type, there are some limitations to it.
For example, what if you want to return with a different status_code
? Or even attach a task
to it?
The custom encoder does not handle that for you but the make_response
does!
Let us see how it would look like now using the make_response
.
from attrs import define
from msgspec import Struct
from pydantic import BaseModel
from lilya import status
from lilya.apps import Lilya
from lilya.responses import make_response
from lilya.routing import Path
class User(BaseModel):
name: str
age: int
class Item(Struct):
name: str
age: int
@define
class AttrItem:
name: str
age: int
def pydantic_response():
data = User(name="lilya", age=24)
return make_response(
data,
status_code=status.HTTP_200_OK,
)
def pydantic_response_list():
data = [User(name="lilya", age=24)]
return make_response(
data,
status_code=status.HTTP_201_CREATED,
background=...,
headers=...,
)
def msgspec_struct():
return make_response(Item(name="lilya", age=24))
def msgspec_struct_list():
return make_response(
[Item(name="lilya", age=24)],
status_code=...,
)
def attrs_response():
return make_response(
AttrItem(name="lilya", age=24),
status_code=...,
)
def attrs_response_list():
return make_response(
[AttrItem(name="lilya", age=24)],
status_code=...,
)
app = Lilya(
routes=[
Path("/pydantic", pydantic_response),
Path("/pydantic-list", pydantic_response_list),
Path("/msgspec", msgspec_struct),
Path("/msgspec-list", pydantic_response_list),
Path("/attrs", attrs_response),
Path("/attrs-list", attrs_response_list),
]
)
Use with custom json encoder¶
Sometimes you might want a more performant json parser that is not the default python json built-in, for example, the orjson json serializer/deserializer.
This is no problem for Lilya:
from lilya.encoders import json_encode
import orjson
# orjson serializes to bytes, so apply str
json_string = json_encode({"hello": "world"}, json_encode_fn=orjson.dumps, post_transform_fn=str)
# or for simplifying
json_simplified = json_encode({"hello": "world"}, json_encode_fn=orjson.dumps, post_transform_fn=orjson.loads)
Or the make_response
example with orjson
from attrs import define
from msgspec import Struct
from pydantic import BaseModel
import orjson
from lilya import status
from lilya.apps import Lilya
from lilya.responses import make_response
from lilya.routing import Path
class User(BaseModel):
name: str
age: int
class Item(Struct):
name: str
age: int
@define
class AttrItem:
name: str
age: int
def pydantic_response():
data = User(name="lilya", age=24)
return make_response(
data,
status_code=status.HTTP_200_OK,
json_encode_extra_kwargs={
"json_encode_fn": orjson.dumps, "post_transform_fn": orjson.loads
}
)
def pydantic_response_list():
data = [User(name="lilya", age=24)]
return make_response(
data,
status_code=status.HTTP_201_CREATED,
background=...,
headers=...,
json_encode_extra_kwargs={
"json_encode_fn": orjson.dumps, "post_transform_fn": orjson.loads
}
)
def msgspec_struct():
return make_response(Item(name="lilya", age=24))
def msgspec_struct_list():
return make_response(
[Item(name="lilya", age=24)],
status_code=...,
json_encode_extra_kwargs={
"json_encode_fn": orjson.dumps, "post_transform_fn": orjson.loads
}
)
def attrs_response():
return make_response(
AttrItem(name="lilya", age=24),
status_code=...,
json_encode_extra_kwargs={
"json_encode_fn": orjson.dumps, "post_transform_fn": orjson.loads
}
)
def attrs_response_list():
return make_response(
[AttrItem(name="lilya", age=24)],
status_code=...,
json_encode_extra_kwargs={
"json_encode_fn": orjson.dumps, "post_transform_fn": orjson.loads
}
)
app = Lilya(
routes=[
Path("/pydantic", pydantic_response),
Path("/pydantic-list", pydantic_response_list),
Path("/msgspec", msgspec_struct),
Path("/msgspec-list", pydantic_response_list),
Path("/attrs", attrs_response),
Path("/attrs-list", attrs_response_list),
]
)