Usage¶

Installation¶

ddbcereal is hosted on PyPI so it’s as simple as using your favorite installer:

python -m pip install ddbcereal

poetry add ddbcereal

The package uses semantic versioning to indicate backwards compatible changes to the API.

Communicating with DynamoDB¶

ddbcereal does not transport data to or from DynamoDB. It’s up to you to provide that layer. This is usually done with an AWS SDK or an HTTP library.

AWS SDKs¶

ddbcereal is known to work with these libraries:

aiobotocore
botocore
aioboto3’s low-level interface
boto3’s low-level interface

aiobotocore will be used for most examples in this documentation.

Raw HTTP API¶

ddbcereal can also work directly with the DynamoDB HTTP API. Before and after JSON processing, dicts can be fed to Serializers and Deserializers constructed with raw_transport=True.

Basic Usage¶

Create a Serializer to process data into the native DynamoDB format:

serializer.serialize(value) to serialize individual values
serializer.serialize_item(mapping) to serialize an entire dict of values.

Create Deserializer for getting DynamoDB data into native Python values:

deserializer.deserialize(value) to deserialize individual values
deserializer.deserialize_item(mapping) for complete items from the AWS SDK

Serialize Python Data for DynamoDB¶

To prepare data for DynamoDB, construct a serializer object and use it to serialize items and attribute values as needed.

import aiobotocore
import ddbcereal

# Usually done at module level:
serializer = ddbcereal.Serializer()
aws = aiobotocore.Session()

async with aws.create_client('dynamodb') as ddb:
    # For a given dict:
    await ddb.put_item(
        TableName='MyItems',
        Item=serializer.serialize_item(my_dict)
    )

    # Adjust a single value:
    await ddb.update_item(
        TableName='Customers',
        Key={'id': serializer.serialize(customer_id)},
        UpdateExpression='SET displayName = :name, licenses = :licenses',
        ExpressionAttributeValues={
            ':name': serializer.serialize('ACME, Inc.'),
            ':licenses': serializer.serialize(20)
        }
    )

Serializer Options¶

Serializers can be configured to handle data in different ways according to your needs.

class Serializer(allow_inexact=False, validate_numbers=True, raw_transport=False, datetime_format=ddbcereal.ISO_8601, fraction_type=ddbcereal.NUMBER, empty_set_type=ddbcereal.NUMBER_SET)¶

Parameters

allow_inexact (bool) – Whether to allow numbers whose exact value can’t be represented in DynamoDB or Python. DynamoDB’s Number type stores exact numbers (fixed decimals). floats are considered inexact by their nature and are only accepted with this option enabled.
validate_numbers (bool) –
Whether to check inputted numbers to determine if they’re valid for storage in DynamoDB and whether or not they conform to the allow_inexact parameter.

When enabled, attempts to serialize invalid numbers will result in a ValueError being raised. When disabled, serialization is faster, but mistakes might only be caught after the serialized value has been sent to DynamoDB.
raw_transport (bool) – Indicates that values have not been pre-processed. For example, Base 64 strings have not been converted to bytes. Use this when using the AWS HTTP API without an AWS SDK.
datetime_format (DateFormat) – Determines how Python datetimes should be serialized. Possible enumerations are available on the ddbcereal top level module and the DateFormat enum.
fraction_type (DynamoDBType) – Determines how Python Fractions should be serialized. Must be NUMBER or STRING. Enumerations are available on the ddbcereal top level module and the DynamoDBType enum.
empty_set_type (DynamoDBType) – When an empty set is serialized, make the set this DynamoDB type. Must be NUMBER_SET, STRING_SET, or BINARY_SET. Enumerations are available on the ddbcereal top level module and the DynamoDBType enum.

class ddbcereal.DateFormat(value)¶

An enumeration.

ISO_8601 = 3¶: DynamoDB String containing an ISO 8601 date string e.g. 2021-07-18T05:40:59.442117+00:00

UNIX_MILLISECONDS = 2¶: DynamoDB Number holding number of milliseconds since the Unix epoch.

UNIX_SECONDS = 1¶: DynamoDB Number holding number of seconds since the Unix epoch. This is the only date/time format that can used as a TTL field for automatic item expiration in DynamoDB.

class ddbcereal.DynamoDBType(value)¶

An enumeration.

BINARY_SET = 'BS'¶

NUMBER = 'N'¶

NUMBER_SET = 'NS'¶

STRING = 'S'¶

STRING_SET = 'SS'¶

Deserialize DynamoDB Data into Python¶

Construct a Deserializer object and use it to deserialize items or attribute values as needed.

import aiobotocore
import ddbcereal

deserializer = ddbcereal.Deserializer()

serializer = ddbcereal.Serializer()
aws = aiobotocore.Session()

async with aws.create_client('dynamodb') as ddb:
    response = await ddb.query(
        TableName='Companies',
        KeyConditionExpression='id = :id',
        ExpressionAttributeValues={
            ':id': serializer.serialize(target_id)
        }
    )
    companies = [
        deserializer.deserialize_item(item)
        for item in response.get('Items', ())
    ]
    process_companies(companies)

Deserializer Options¶

class Deserializer(allow_inexact=False, raw_transport=False, number_type: PythonNumber = PythonNumber.DECIMAL_ONLY, null_value: Any = None, null_factory: Callable[[], Any] = None)¶

Parameters

allow_inexact (bool) – Whether to allow conversion to a Python number that won’t exactly convey the value stored in DynamoDB (e.g. rounding of significant digits is required). Deserializing numbers to floats is only possible when this is enabled.
raw_transport (bool) – Indicates to deserialize values to be transported without additional processing. Bytes will be transported as Base 64 strings. Use this when using the AWS HTTP API without an AWS SDK.
python_number (PythonNumber) –
Determines how DynamoDB Numbers should be serialized. Possible enumerations are available on the ddbcereal top level module and the PythonNumber enum:

class PythonNumber(value)¶

An enumeration.

PythonNumber.DECIMAL_ONLY = 1¶

Only use decimal.Decimal. This is the default and the Python equivalent of DynamoDB Numbers.

PythonNumber.FLOAT_ONLY = 2¶

Convert to Python float, losing accuracy if necessary.

PythonNumber.FRACTION_ONLY = 3¶

Convert to fractions.Fraction. Maintains exactness.

PythonNumber.INT_ONLY = 4¶

Only use Python ints. Will round if necessary if allow_inexact is True

PythonNumber.INT_OR_DECIMAL = 5¶

Use int if the Number is whole, otherwise use decimal.Decimal

PythonNumber.INT_OR_FLOAT = 6¶

Use int if the Number is whole, otherwise use a float, losing accuracy if necessary.

PythonNumber.MOST_COMPACT = 7¶

Use int if the Number is whole, use float if a float’s decimal character representation matches the number, otherwise use a decimal.Decimal.
python_null_value – The Python value to convert DynamoDB Nulls to. Defaults to None. An immutable value is recommended. Ignored if python_null_factory is supplied.
python_null_factory (Callable[[], Any]) – A function invoked for every DynamoDB Null value. The Null is converted to the return value of the function. python_null_value is ignored if this is supplied.

Going Beyond the Basic Types¶

ddbcereal deserializers don’t know the final shape you want your data to conform to. They find appropriate Python types for the few types of data that DynamoDB can store. If you want to deserialize values into more advanced types, consider using a marshalling library like marshmallow or Pydantic.

They can take the dict produced by deserialize_item and create an objec based on a schema, an object with fields of built-in types like dates, deques and of custom types.

See marshmallow.Schema.load() and Pydantic Models - Helper Functions.

Exceptions¶

exception ddbcereal.NumberInexactError¶

Bases: ValueError

A supplied number can’t be represented exactly by the target type and would either lose intent or data.

exception ddbcereal.NumberNotAllowedError¶