Schema Generation

Karrio uses strongly typed data structures for carrier integrations to ensure type safety and improve developer experience. This section covers how to generate Python data types from carrier API documentation.

Understanding API Schemas

Before generating schemas, you need to understand the carrier’s API structure. This involves:

Studying the carrier’s API documentation
Identifying key API endpoints (rating, shipping, tracking, etc.)
Understanding the request and response formats
Collecting sample API requests and responses

Schema Files Preparation

Karrio supports both JSON and XML API formats. The approach differs slightly depending on which format your carrier uses.

For JSON APIs

Create .json files based on the carrier’s API documentation and place them in the schemas/ directory of your extension.
For each API operation (rating, shipping, tracking), create at least:
- A request schema (e.g., rate_request.json)
- A response schema (e.g., rate_response.json)

Example JSON schema file for a rate request:

1{
2  "containerSpecifications": [
3    {
4      "dimensions": {
5        "height": 10,
6        "length": 15,
7        "unit": "IN",
8        "width": 12
9      },
10      "weight": {
11        "unit": "LB",
12        "value": 5
13      }
14    }
15  ],
16  "serviceTypes": ["EXPRESS", "GROUND"],
17  "shipFrom": {
18    "addressLine1": "123 Shipper St",
19    "city": "Shiptown",
20    "countryCode": "US",
21    "name": "Shipper Name",
22    "postalCode": "12345",
23    "stateOrRegion": "CA",
24    "phoneNumber": "123-456-7890"
25  },
26  "shipTo": {
27    "addressLine1": "456 Recipient Rd",
28    "city": "Receiverville",
29    "countryCode": "US",
30    "name": "Recipient Name",
31    "postalCode": "54321",
32    "stateOrRegion": "NY",
33    "phoneNumber": "098-765-4321"
34  },
35  "shipDate": "2023-09-15T12:00:00Z"
36}

For XML/SOAP APIs

Collect .xsd files from the carrier’s API documentation and place them in the schemas/ directory.
For SOAP-based APIs, extract the <xs:schema> sections from .wsdl files into separate .xsd files.

Example XML schema structure:

1<?xml version="1.0" encoding="UTF-8"?>
2<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
3  <xs:element name="RateRequest">
4    <xs:complexType>
5      <xs:sequence>
6        <xs:element name="FromPostalCode" type="xs:string"/>
7        <xs:element name="ToPostalCode" type="xs:string"/>
8        <xs:element name="Weight" type="xs:decimal"/>
9        <xs:element name="WeightUnit" type="xs:string"/>
10        <!-- Additional elements... -->
11      </xs:sequence>
12    </xs:complexType>
13  </xs:element>
14</xs:schema>

Creating the Generate Script

The generate script in your extension’s root directory is responsible for running the appropriate code generation tools. The script should be updated to include your schema files.

For JSON APIs

Karrio uses a modified version of quicktype to generate Python data classes from JSON schemas:

1#!/bin/bash
2
3# Ensure script exits on error
4set -e
5
6# Get the directory of this script
7DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
8
9# Define the source directory and the output package directory
10SOURCE_DIR="$DIR/schemas"
11PACKAGE_DIR="$DIR/karrio/schemas/freight_express"
12
13# Create output directory if it doesn't exist
14mkdir -p "$PACKAGE_DIR"
15
16# Run code generation for each schema file
17kcli codegen generate \
18  --src="$SOURCE_DIR/rate_request.json" \
19  --out="$PACKAGE_DIR/rate_request.py" \
20  --toplevel-class="RateRequest"
21
22kcli codegen generate \
23  --src="$SOURCE_DIR/rate_response.json" \
24  --out="$PACKAGE_DIR/rate_response.py" \
25  --toplevel-class="RateResponse"
26
27# Add more schema files as needed...
28
29# Create or update __init__.py to expose the generated classes
30echo "# Generated code - DO NOT modify by hand
31from karrio.schemas.freight_express.rate_request import RateRequest
32from karrio.schemas.freight_express.rate_response import RateResponse
33# Add more imports as needed...
34" > "$PACKAGE_DIR/__init__.py"
35
36echo "Code generation completed successfully!"

For XML/SOAP APIs

Karrio uses generateDS to transform XML schemas into Python classes:

1#!/bin/bash
2
3# Ensure script exits on error
4set -e
5
6# Get the directory of this script
7DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
8
9# Define the source directory and the output package directory
10SOURCE_DIR="$DIR/schemas"
11PACKAGE_DIR="$DIR/karrio/schemas/freight_express"
12
13# Create output directory if it doesn't exist
14mkdir -p "$PACKAGE_DIR"
15
16# Run generateDS for each schema file
17generateDS.py \
18  --no-dates \
19  --no-versions \
20  --member-specs=list \
21  -o "$PACKAGE_DIR/rate_request.py" \
22  "$SOURCE_DIR/rate_request.xsd"
23
24generateDS.py \
25  --no-dates \
26  --no-versions \
27  --member-specs=list \
28  -o "$PACKAGE_DIR/rate_response.py" \
29  "$SOURCE_DIR/rate_response.xsd"
30
31# Add more schema files as needed...
32
33# Create or update __init__.py to expose the generated classes
34echo "# Generated code - DO NOT modify by hand
35from karrio.schemas.freight_express.rate_request import *
36from karrio.schemas.freight_express.rate_response import *
37# Add more imports as needed...
38" > "$PACKAGE_DIR/__init__.py"
39
40echo "Code generation completed successfully!"

Running Code Generation

After setting up your schema files and the generate script, you can run the code generation process:

Run the generate script
1bin/run-generate-on modules/connectors/[carrier_name]

This command runs your generate script to create Python data types from your schemas.

Generated Code Structure

The generated code will be placed in the karrio/schemas/[carrier_name]/ directory of your extension. Each schema file will generate a corresponding Python file with data classes that represent the API structures.

Example generated Python class for a JSON API:

1import attr
2import jstruct
3import typing
4
5@attr.s(auto_attribs=True)
6class Dimensions:
7    height: typing.Optional[float] = None
8    length: typing.Optional[float] = None
9    unit: typing.Optional[str] = None
10    width: typing.Optional[float] = None
11
12@attr.s(auto_attribs=True)
13class Weight:
14    unit: typing.Optional[str] = None
15    value: typing.Optional[float] = None
16
17@attr.s(auto_attribs=True)
18class ContainerSpecification:
19    dimensions: typing.Optional[Dimensions] = None
20    weight: typing.Optional[Weight] = None
21
22# More classes...
23
24@attr.s(auto_attribs=True)
25class RateRequest:
26    containerSpecifications: typing.Optional[typing.List[ContainerSpecification]] = jstruct.JList[ContainerSpecification]
27    serviceTypes: typing.Optional[typing.List[str]] = jstruct.JList[str]
28    shipFrom: typing.Optional[Address] = None
29    shipTo: typing.Optional[Address] = None
30    shipDate: typing.Optional[str] = None

Using Generated Types

Once generated, these Python data types become the foundation of your carrier integration. You’ll use them to:

Create carrier-specific API requests
Parse carrier-specific API responses
Map between Karrio’s unified format and carrier-specific formats

In the next sections, we’ll cover how to use these generated types in your integration.

Troubleshooting

If you encounter issues during code generation:

Ensure your schema files are valid (for JSON: valid JSON, for XML: valid XML schema)
Check that your generate script references the correct schema files
For XML APIs, ensure generateDS is installed with Karrio’s development dependencies

CLI Tools for Schema Generation

Karrio provides several CLI tools to help with schema generation and code transformation. The kcli command includes utilities for code generation, particularly for transforming JSON schemas into Python code using jstruct.

Transform

The transform command converts Python code generated by quicktype (using dataclasses) into code that uses attrs and jstruct decorators.

Transform from stdin to stdout
1cat input.py | kcli codegen transform > output.py
2
3# Transform from file to file
4kcli codegen transform input.py output.py
5
6# Transform from file to stdout
7kcli codegen transform input.py
8
9# Disable appending 'Type' to class names
10kcli codegen transform input.py output.py --no-append-type-suffix

Generate

The generate command generates Python code with jstruct from a JSON schema file using quicktype.

Generate Python code with jstruct from a JSON schema
1kcli codegen generate --src=schema.json --out=output.py
2
3# Specify Python version
4kcli codegen generate --src=schema.json --out=output.py --python-version=3.8
5
6# Generate without --just-types
7kcli codegen generate --src=schema.json --out=output.py --just-types=false
8
9# Disable appending 'Type' to class names
10kcli codegen generate --src=schema.json --out=output.py --no-append-type-suffix

Create Tree

The create-tree command generates a Python code tree from a class definition. It’s useful for visualizing the structure of complex nested objects and generating initialization code templates.

Generate a tree for a class
1kcli codegen create-tree --module=karrio.schemas.allied_express.label_request --class-name=LabelRequest
2
3# Generate a tree with a module alias
4kcli codegen create-tree --module=karrio.schemas.allied_express.label_request --class-name=LabelRequest --module-alias=allied

Getting Started

OSS contribution

Guides

Schema Generation

Understanding API Schemas

Schema Files Preparation

For JSON APIs

For XML/SOAP APIs

Creating the Generate Script

For JSON APIs

For XML/SOAP APIs

Running Code Generation

Generated Code Structure

Using Generated Types

Troubleshooting

CLI Tools for Schema Generation

Transform

Generate

Create Tree