📖 Looking for karrio's legacy docs? Visit docs.karrio.io

Schema Generation

Karrio uses strongly-typed data structures to ensure data integrity and improve the developer experience. Schema generation converts a carrier’s API specification into Python data classes that represent the exact structure of requests and responses for the carrier’s API.

Overview

The schema generation process involves:

  1. Prepare Schema Files: Create JSON or XML files with API examples
  2. Configure Generation Script: Set up the generate script with correct parameters
  3. Run Generation: Execute ./bin/run-generate-on [path] to generate Python classes
  4. Use Generated Types: Import and use the generated classes in your provider implementation

Step 1: Prepare Schema Files

Create schema files in the schemas/ directory of your extension based on the carrier’s API documentation.

For JSON APIs

Create JSON files with actual API request/response examples (not JSON Schema definitions):

1schemas/
2├── error_response.json      # Error response format
3├── rate_request.json        # Rate request format
4├── rate_response.json       # Rate response format
5├── shipment_request.json    # Shipment request format
6├── shipment_response.json   # Shipment response format
7├── tracking_request.json    # Tracking request format
8└── tracking_response.json   # Tracking response format

Example rate_request.json:

1{
2  "shipper": {
3    "addressLine1": "123 Main St",
4    "city": "New York",
5    "postalCode": "10001",
6    "countryCode": "US"
7  },
8  "recipient": {
9    "addressLine1": "456 Oak Ave",
10    "city": "Los Angeles",
11    "postalCode": "90210",
12    "countryCode": "US"
13  },
14  "packages": [
15    {
16      "weight": 5.0,
17      "weightUnit": "KG",
18      "dimensions": {
19        "length": 10.0,
20        "width": 10.0,
21        "height": 10.0,
22        "unit": "CM"
23      }
24    }
25  ]
26}

For XML/SOAP APIs

Create XSD schema files or XML examples:

1schemas/
2├── error_response.xsd
3├── rate_request.xsd
4├── rate_response.xsd
5└── ...

Note: Use actual API samples from the carrier’s documentation, not abstract schema definitions.

Step 2: Configure the Generate Script

The CLI automatically creates a generate script in your extension root. You need to configure it with the correct parameters for your carrier’s API.

Understanding Generation Parameters

The key parameters that affect how Python classes are generated:

ParameterUsageAPI Field FormatExample
--nice-property-namesConvert to snake_caseAPIs with camelCase fieldsEasyship
--no-nice-property-namesPreserve original namingAPIs with camelCase that should stay camelCaseUPS, FedEx, SEKO
--no-append-type-suffix --no-nice-property-namesFor PascalCase APIsAPIs with PascalCase fieldsSome enterprise APIs

Example Generate Script

File: modules/connectors/[carrier_name]/generate

1#!/bin/bash
2set -e
3
4SCHEMAS=./schemas
5LIB_MODULES=./karrio/schemas/[carrier_name]
6ROOT="$(pwd)"
7
8# Clean existing generated files
9find "${LIB_MODULES}" -name "*.py" -exec rm -r {} \;
10touch "${LIB_MODULES}/__init__.py"
11
12# Generation function
13generate_schema() {
14    echo "Generating $1..."
15    "${ROOT}/bin/cli" codegen generate "$1" "$2" --no-nice-property-names
16}
17
18# Generate all required schemas
19generate_schema "${SCHEMAS}/error_response.json" "${LIB_MODULES}/error_response.py"
20generate_schema "${SCHEMAS}/rate_request.json" "${LIB_MODULES}/rate_request.py"
21generate_schema "${SCHEMAS}/rate_response.json" "${LIB_MODULES}/rate_response.py"
22generate_schema "${SCHEMAS}/shipment_request.json" "${LIB_MODULES}/shipment_request.py"
23generate_schema "${SCHEMAS}/shipment_response.json" "${LIB_MODULES}/shipment_response.py"
24generate_schema "${SCHEMAS}/tracking_request.json" "${LIB_MODULES}/tracking_request.py"
25generate_schema "${SCHEMAS}/tracking_response.json" "${LIB_MODULES}/tracking_response.py"

Choose Correct Parameters

For camelCase APIs (UPS, FedEx, SEKO):

1"${ROOT}/bin/cli" codegen generate "$1" "$2" --no-nice-property-names

For snake_case APIs (Easyship):

1"${ROOT}/bin/cli" codegen generate "$1" "$2" --nice-property-names

For PascalCase APIs:

1"${ROOT}/bin/cli" codegen generate "$1" "$2" --no-append-type-suffix --no-nice-property-names

🚨 CRITICAL RULES BEFORE GENERATION:

  1. Generated Files Are READ-ONLY:

    • NEVER edit files in karrio/schemas/[carrier_name]/
    • Generated files are completely overwritten on each run
    • Any manual changes will be lost permanently

  2. Source-Only Editing:

    • Only modify files in schemas/ directory (your source files)
    • To change API structure: Edit source schemas → regenerate
    • Never create schema files manually outside the CLI

  3. Generation Must Succeed:

    • If generation fails, STOP IMMEDIATELY
    • Fix the source schema files or CLI parameters
    • Never proceed with incomplete/broken generation

  4. Verification Required:

    • Always test imports after generation
    • Verify class names match expectations
    • Ensure type annotations are correct

Step 3: Run Schema Generation

Execute the generation command from the project root:

Activate environment first (CRITICAL)
1source ./bin/activate-env
2
3# Generate schemas for your carrier
4./bin/run-generate-on modules/connectors/[carrier_name]
5
6# For hub carriers
7./bin/run-generate-on community/plugins/[carrier_name]
8
9# Make the script executable if needed
10chmod +x modules/connectors/[carrier_name]/generate

This command:

  1. Finds the generate script in your extension
  2. Executes it to process all schema files
  3. Creates Python classes in karrio/schemas/[carrier_name]/

Step 4: Verify Generated Code

After successful generation, verify the output:

Check generated files
1ls -la modules/connectors/[carrier_name]/karrio/schemas/[carrier_name]/
2
3# Test imports
4python -c "
5import karrio.schemas.[carrier_name].rate_request as req
6import karrio.schemas.[carrier_name].rate_response as res
7print('Schema imports successful')
8"

Generated Code Structure

The generated Python classes use attrs and jstruct decorators for type safety and serialization:

Example Generated Class:

karrio/schemas/[carrier_name]/rate_request.py
1import attr
2import jstruct
3import typing
4
5@attr.s(auto_attribs=True)
6class Address:
7    addressLine1: typing.Optional[str] = None
8    city: typing.Optional[str] = None
9    postalCode: typing.Optional[str] = None
10    countryCode: typing.Optional[str] = None
11
12@attr.s(auto_attribs=True)
13class Package:
14    weight: typing.Optional[float] = None
15    weightUnit: typing.Optional[str] = None
16    dimensions: typing.Optional[Dimensions] = None
17
18@attr.s(auto_attribs=True)
19class RateRequestType:
20    shipper: typing.Optional[Address] = None
21    recipient: typing.Optional[Address] = None
22    packages: typing.Optional[typing.List[Package]] = jstruct.JList[Package]

Class Naming Conventions:

  • With default settings: Classes have Type suffix (e.g., RateRequestType)
  • With --no-append-type-suffix: Classes named exactly as in schema (e.g., RateRequest)

Step 5: Using Generated Types

Import and use the generated classes in your provider implementation:

In Rate Implementation:

karrio/providers/[carrier_name]/rate.py
1import karrio.schemas.[carrier_name].rate_request as carrier_req
2import karrio.schemas.[carrier_name].rate_response as carrier_res
3
4def rate_request(payload: models.RateRequest, settings: Settings) -> lib.Serializable:
5    # Use generated request type
6    request = carrier_req.RateRequestType(
7        shipper=carrier_req.Address(
8            addressLine1=payload.shipper.address_line1,
9            city=payload.shipper.city,
10            # ... map other fields
11        ),
12        recipient=carrier_req.Address(
13            addressLine1=payload.recipient.address_line1,
14            city=payload.recipient.city,
15            # ... map other fields
16        ),
17        packages=[
18            carrier_req.Package(
19                weight=package.weight.value,
20                weightUnit=package.weight.unit,
21                # ... map other fields
22            )
23            for package in packages
24        ]
25    )
26
27    return lib.Serializable(request, lib.to_dict)
28
29def parse_rate_response(response: lib.Deserializable, settings: Settings):
30    data = response.deserialize()
31
32    # Use generated response type
33    rate_response = lib.to_object(carrier_res.RateResponseType, data)
34
35    # Extract rates using typed access
36    rates = [
37        models.RateDetails(
38            carrier_id=settings.carrier_id,
39            service=rate.serviceCode,
40            total_charge=lib.to_money(rate.totalCharge),
41            currency=rate.currency,
42        )
43        for rate in (rate_response.rates or [])
44    ]
45
46    return rates, []

Advanced CLI Tools

Create Tree Command

Generate initialization templates for complex nested objects:

Generate a tree structure for a class
1kcli codegen tree --module=karrio.schemas.[carrier_name].rate_request --class-name=RateRequestType --module-alias=carrier
2
3# Output example:
4carrier.RateRequestType(
5    shipper=carrier.Address(
6        addressLine1=None,
7        city=None,
8        postalCode=None,
9        countryCode=None,
10    ),
11    recipient=carrier.Address(
12        addressLine1=None,
13        city=None,
14        postalCode=None,
15        countryCode=None,
16    ),
17    packages=[
18        carrier.Package(
19            weight=None,
20            weightUnit=None,
21            dimensions=carrier.Dimensions(
22                length=None,
23                width=None,
24                height=None,
25                unit=None,
26            ),
27        )
28    ],
29)

This output can be copied directly into your mapping functions as a starting template.

Troubleshooting

Common Issues

Generation Fails:

  • Ensure schema files contain valid JSON/XML
  • Check that generate script is executable: chmod +x generate
  • Verify environment is activated: source ./bin/activate-env

Import Errors:

  • Verify extension is installed: pip install -e modules/connectors/[carrier_name]
  • Check __init__.py files exist in schema directories
  • Ensure class names match generated output

Wrong Field Names:

  • Review generation parameters in your generate script
  • Use --nice-property-names for snake_case conversion
  • Use --no-nice-property-names to preserve original naming

Testing Generation

Test your generation setup:

Test CLI command directly
1./bin/cli codegen generate schemas/rate_request.json test_output.py --no-nice-property-names
2
3# Check output
4cat test_output.py

Best Practices

  1. Use Real API Examples: Create schema files from actual carrier API documentation
  2. Choose Correct Parameters: Match generation parameters to your API’s field format
  3. Test Imports Early: Verify generated classes import correctly before implementing logic
  4. Use Generated Types: Always import and use generated classes in provider implementations
  5. Regenerate When Needed: Only modify source schema files, never edit generated Python files

XML API Considerations

For XML/SOAP APIs, the process is similar but uses generateDS instead:

XML generation in generate script
1generateDS --no-namespace-defs -o "${LIB_MODULES}/rate_request.py" $SCHEMAS/rate_request.xsd

Generated XML classes have different patterns:

  • No Type suffix by default
  • Use lib.to_element for deserialization
  • Use request.serialize() for XML serialization

Testing Generated Types

MANDATORY: Validate your generated types work correctly in tests:

1. Import Verification Test

1def test_schema_imports(self):
2    """Verify all generated schema types import correctly."""
3    try:
4        import karrio.schemas.[carrier_name].rate_request as req
5        import karrio.schemas.[carrier_name].rate_response as res
6        print("All schema imports successful")
7        self.assertTrue(hasattr(req, 'RateRequestType'))  # Adjust for your naming
8        self.assertTrue(hasattr(res, 'RateResponseType'))
9    except ImportError as e:
10        self.fail(f"Schema import failed: {e}")

2. Type Validation Test

1def test_generated_types_usage(self):
2    """Verify generated types work in mapping functions."""
3    request = gateway.mapper.create_rate_request(self.RateRequest)
4
5    # Verify the request uses generated types correctly
6    serialized = request.serialize()
7    self.assertIsInstance(serialized, dict)  # For JSON APIs
8
9    # Verify specific required fields exist
10    self.assertIn('shipper', serialized)
11    self.assertIn('recipient', serialized)
12    self.assertIn('packages', serialized)

3. Schema Regeneration Test

Always verify that schema regeneration still works:

Test regeneration doesn't break
1./bin/run-generate-on modules/connectors/[carrier_name]
2python -c "import karrio.schemas.[carrier_name] as schemas; print('Regeneration successful')"

Next Steps

Once schema generation is complete:

  1. Configure Metadata: Set up connection settings and data units
  2. Implement API Requests: Create the HTTP communication layer
  3. Implement Data Mapping: Transform between Karrio and carrier formats
  4. Write Tests: Validate your integration works correctly with exact patterns