Local Development

Welcome to the Karrio development guide! This comprehensive resource will help you set up your local environment to contribute to Karrio or build custom integrations. Whether you’re adding a new carrier integration, enhancing the API, or improving the dashboard, this guide has you covered.

Architecture Overview

Karrio is built with a powerful and flexible microservices architecture, consisting of several components:

• API Server: Django REST Framework application that provides the core API
• Dashboard: React application for the user interface
• Worker: Huey workers for background tasks
• Database: PostgreSQL for data storage (SQLite supported for development)
• Cache: Redis for caching and as a message broker

Prerequisites

Before you begin, make sure you have the following installed:

• Git
• Node.js v16.15.1+
• Python 3.8+
• Docker & Docker Compose (optional, for Docker-based setup)

Manual Setup (Recommended)

Follow these steps to set up your development environment manually:

1. Clone the Repository

First, fork the Karrio repository on GitHub, then clone your fork:

1git clone https://github.com/[YOUR_USERNAME]/karrio.git
2cd karrio

2. Install System Dependencies

Install the required system dependencies:

1./bin/install-binaries

If you encounter permission issues:

1chmod +x ./bin/*

3. Set up Python Environment

Create and configure a Python virtual environment with all dependencies:

1source ./bin/setup-server-env

This command will:

• Create a new Python virtual environment
• Install server development requirements
• Install insiders development requirements (if available)
• Install platform development requirements (if available)
• Install Node.js dependencies
• Set up logging directories

4. Apply Database Migrations

By default, Karrio uses SQLite for development to simplify setup:

1karrio migrate

1- DATABASE_ENGINE=sqlite3
2- # DATABASE_ENGINE=postgresql_psycopg2
3- # DATABASE_USERNAME=postgres
4- # DATABASE_PASSWORD=postgres
5- # DATABASE_NAME=db
6+ # DATABASE_ENGINE=sqlite3
7+ DATABASE_ENGINE=postgresql_psycopg2
8+ DATABASE_USERNAME=postgres
9+ DATABASE_PASSWORD=postgres
10+ DATABASE_NAME=db

1docker compose -f docker/docker-compose.yml up -d db

5. Collect Static Assets

Gather static files for the web interface:

1karrio collectstatic --noinput

6. Create Admin User

Create your admin account:

1karrio createsuperuser

7. Start the Server

Launch the Karrio server:

1./bin/start

Docker-based Setup (Alternative)

If you prefer using Docker for development, here’s how to set it up:

Quick Start with Docker

Clone the repository
1git clone https://github.com/[YOUR_USERNAME]/karrio.git
2cd karrio
3
4# Create and setup the Docker development environment
5./bin/docker-env create
6
7# Start the Karrio server
8./bin/docker-env exec './bin/start-server'

Once running, Karrio will be available at http://localhost:5002

For more Docker commands and options:

1./bin/docker-env

This will display a help menu with all available commands like:

1Help: You can pass any the following commands to the './bin/docker-env' scripts
2-----
3create     Create and setup docker dev environment.
4destroy    Destroy docker dev environment.
5on         Start docker dev environment.
6off        Stop docker dev environment.
7shell      Start docker dev environment shell.
8exec       Execute command in docker dev environment.

Example Docker Commands

Run server tests:

1./bin/docker-env exec './bin/run-server-tests'

Run tests for a specific carrier:

1./bin/docker-env exec 'python -m unittest discover -v -f modules/connectors/dhl_express/tests'

Setting Up the Dashboard

The Karrio dashboard is now part of the monorepo structure. To set it up:

1. Install Node Dependencies

At the root of the repository:

1npm i

2. Configure Environment

Copy the sample environment file:

1cp apps/dashboard/.env.sample apps/dashboard/.env

3. Start the Dashboard

Launch the development server:

1npm run dev

Development Workflows

Karrio provides specialized workflows for different development tasks.

Working on the SDK and Carrier Extensions

Set up the SDK environment:

1cd karrio
2source ./bin/setup-sdk-env

Run all SDK tests:

1./bin/run-sdk-tests

Working on a Single Carrier

Create a Python environment with the necessary dependencies:

1cd karrio
2./bin/create-new-env

Install the extension in development mode:

Install the core SDK
1pip install -e modules/sdk
2
3# Install a specific carrier extension
4pip install -e modules/connectors/fedex

Run tests for a specific carrier:

Format:
1# python -m unittest discover -v -f modules/connectors/[carrier_extension]/tests
2python -m unittest discover -v -f modules/connectors/fedex/tests

Working on the Server

Set up the server environment:

1cd karrio
2source ./bin/setup-server-env

Run all server tests:

1./bin/run-server-tests

Run tests for a specific module:

Format:
1# karrio test --failfast karrio.server.[module].tests
2karrio test --failfast karrio.server.manager.tests

Database Management

Reset Database

To completely reset your database for a fresh installation:

Stop and remove the database container
1docker compose -f docker/docker-compose.yml down
2
3# Start a fresh PostgreSQL container
4docker compose -f docker/docker-compose.yml up -d db

Backup Data

Create a backup of your current database:

1karrio dumpdata --natural-foreign --natural-primary -o [BACKUP_PATH]/data[VERSION].json
2# Example:
3# karrio dumpdata --natural-foreign --natural-primary -o ../backup/data2024.1.json

Load Development Data

Restore data from a backup:

1karrio loaddata [BACKUP_PATH]/data[VERSION].json
2# Example:
3# karrio loaddata ../backup/data2024.1.json

Code Organization

The Karrio codebase is organized as follows:

• /apps: Web applications and interfaces
  • /apps/api: Django REST API server for external integrations
  • /apps/dashboard: React-based admin dashboard UI
  • /apps/web: Next.js documentation and marketing website
  • /apps/www: Legacy website components
• /modules: Core functionality and services
  • /modules/sdk: Core shipping SDK that powers all integrations
  • /modules/core/: Core shared functionality
  • /modules/manager/: Account and organization management
  • /modules/documents/: Document generation services
  • /modules/events/: Event processing and webhooks
  • /modules/orders/: Order management and fulfillment
  • /modules/graph/: GraphQL API implementation
• /bin: Helper scripts for development, testing, and deployment
• /docker: Docker configurations and container definitions
• /schemas: API schema definitions and validations
• /ee: Enterprise Edition extensions (when applicable)

Adding a New Carrier

One of the most common contributions is adding support for a new shipping carrier:

1. Create a new carrier module in /modules/connectors/
2. Implement the required interfaces:
   • CarrierInterface: Base interface for all carriers
   • ShippingInterface: For shipping operations
   • TrackingInterface: For tracking operations
   • RateInterface: For rate calculation
3. Add carrier-specific settings
4. Add comprehensive tests
5. Update documentation

For a detailed guide, see Adding a New Carrier.

Troubleshooting

Common Issues

1chmod +x ./bin/*

1karrio collectstatic --noinput

1docker compose -f docker/docker-compose.yml ps