
# Why should we care about MCP?

For the same reasons you might want to use microservices or a backend-for-frontend architecture; to centralize logic for maintainability, easier scaling, finer grained control of utilization and monitoring. Eventually simplified client libraries and higher level tooling to speed up integration. A little more specifically: 

## Some reasons to centralize tools outside of the LLM's you are building

- you can reuse tooling and data collection for future agents
- when tooling changes, you can maintain a single place (no need to update all your agents)

## Some reasons to centralize tools in an accepted convention (industry standard MCP)

- simplify developer experience
- reduce cognitive load when connecting to other Toolboxes
- new offerings are easy to 'sign up' to, if they use the same standard that your agents already use

# Serving tool definitions to LLMs from the MCP server (tool capabilities, data stores, and  prompts) 

> instead of each client app having to define its own JSON tool schemas for each model it supports, the MCP server acts as a central registry that provides tool definitions, prompts, and resource descriptions to whatever agent or LLM client connects to it at runtime.

## Without MCP (using a function call directly within an LLM)

- Define model-specific schemas: JSON descriptions of the function, acceptable parameters, and expected response format
- Implement handlers for the functions
- Create different implementations for each model you support (they have different tool tags in Claude vs ChatGPT etc)

## Using MCP:

- You are not bound to the integrations that only ChatGPT or Claude support (using their tool calling)
- Ability to discover tools by the agent itself at runtime
- no need to build out json tool descriptions in each client app
- contribute to a universal format where any AI app can use any tool without custom integration code

![MCP high level](//images.ctfassets.net/kot48ke411n3/7GFjpaOyOTBwbhzdTPnU8t/99675fa02c38a7532140af45bdb8ef81/Screenshot_2025-11-18_at_13.02.07.png)
source: https://www.descope.com/learn/post/mcp

From a high level, this architecture is broken down into 
1. host (the process running the LLM, or the application itself)
2. client (the tool used to connect to the MCP server. Uses [JSON-RPC 2.0](https://www.jsonrpc.org/specification "Link to JSON RPC 2") 
3. MCP server. Collection of prompts for both tools and resources. API's and databases
4. communication (json rpc2)

# Sequence Diagram Flow

![Sequence Diagram of MCP](//images.ctfassets.net/kot48ke411n3/01xOhkcj2NVnvaFWtdrfp6/0f66bbded65cb1e39c059cb31cb30795/Screenshot_2025-11-18_at_13.07.28.png)
source: Claude

1. Need recognition: Claude analyzes the question and recognizes it needs external, real-time information (not available during training)
2. Tool selection: Claude identifies it needs MCP capability to fulfill the request.
3. Permission request: The client could displays a permission prompt to a user
4. Information exchange: Once approved, the client sends a request to the appropriate MCP server using the MCP protocol
5. External processing: The MCP server processes the request (api call, db query etc)
6. Results return in a standardized format.
7. Context integration: Claude incorporates the response into context
8. Response generation: Claude generates an up to date response

This should happen seamlessly, creating a streamlined experience where Claude can access live data / private / custom tools. ie: not from training data

## client features

In addition to making use of context provided by servers, clients may provide several features to servers. These client features allow server authors to build richer interactions.

## Feature	Explanation	Example

### Sampling	

| Description     | Example     |
| ---------- | ---------- |
|        |        |
| Sampling allows servers to *request LLM completions* through the client, enabling an agentic workflow. This approach puts the client in complete control of user permissions and security measures.	       | A server for booking travel may send a list of flights to an LLM and request that the LLM pick the best flight for the user.       |

### Roots	

Roots allow clients to specify which *directories* servers should focus on, communicating intended scope through a coordination mechanism.	A server for booking travel may be given access to a specific directory, from which it can read a user’s calendar.
### Elicitation	

Elicitation enables servers to *request specific information* from users during interactions, providing a structured way for servers to gather information on demand.	A server booking travel may ask for the user’s preferences on airplane seats, room type or their contact number to finalise a booking.

# Tooling

check out tooling here: [descope](https://www.descope.com/learn/post/mcp "A link to the article with the actual links")

You can also make use of existing off the shelf MCP tools as references:

- [Fetch](https://github.com/modelcontextprotocol/servers/tree/main/src/fetch?_gl=1*1agfoeg*_gcl_au*NzA4Mjk5ODczLjE3NjM0NjQzNTA.)
- [filesystem](https://github.com/modelcontextprotocol/servers/tree/main/src/filesystem?_gl=1*lo8lhj*_gcl_au*NzA4Mjk5ODczLjE3NjM0NjQzNTA.)
- [appify](https://github.com/apify/apify-mcp-server?_gl=1*13igf4*_gcl_au*NzA4Mjk5ODczLjE3NjM0NjQzNTA. "very cool, lots of tools in here that you can use (your LLM can use)")

# Serious Security Considerations

### Always treat these tools as idiots that will delete you database / code by mistake, easily. Backup before using!!!

[replit](https://fortune.com/2025/07/23/ai-coding-tool-replit-wiped-database-called-it-a-catastrophic-failure/ "replit AI agent deleted their database")

This is why is it vital to keep HUMANS IN THE LOOP
Clients must request explicit permissions, and users must understand the implications 

Also: [Findings reveal a significant number of internet-exposed MCP servers operating in production environments, many lacking authentication or adequate safeguards. These servers respond to handshake requests and expose internal tools without validating the requester’s identity. Effectively, they publicly broadcast their capabilities.](https://www.knostic.ai/blog/mapping-mcp-servers-study "Reasons to use authentication in MCP servers")

MCP Architecture for Agentic Systems

# High Level Setup

Spilo runs PostgreSQL with Patroni, which manages the cluster’s leader and replicas. The leader accepts writes. Replicas follow it by replaying PostgreSQL’s write-ahead log (WAL), and can usually serve read-only queries. If the leader fails, Patroni can promote a healthy replica so the database can continue operating.

The important question is **where those instances run**. Two database pods on one machine provide little protection from a machine failure. Placing them on separate Kubernetes nodes protects against that failure; placing them in separate availability zones can protect against a wider outage. Replication improves availability, but it is not a backup: an accidental deletion or bad migration is replicated too.

An *edge* setup is simply one where the database runs close to the application or users, perhaps at a site with an unreliable connection to a central cloud. Spilo works there, but it is equally useful in a conventional data center or cloud cluster. At an edge site, it is especially important to ask which resources share a failure domain: the database, its replicas, the Kubernetes control plane, and the backup storage.

Kubernetes can include nodes in different zones or regions. For example, a K3s worker can be joined with the following command, after providing the server address and join token:

```sh
curl -sfL https://get.k3s.io | \
  K3S_URL=https://SERVER_ADDRESS:6443 \
  K3S_TOKEN=JOIN_TOKEN sh -
```

Joining a node does not automatically place a replica there. Node labels and pod placement rules determine where database instances run, while the storage and network must support that placement. Distance also adds latency, which matters particularly for replication and failover. [Kubernetes describes scheduling across failure zones](https://kubernetes.io/docs/setup/best-practices/multiple-zones/).

## Backups

Spilo can use WAL-G to take periodic **base backups** and archive completed WAL segments to object storage. The base backup provides a starting copy of the database; the WAL records subsequent changes. A restore needs both a usable base backup and the WAL sequence leading to the desired recovery point. [PostgreSQL calls this point-in-time recovery](https://www.postgresql.org/docs/16/continuous-archiving.html).

The backup schedule and destination are configuration choices. One design sends backups to in-cluster object storage such as MinIO, then copies them to another region or cloud provider. That can be useful when an edge site temporarily loses its internet connection. Another design sends WAL-G backups directly to remote object storage.

If local object storage is used, it should at least be on a different node from the database. A copy in another site or region protects against losing the entire local cluster. The copies also need appropriate retention and regular restore tests; seeing backup files in a bucket does not establish that they can be restored.

## Recovery

For a failed database node, the quickest path is often promotion of a replica. Applications reconnect to the new leader, and the failed instance can later be rebuilt as a replica. With asynchronous replication, the newest transactions might not have reached the promoted replica. [Patroni explains its replication modes and failover behavior](https://patroni.readthedocs.io/en/latest/replication_modes.html).

Backups handle a different class of problem: lost storage, replicated mistakes, or an outage that takes down the whole site. Recovery involves starting a new cluster from a base backup and replaying archived WAL to a chosen point. If backups were first stored locally and then copied off-site, a disaster recovery procedure could copy the archive into replacement storage before restoring it. The exact path matters less than proving that the full procedure works.

## Replicas in Another Region

Spilo can support regional redundancy. One option is to run replicas in a Kubernetes cluster whose nodes span regions. This requires suitable pod placement, network connectivity, and storage in each region. It also ties those nodes to the same Kubernetes control plane, so a remote replica alone does not make the whole system region-independent.

Another option is a **separate standby cluster** in the second region. Patroni supports a standby leader that follows a remote primary, with further replicas following that standby leader. The Zalando Postgres Operator can configure this using a connection to the primary or an S3/GCS WAL archive. If the first region is lost, operators can deliberately promote the standby and redirect applications to it. That decision must account for replication lag and ensure the old primary cannot also accept writes. [Patroni](https://patroni.readthedocs.io/en/latest/standby_cluster.html) and the [Postgres Operator](https://postgres-operator.readthedocs.io/en/latest/user/#setting-up-a-standby-cluster) document this pattern.

## Drawbacks

- Every replica needs storage for another copy of the database.
- Cross-region replication uses bandwidth and introduces delay. Requiring a remote replica to confirm every write can increase application latency.
- Failover can lose recent transactions if replication is asynchronous.
- A local object store may fail alongside the database if both depend on the same site.
- Remote failover needs a plan for promotion, application routing, and preventing two writable leaders.

## Benefits

- Replicas on separate nodes can shorten recovery from a node failure.
- Placement across zones reduces reliance on one data center.
- A standby cluster can keep a recent database copy ready in another region.
- Off-site base backups and WAL allow recovery to an earlier point after replicated mistakes.
- Kubernetes provides a way to add nodes and express placement rules as the system grows.

Backups for a database running at the edge

# How Spilo and Patroni Configure PostgreSQL and WAL-G

Last repository review: 2026-10-04.

## Scope

This article describes an edge deployment where Spilo archives PostgreSQL
WAL and base backups to a local S3-compatible MinIO service. Keeping the first
copy local allows backups to continue when the connection to a cloud
provider is unreliable. The local archive can then be copied or replicated to
durable off-site storage when connectivity is available.

Local MinIO is not automatically an off-site backup. If PostgreSQL and MinIO
share a machine, disk, site, power source, or administrator, one incident can
destroy both the database and its backup. So keep in mind, local duplication and multiple local nodes is very important.

Spilo packages PostgreSQL, Patroni, and backup/restore tooling such as WAL-G (WAL-E is now legacy)
. Exact paths and generated commands vary by
Spilo version, so inspect the running pod before relying on any example here.

## Glossary

- **PostgreSQL** is the database that stores the application’s durable data. It writes every change to its Write-Ahead Log before updating the main database files.

- **Spilo** is a container image that packages PostgreSQL with high-availability and backup tooling. It converts environment variables into the runtime configuration used by PostgreSQL, Patroni, and WAL-G.

- **Patroni** manages PostgreSQL availability. It decides which instance is the primary, maintains replicas, and coordinates controlled failover when the primary becomes unavailable.

- **WAL (Write-Ahead Log)** is PostgreSQL’s stream of database changes. Archived WAL files can be replayed after a base backup to restore the database to a particular point in time.

- **WAL-G** uploads PostgreSQL base backups and WAL files to object storage and retrieves them during recovery. In this edge design, it writes to an S3-compatible MinIO service.

- **MinIO** provides S3-compatible object storage close to the edge database. It allows backups to continue during cloud connectivity problems, though those backups should later be copied to another machine, site, or cloud bucket.

- **Zalando Postgres Operator** converts a Kubernetes PostgreSQL resource into StatefulSets, Services, Secrets, and Spilo configuration. It also continually reconciles the running cluster with the declared configuration.

- **Kubernetes** runs and supervises the database containers. It provides storage, networking, Secrets, configuration sources, and replacement of failed pods.

- **A base backup** is a physical snapshot of the PostgreSQL data directory. Recovery begins with a base backup and then replays archived WAL to reach the desired recovery time.

- **Point-in-time recovery (PITR)** combines a base backup with an uninterrupted WAL archive. It allows PostgreSQL to be restored to a selected moment rather than only to the time of the last base backup.

## Configuration flow

In a Zalando Postgres Operator deployment, configuration begins before the
Spilo container starts:

```text
Postgresql custom resource
        |
        v
Zalando Postgres Operator
  - reads structured fields such as clone: and postgresql.parameters
  - combines operator defaults and explicitly supplied pod environment
  - renders a StatefulSet, Services and credential Secrets
        |
        v
Kubernetes starts the Spilo container with its final environment
        |
        v
Spilo initialization (/launch.sh and configure_spilo.py)
  |-- generates Patroni configuration (/run/etc/patroni.yml)
  |-- generates WAL-G envdir files (/run/etc/wal-e.d/env/)
  |-- configures PostgreSQL archiving and recovery
  `-- installs the scheduled base-backup cron job
        |
        v
Patroni starts and manages PostgreSQL
        |
        v
WAL-G archives completed WAL and creates scheduled base backups
```

The container environment may be assembled from the PostgreSQL custom
resource, operator-wide configuration, ConfigMaps, Secrets, and Kubernetes
metadata. Spilo does not retain the origin of each value. At startup it sees
only the final process environment.

## What happens at startup

### 1. The operator renders the pod template

The operator converts structured custom-resource fields into a StatefulSet.
For example, it can translate a `clone:` block into `CLONE_*` environment
variables expected by Spilo. Values in `spec.env` are already low-level Spilo
inputs and are copied into the container environment.

This distinction establishes ownership:

- use structured operator fields where the custom resource provides them;
- use `spec.env` for non-overlapping Spilo settings;
- use Secrets for credentials and mount or reference them deliberately;
- do not manually recreate environment variables that the operator generates.

There is no useful general rule that "the CR always wins" or "the ConfigMap is
read later." Kubernetes constructs one environment before starting the
container. Explicit `env` entries override values imported through `envFrom`
when names overlap, but duplicate names in an `env` list remain ambiguous and
must not be used as a configuration strategy. Controllers may also compare the
ordered list while reconciling the StatefulSet.

### 2. Spilo generates runtime configuration

Spilo's startup scripts consume the recognized environment variables, validate
the configuration, and generate files used by Patroni, PostgreSQL, and WAL-G.
A simplified Patroni example looks like this:

```yaml
# Conceptual example only; inspect the running Spilo version for exact output.
scope: acid-minimal-cluster
name: pod-name

postgresql:
  listen: 0.0.0.0:5432
  connect_address: pod-ip:5432
  data_dir: /home/postgres/pgdata/pgroot/data

  parameters:
    archive_mode: "on"
    archive_timeout: 60s
    wal_level: replica
    archive_command: "<Spilo-generated WAL-G command>"

  recovery_conf:
    restore_command: "<Spilo-generated WAL-G command>"

  create_replica_methods:
    - basebackup_fast_xlog

  basebackup_fast_xlog:
    command: /scripts/basebackup.sh
    retries: 2
```

The base-backup schedule is not a Patroni backup schedule. Spilo installs it in
the pod's crontab.

### 3. Spilo generates the WAL-G envdir

For the edge MinIO deployment, `/run/etc/wal-e.d/env/` can contain files such
as:

```text
/run/etc/wal-e.d/env/
|-- AWS_ACCESS_KEY_ID
|-- AWS_SECRET_ACCESS_KEY
|-- AWS_ENDPOINT
|-- AWS_REGION
|-- AWS_S3_FORCE_PATH_STYLE
|-- WALG_DISABLE_S3_SSE
|-- WALG_S3_PREFIX
`-- WALE_S3_PREFIX
```

The directory name and `WALE_*` compatibility variables are historical. Their
presence does not by itself prove that the WAL-E binary is handling backups.

`WAL_S3_BUCKET` is an input consumed by Spilo. Spilo combines it with the
cluster scope and other settings to generate a final prefix such as
`WALG_S3_PREFIX`, which WAL-G consumes. Credentials should come from a Secret;
do not put their values in documentation or print them during routine checks.

### 4. PostgreSQL archives completed WAL

With WAL-G backup support enabled, Spilo generates an `archive_command` that
ultimately invokes WAL-G to upload a completed WAL segment. A simplified form
is:

```sql
archive_command = 'envdir /run/etc/wal-e.d/env wal-g wal-push %p'
```

- `envdir` exposes the generated WAL-G configuration.
- `wal-g wal-push` uploads the completed segment.
- PostgreSQL substitutes `%p` with the segment's local path.

The installed Spilo release may use a wrapper around this command. Treat the
result of `SHOW archive_command` as runtime truth.

If backup support is disabled, Spilo may generate a no-op archive command such
as `/bin/true`, or archiving may be disabled another way. Check the live
PostgreSQL settings rather than assuming a particular fallback.

### 5. Cron starts base backups

When backups and `BACKUP_SCHEDULE` are enabled, Spilo installs a cron entry. A
typical entry uses Spilo's wrapper:

```cron
15 2 * * * envdir "/run/etc/wal-e.d/env" /scripts/postgres_backup.sh "/home/postgres/pgdata/pgroot/data"
```

The wrapper invokes WAL-G to create a physical base backup and applies the
retention behavior implemented by that Spilo version. Inspect `crontab -l` and
the wrapper before assuming that `BACKUP_NUM_TO_RETAIN` means a count; some
versions interpret it as an age in days.

Base backups and WAL archiving serve different purposes:

- a base backup provides the starting filesystem image;
- archived WAL supplies the changes after that image and enables point-in-time
  recovery;
- a recoverable chain requires both a usable base backup and continuous WAL.

### 6. Patroni manages PostgreSQL

Patroni reads the generated configuration, starts PostgreSQL, manages leader
election and replication, and applies supported dynamic configuration changes.
It is not responsible for the object-store durability of WAL-G backups.

## Why `WAL_S3_BUCKET` can appear not to work

Spilo cannot use a ConfigMap that was never injected into its pod, and it does
not know that an environment value originally came from a ConfigMap. If
`WAL_S3_BUCKET` appears ineffective, check the entire path:

1. Is the ConfigMap actually referenced by the rendered StatefulSet?
2. Does the running container contain the expected variable name?
3. Is another explicit `env` entry using the same name?
4. Is `USE_WALG_BACKUP` enabled?
5. Did the pod restart after its environment-bearing pod template changed?
6. Did Spilo generate the expected `WALG_S3_PREFIX`?
7. What do `SHOW archive_command` and `crontab -l` report?

Defining the bucket directly in the PostgreSQL custom resource is explicit and
keeps the setting close to the cluster it controls:

```yaml
apiVersion: acid.zalan.do/v1
kind: postgresql
metadata:
  name: acid-minimal-cluster
spec:
  env:
    - name: USE_WALG_BACKUP
      value: "true"
    - name: WAL_S3_BUCKET
      value: postgresql
```

This does not mean every value must live in `spec.env`. Credentials belong in a
Secret, and values generated from structured fields such as `clone:` should
remain owned by the operator.

## Safe runtime verification

Confirm the namespace and cluster before running commands. Avoid printing
credential files or the full process environment.

```bash
# Generated Patroni configuration. Review before sharing because deployments
# can embed endpoints or other environment-specific details.
kubectl exec -n default "$POD" -- sed -n '1,240p' /run/etc/patroni.yml

# Show filenames only; do not cat credential files.
kubectl exec -n default "$POD" -- ls -la /run/etc/wal-e.d/env/

# Verify selected non-secret settings.
kubectl exec -n default "$POD" -- sh -c \
  'printf "WALG_S3_PREFIX="; cat /run/etc/wal-e.d/env/WALG_S3_PREFIX; \
   printf "AWS_ENDPOINT="; cat /run/etc/wal-e.d/env/AWS_ENDPOINT'

# PostgreSQL runtime truth.
kubectl exec -n default "$POD" -- su - postgres -c \
  'psql -c "SHOW archive_mode;" -c "SHOW archive_command;" -c "SHOW archive_timeout;"'

# Scheduled base backups.
kubectl exec -n default "$POD" -- crontab -l
```

Object names or live WAL-G configuration must be inspected before claiming a
compression format. Historical `.lzo` objects may have been written by WAL-E
and do not prove that current WAL-G uploads use LZO. If documenting a measured
format, state when and where it was observed.

## Applying configuration changes

Container environment variables are fixed for the lifetime of a container.
Updating a ConfigMap does not change the environment of an already running
process. A changed operator-managed pod template normally results in a
controlled rollout after the PostgreSQL custom resource or operator
configuration is updated.

Some PostgreSQL and Patroni settings can be reloaded dynamically; others need a
PostgreSQL restart or pod replacement. Do not use pod deletion as a generic
configuration mechanism. Preview the operator-owned change, apply it
deliberately, and observe Patroni health throughout the rollout.

## Case study: duplicate clone variables caused a reconciliation loop

The later GCS-backed Horse deployment exposed an important consequence of the
configuration layers described above. Its PostgreSQL custom resource contained
a structured `clone:` block, while `spec.env` also manually declared variables
that the Postgres Operator derived from that block.

The manually duplicated variables were:

```text
CLONE_METHOD
CLONE_WAL_GS_BUCKET
CLONE_GOOGLE_APPLICATION_CREDENTIALS
```

The operator also generated clone method, bucket, credential path, archive
scope, source UID, and recovery target settings. Some Spilo clone method names
retain `WALE` for historical compatibility even when WAL-G performs the actual
transfer, so a name such as `CLONE_WITH_WALE` does not by itself mean that the
legacy WAL-E binary is in use.

### How the effective configuration is prioritized

The important priority is ownership, not just which duplicate happens to be
last:

```text
clone: structured configuration
        |
        `-- Postgres Operator generates CLONE_* environment entries

spec.env manual configuration
        |
        `-- Operator also places these entries in the pod template

rendered StatefulSet
        |
        `-- now contains overlapping or differently rendered entries

Spilo process environment
        |
        `-- one effective value is observed at startup

generated Patroni/WAL-G configuration
        |
        `-- controls clone/restore behavior in PostgreSQL
```

Even if the effective process value happens to be correct, the controller must
still reconcile the declarative pod template. Duplicate names, different
ordering, or conflicting representations can make its desired environment
differ from the environment stored in the existing StatefulSet. Relying on
"last value wins" therefore does not make the resource stable.

### What happened

The Postgres Operator resynchronized the cluster every 30 minutes. On each
pass, it reported:

```text
new statefulset containers's postgres (index 0) environment does not match the current one
```

It treated the StatefulSet as outdated and initiated another rolling update.
Patroni kept the database available by moving roles as necessary, but the pod
replacement and PostgreSQL lifecycle activity were followed by another base
backup. The bucket consequently showed backups about every 30 minutes even
though the configured backup cron ran only every six hours.

An observed sequence on 2026-10-04 was:

```text
06:23 UTC  base backup after reconciliation activity
06:53 UTC  another base backup
07:23 UTC  another base backup
07:53 UTC  another base backup
08:23 UTC  another base backup from the old loop

08:29 UTC  duplicate manual CLONE_* entries removed and corrected CR applied
08:34 UTC  expected final backup associated with the intentional rollout
08:48 UTC  next periodic operator resync started
08:49 UTC  resync completed without mismatch, rollout, failover, or backup
```

The backup cron was not creating the 30-minute cadence. The repeated
reconciliation and rolling updates were the trigger.

### The fix and operational rule

The manual `CLONE_METHOD`, `CLONE_WAL_GS_BUCKET`, and
`CLONE_GOOGLE_APPLICATION_CREDENTIALS` entries were removed. The operator now
derives them solely from `clone:`. Non-overlapping Spilo controls such as
`USE_WALG_RESTORE` and `CLONE_USE_WALG_RESTORE` remain explicit where needed.

The general rule is:

> Give every generated environment variable one owner. Prefer structured
> operator fields, use explicit environment variables only for settings the
> operator does not generate, and verify both the rendered StatefulSet and the
> resulting PostgreSQL/WAL-G runtime configuration.

This incident also shows why an unexpected backup frequency should be
correlated with operator logs, StatefulSet revisions, pod creation times, and
Patroni role changes before changing the backup schedule.

## Summary

1. The Postgres Operator renders the Kubernetes resources and final pod
   environment from several configuration sources.
2. Spilo consumes that environment at container startup and generates Patroni,
   PostgreSQL, WAL-G, and cron configuration.
3. PostgreSQL archives completed WAL continuously; cron starts periodic base
   backups.
4. Local MinIO supports an edge deployment during cloud outages, but a durable
   copy must exist in another failure domain.
5. Environment changes require pod replacement, while some Patroni/PostgreSQL
   settings can be changed dynamically.
6. Operator-generated variables must not also be declared manually. Duplicate
   ownership can cause perpetual drift, rolling updates, and expensive
   secondary effects even when the effective value looks correct inside a pod.

A look into database replication at the edge. With Spilo

## Table of Contents

# Nvidia Jetson Edge AI

AI at the edge requires hardware that is very capable at number crunching while being under certain constraints such as reduced power consumption, limited cooling ability, size restrictions and likely network limitations.

the Raspberry Pi has been widely adopted in robotics, but NVIDIA has since brought purpose-built AI hardware to the space. These options come at a premium, but their processing power and power efficiency make them the strongest candidates for production edge AI deployments. These are professional grade hardware SOC boards, that run ARM CPU's and find application across logistics, health and safety, production lines, robotics and healthcare. 
Privacy is equally a big driver for adoption. Medical data is heavily regulated, and keeping inference on-device means raw patient data never leaves the room.

## The Jetson Options

Nvidia offers 3 options, in order of power: 
AGX, NX and Nano.
Processing Power is measured in TOPs (Trillions Operations per Second) and this is calculated by finding the Operations per cycle X Clock frequency (cycles per second) X Number of Cores.
For these variants, the numbers are roughly for AGX: 200 TOPS (32GB) and 275 TOPS (64GB), For NX: 100, and for Orin Nano Super: 67 TOPs. 

Another great benefit of the AGX, and Nvidia, is that they also support dedicated components for VPI (Vision Programming Interface), the OFA (Optical Flow Accelerator): for example for dual camera setup, the depth maps are generated outside of the GPU, freeing up more compute for your AI workloads, whilst you are able to integrate depth into your application. 
Depth can help with occlusions for tracking, avoidance problems and more.

The other two: NX and Nano can still run the VPI but they rely on CUDA and the GPU, so they take some cycles away from the AI model.

[Here is a table breakdown of the differences](https://connecttech.com/jetson/jetson-module-comparison/)

## The Jetson Architecture

The Jetsons run as combined ARM CPU with CUDA GPU's. 
ARM as you know, are most commonly used in mobile and edge devices because they offer efficient compute and are effective for multiple repeated math operations. They don't offer the same peak performance in a single thread as does the x86 systems, but low power and heat is the name of the game in edge AI. 
The main reason why ARM dominates here is due to SOC's. (System on a Chip). Historically ARM sells the designs of their CPU's to other manufacturers, compared to how Intel and AMD design and build their own flavors of x86 chips. This means manufacturers can more easily integrate these CPU's into SOC's which ultimately share the silicon with other components such as custom NPU's, reducing power consumption, latency, and physical footprint while enabling better performance in specialised accelerators.

x86 have started on the SOC road now, with products such as Intel Core Ultra (Meteor Lake) or AMD Ryzen AI chips, which integrate CPU + GPU + NPU on one die for consumer/edge devices.

## Jetson Deployment Options

The AGX has components in the SOC that will offload vision tasks to a dedicated unit. This means the NX and Nano will need to share the GPU when running tools from the Nvidia VPI (Vision Programming Interface).
This doesn't always need to be the case. In certain applications you could use a camera that does the compute for depth maps directly and interface using the USB ports, but this does come with limitations. 
When using the depth maps from a device such as the [OAK-D](https://shop.luxonis.com/products/oak-d?srsltid=AfmBOooQUFGRN_xmMNzQ85gc0scJm9kVxYHlAGUtNo6MiJq9LPOZ-Yjj), you would not be able to make use of AI vision depth models like [Foundation Stereo](https://nvlabs.github.io/FoundationStereo/)
or [CREStereo](https://github.com/megvii-research/CREStereo) which are very popular in robotics.
Also, this decision has a few things to consider weighing up USB and CSI:

- Latency and overhead
  CSI is a direct point-to-point hardware connection - data goes straight from the image sensor into the Jetson's ISP (Image Signal Processor) with minimal overhead. USB is a shared protocol with handshaking, packetisation, error checking, and a host controller managing the bus. Each of those steps adds latency. For a single camera the difference might be 1–5ms, but in a tight real-time control loop that matters.
- CPU involvement
CSI data arrives via DMA (Direct Memory Access - not similar to RDMA) - it goes straight into memory without the CPU touching it. The Jetson's ISP picks it up directly. USB transfers require the CPU to manage the transfer, consuming CPU cycles and adding jitter.
- Firmware dependency
 Your depth pipeline is tied to external firmware updates. If there's a bug in their depth algorithm or they change behaviour in an update, you have limited ability to fix or control it.
- Pre-calibrated fixed baseline
  The stereo baseline (distance between the two lenses) is fixed by the physical hardware design. With two separate CSI cameras you can choose your baseline distance. And as you know from your Geomatics degree, a wider baseline gives better intersection accuracy for longer ranges, narrower baselines work better up close. 

# Conclusion

As you can see, there are a few points to consider when selecting a hardware device that can fit your purpose, and especially important when needing to make this decision at scale. 
The price differences between the AGX, NX and Nano can be significant and knowing about the application requirements will help to decide what compute is required. It is similar to monitoring distributed systems in the cloud; in kubernetes when running microservice applications and wanting to tweak CPU and Memory limits, your ultimate goal is to keep utilization high which in turn keeps costs low in the long run. Overspecifying hardware leads to low utilisation - a cost that compounds quickly when deploying at scale (think hundreds of units across a warehouse).

I'll keep this post updated, so feel free to bookmark it and check back periodically.

Meet the Jetsons

An architect on site, delivering real value

## Table of Contents

# How to add AI value to your business - a supply chain metaphor

**Why understanding where AI value is actually created will save your business time, money, and a lot of confusion.**

---

Everyone's talking about AI. Your competitors claim they're using it and your board wants a strategy for it. But here's the problem: "AI" is one of the most overloaded terms in business today, but all you really want to know is how to create value in your domain?

At WOU (We Optimize You), clarity matters. So we're going to break it down for you — not with jargon, but with a metaphor from the built environment: **construction**.

---

## The AI Stack Is a Supply Chain — Not a Magic Box

AI is like the construction industry. There isn't one company that does everything from mining iron ore to handing you the keys to your new office. There's a supply chain — a stack of specialised layers — and each one plays a different role.

Here are five layers for the AI stack, and what they do.

![Image: The AI Construction Stack — a visual diagram showing five stacked layers from raw materials at the bottom to a finished building at the top, with labels for each layer described below.](//images.ctfassets.net/kot48ke411n3/7DQqLF7jwpyZJBIJBTguIx/c91c0fca05248986f66293bca44c4a7c/full_stack.png)

---

## Layer 1: Research — The Materials Scientists

> **Key takeaway:** Essential, but likely not where your business problems get solved.

At the very foundation, you have research. These are the university labs, research divisions at companies like Google DeepMind and Meta AI, and academic institutions pushing the boundaries of what's mathematically and computationally possible.

In this metaphor they are the **materials scientists** — the people who invent new types of steel, discover stronger concrete composites, and develop the science behind the materials the rest of the industry relies on.

This work is groundbreaking. It's also light years away from your business. Nobody walks into a materials science lab and asks them to build a new kitchen.

---

## Layer 2: Infrastructure — The Quarries and Steel Mills
> **Key takeaway:** Critical infrastructure, but buying GPUs doesn't give you an AI solution any more than buying steel beams gives you a building.

Next, you have infrastructure. This is NVIDIA building GPUs, cloud providers like AWS, Google Cloud, and Azure providing the raw compute power, and the massive data centres that make it all run.

![foundation-models](//images.ctfassets.net/kot48ke411n3/3WfSyrOu2GA7IXpSZj8L4J/6668af0a473a8f78a5957ce362cf29b8/core_products.jpg)

These are your **quarries, steel mills, and lumber yards** — they produce the raw materials at industrial scale. Without them, nothing gets built. But they don't design buildings. They don't know what your business needs. They supply the materials and move on.

You *could* try to go directly to this layer. You'd spend millions on hardware and compute, hire a team of specialists, and still be years away from anything that helps your customers. It would be the equivalent of buying a quarry because you need a new office.

---

## Layer 3: Foundation Models — Prefabricated Components

> **Key takeaway:** Foundation models are powerful raw capabilities. Without architecture, engineering, and integration, they're just components sitting in a warehouse.

This is where things start to feel familiar. Foundation models — GPT, Claude, Llama, Gemini, and others — are the general-purpose AI systems that have captured the world's attention.

Think of these as **prefabricated building components**: standardised walls, beams, window frames, and roofing panels. They're engineered to be versatile. They're manufactured at scale. And they're genuinely impressive.

But here's what most people miss: **prefab components are not a building.**

You can stack prefab walls in a field. You'll have walls. You won't have plumbing, electrical, climate control, fire safety, load-bearing calculations, or a floor plan that makes any sense for how your people actually work.

This is where the majority of businesses get stuck. They sign up for an AI tool, plug in a foundation model, and wonder why the results are inconsistent, unreliable, or disconnected from their actual workflows.

---

## Layer 4: The Architect and General Contractor — This Is Where WOU Works

![Image: An architect reviewing blueprints at a construction site, with prefab components in the background being assembled into a finished structure — representing the integration layer where WOU operates.](//images.ctfassets.net/kot48ke411n3/3Sr5OOAJTx4J9DMAVG2pDl/5239350bab0b90f7a974a92af05bdcd8/on_site.jpg)

> This is the layer where value is actually created for your business. And this is exactly where WOU sits.

We don't manufacture the steel. We don't run the quarries. We don't build the prefab components. **We are the architect and general contractor who designs and builds the actual building to your specifications.**

Here's what that means in practice:

We assess your business problem first — not the technology. We identify which foundation models, tools, and components are the right fit. We design the system architecture: how data flows, how the AI integrates with your existing processes, where the guardrails need to be, and how humans stay in the loop. Then we build, test, and deploy a solution that works reliably in the real world — your world.

This is bespoke work. Just as no good architect would hand you a copy-paste floor plan from another client and call it done, we don't drop a generic AI tool into your business and walk away. Every business has different data, different workflows, different constraints, and different definitions of success.

### Why This Layer Matters Most

> **Key takeaway:** The value isn't in the components — it's in knowing which components to use, how to assemble them, and how to make the finished product reliable, safe, and genuinely useful for your business.

Layers 1 through 3 are commoditising rapidly. The models are getting cheaper. The compute is getting cheaper. Access is getting easier. **The gap is no longer "can we access AI?" — it's "can we make AI actually work for our specific business?"**

That's an architecture and engineering problem, not a technology problem. And it's exactly the problem WOU exists to solve.

---

## Layer 5: The People in the Building — Your End Users

> **Key takeaway:** The best AI system is one your users never have to think about — it just works.

![Image: Happy professionals working inside a modern, well-designed office — representing the end users who benefit from the completed AI system, never needing to think about the infrastructure underneath.](//images.ctfassets.net/kot48ke411n3/4MzrKGWAc20AOmOe9A8I4t/d020ed55027bc111579abb879e6f07db/office.jpg)

At the top of the stack are the people who actually use the finished product: your employees, your customers, your operations team. They don't care about GPUs. They don't care about transformer architectures. They care that the system works, it's intuitive, and it makes their job easier or their experience better.

This is the measure of success. Not how sophisticated the technology is underneath, but whether the person in the building is warm, safe, productive, and happy.

Everything WOU builds is designed with this layer in mind. The technology is only as good as the outcomes it delivers to real people.

---

## So Where Does Your Business Go from Here?

The AI landscape is noisy. Vendors at every layer will tell you they're the answer. But now you have the framework to ask the right questions:

**Are they selling you raw materials** and expecting you to figure out the engineering? **Are they handing you prefab components** with no plan for how they fit your specific business? Or **are they doing the real work** — understanding your problem, designing a system, and building something that delivers measurable results?

At WOU, we believe the last approach is the only one that works. We don't build models. We build solutions — by selecting the right models, designing the right systems, and deploying them in a way that creates real business value for you and your customers.

If you're ready to stop buying steel beams and start building something useful, [let's talk](https://weoptimizeyou.co.za/contact).

---

*We Optimize You (WOU) helps businesses turn AI from a buzzword into a working system. We specialise in designing and deploying bespoke AI solutions that solve real business problems — without asking you to become an AI company yourself.*

You Need the "Right Builders"

A smaller image of the AI stack and where WOU sits.

# You Don't Need to Become an AI Company

You started your business because you're exceptional at something. Maybe it's logistics. Maybe it's financial services. Maybe you've spent fifteen years building deep expertise in healthcare, manufacturing, or retail. Whatever it is, that expertise is the engine of your business. It's the reason your customers trust you and the reason you win.

And then the AI wave hit.

Suddenly, everyone has an opinion. Your board wants an "AI strategy." Your competitors are making noise about automation. LinkedIn is full of people telling you that if you're not using AI, you're already dead. So you do what many responsible business owners do - you start exploring.

You watch the tutorials. You read the articles. You sit through vendor demos. You maybe even hire a data scientist or two. You spin up a few experiments. And before you know it, a significant chunk of your time, your budget, and your leadership attention is no longer focused on the thing that made your business successful. It's focused on trying to become something you never set out to be: a technology company.

This is the trap. Not because AI isn't valuable - it is. But because building, deploying, and maintaining AI systems is an entirely separate discipline. It requires different skills, different infrastructure, and different ways of thinking. Trying to bolt that capability onto your business from scratch doesn't make you innovative. It makes you distracted.

And distraction, for a growing business, is expensive.

## The AI Landscape Is Deeper Than You Think

Part of the problem is that "AI" has become a single buzzword that papers over an enormous amount of complexity. When someone says "we should use AI," they're glossing over a stack of technology that runs from pure academic research all the way down to the software your team actually touches every day.

Think of it like construction.

When you walk into a modern office building, you see the finished product - the open-plan workspace, the meeting rooms, the lights that turn on when you walk in. What you don't see is everything underneath: the materials science that produced the steel, the quarries and mills that supplied the raw materials, the prefabricated components that were engineered off-site, or the architect and general contractor who turned all of those pieces into the building you're standing in.

AI works the same way. There's a stack, and each layer requires fundamentally different expertise.

![full-stack](//images.ctfassets.net/kot48ke411n3/7DQqLF7jwpyZJBIJBTguIx/c91c0fca05248986f66293bca44c4a7c/full_stack.png)

### The research layer

At the very bottom, you have pure research. Universities and labs like DeepMind and OpenAI's research teams are pushing the boundaries of what's theoretically possible. This is materials science — inventing new types of steel and concrete. It's essential work, but nobody at this layer is thinking about your invoicing problem.
![foundation-models](//images.ctfassets.net/kot48ke411n3/3WfSyrOu2GA7IXpSZj8L4J/6668af0a473a8f78a5957ce362cf29b8/core_products.jpg)

### The infrastructure layer

Above that sits infrastructure. The GPUs, the cloud compute, the data centres run by NVIDIA, AWS, Google, and Microsoft. These are the quarries and steel mills — producing raw computational power at massive scale. Your business doesn't need to buy a data centre any more than a restaurant needs to own a cattle farm.

### The foundation models

Next come the foundation models themselves - GPT, Claude, Llama, Gemini, and dozens of others. These are the prefabricated building components: powerful, standardised, and increasingly available to anyone. They can do remarkable things out of the box. But "out of the box" is the key phrase. A stack of prefabricated wall panels is not a building. And a foundation model, on its own, is not a business solution.

This is where most people's understanding of AI stops. They see the foundation model, they're impressed by what it can do in a demo, and they assume the hard part is over.

It isn't. It's just beginning.

### The application and integration layer

![architect-on-site](//images.ctfassets.net/kot48ke411n3/3Sr5OOAJTx4J9DMAVG2pDl/5239350bab0b90f7a974a92af05bdcd8/on_site.jpg)

This is where our real work happens. This is the architect and general contractor — the layer where someone takes those powerful but generic model capabilities and combines them with your business logic, your data, your workflows, your compliance requirements, and your operational reality to build something that actually works. Reliably. At scale. In production. Not as a demo. Not as a proof of concept. As a system your team depends on every day.

This layer involves model selection, prompt engineering, data pipelines, testing, guardrails, monitoring, integration with existing systems, and ongoing maintenance. It's not glamorous. But it's where value is created or destroyed.

You wouldn't hand your employees a pile of prefabricated walls and say "figure it out." You'd hire someone who knows how to turn components into a functional building - one that meets code, serves your specific needs, and doesn't fall down six months later.

### The business impact layer

At the top of the stack are the people who actually live and work in the building. Your team. Your customers. The end users whose daily work gets faster, easier, or more effective. The finance director who sees the ROI. This is the only layer that matters to your business - and every layer below it exists to serve this one.
![happy-office](//images.ctfassets.net/kot48ke411n3/4MzrKGWAc20AOmOe9A8I4t/d020ed55027bc111579abb879e6f07db/office.jpg)

## You Don't Need to Build the Whole Stack

Here's the thing that nobody selling you AI wants to admit: the bottom three layers of this stack are commoditising fast. The models are getting cheaper, more capable, and more accessible every quarter. Having access to a foundation model is no longer a competitive advantage. Everyone has access.

> The advantage lives in Layer 4 — in knowing which models to use for which problems, how to integrate them into real business workflows, how to make them reliable and safe, and how to do all of that without burning through months of experimentation and six figures of wasted budget.

That's what we do.

We don't build foundation models. We don't run data centres. We don't do AI research. What we do is live at the application and integration layer — the architect and general contractor level — and we build bespoke AI systems that solve specific business problems for our clients.

We find the right models. We design the right architecture. We handle the integration, the testing, the deployment, and the ongoing performance. And we do it so that you don't have to become an AI company.

## Go Back to What You're Great At

The businesses that win with AI over the next five years won't be the ones that tried to do everything themselves. They'll be the ones that stayed focused on their core strengths and partnered with people who could handle the technology.

You don't build your own power station because your business uses electricity. You don't become a construction company because you need an office. And you don't need to become an AI company because your business can benefit from AI.

You need a partner who already understands the stack — so you can get back to doing the thing that made your business worth building in the first place.

Latest Posts

MCP Architecture for Agentic Systems

Backups for a database running at the edge

A look into database replication at the edge. With Spilo

Meet the Jetsons

You Need the "Right Builders"

Go Back to What You're Great At