depth-anything.cpp

Brought to you by the LocalAI team, the folks behind LocalAI, the open-source AI engine that runs any model (LLMs, vision, voice, image, video) on any hardware, no GPU required.

A from-scratch C++17/ggml port of Depth Anything 3 (ByteDance) for dependency-free monocular metric depth + camera pose inference. One self-contained GGUF file, no Python, no PyTorch, no CUDA toolkit at inference, just a small native library and CLI, and now faster than PyTorch on CPU, bit-exact against the original.

The same photo, depth computed side by side on CPU: identical output, depth-anything.cpp gets there first (full clip).

Given an image it recovers a dense metric depth map, per-pixel confidence, the camera extrinsics (3x4) and intrinsics (3x3), an optional sky mask, a back-projected 3D point cloud, and exports to glb / COLMAP / PLY. Everything is verified numerically equal to the reference DA3 forward (correlation 1.0), component by component.

Features

• Monocular metric depth + camera pose from a single image, plus multi-view depth+pose.
• Full DA3 family. small (ViT-S), base (ViT-B), large (ViT-L), giant (ViT-g), metric-large, mono-large, and the nested giant+large metric model - all driven by metadata baked into the GGUF.
• The whole output surface: depth, confidence, sky mask, extrinsics/intrinsics, ray-based pose, 3D Gaussians / point cloud, and glb / COLMAP / PLY export.
• Self-contained GGUF. Every dimension, hyperparameter and preprocessing constant lives inside the file. The loader reads them; nothing is hardcoded, no external config or vocab is shipped.
• Quantization to f16 / q8_0 / q6_k / q5_k / q4_k - q4_k is 99 MB (0.25x the f32) and near-lossless.
• CPU-first, GPU-ready. Tuned CPU path (tinyBLAS, Winograd, flash-attention) plus CUDA / Metal / Vulkan ggml backends.
• Flat C API (include/da_capi.h) - embed from C, C++, Go, or Rust. Powers the LocalAI backend.
• Parity-first. Every component is gated against PyTorch-dumped reference tensors; the end-to-end depth matches the real net() at correlation 1.0.

Supported models

Convert any of the official Depth-Anything-3 checkpoints to GGUF. All run through the same metadata-driven engine:

Depth Anything V2

The same engine also runs Depth Anything V2 checkpoints (single-image depth only — no confidence, pose or sky). Relative models emit inverse depth (ReLU); metric models emit depth in metres (Sigmoid × max_depth).

DA2 is depth only (no pose/confidence). The ViT-g (Giant) DA2 checkpoint is not shipped — its Depth-Anything-V2-Giant HF repo is gated/unreleased.

Performance

depth-anything.cpp is now faster than PyTorch on CPU: 1.20x at f32 and 1.31x at q8_0 on the production @504 path, while also running the same model in half the memory, loading ~6.7x faster, shipping as a 99 MB quantized file, and needing no Python / PyTorch / CUDA at inference, all while staying bit-exact (correlation 1.0 vs the reference).

CPU, AMD Ryzen 9 9950X3D (16-core / 32-thread x86), threads=16, 504x336, sustained (repeat=25), PyTorch f32 baseline (full methodology + the CPU-optimization history in benchmarks/BENCHMARK.md):

What flipped it: two positional embeddings (the DPT head's UV embedding ~90 ms, the backbone's bicubic pos-embed ~10 ms) were recomputed every forward with single-threaded scalar sin/cos and bicubic loops, even though they depend only on the input geometry and are identical every call. Caching them removed ~95 ms of per-forward host overhead (PyTorch builds the same embeddings with vectorized ops). These are x86 + oneDNN numbers; see benchmarks/BENCHMARK.md. Camera pose adds a few ms on top of depth at f32. q8_0 is near-lossless; q4_k is the smallest. Bit-exact parity holds at every quant down to f16.

On GPU (NVIDIA GB10, via -DDA_GGML_CUDA=ON) the ggml CUDA path with flash attention ties PyTorch's tuned cuDNN (47.3 vs 47.3 ms/forward, ~47 ms across every quant) and loads 1.75-2.9x faster, so it wins the cold start (~548 vs ~926 ms). Details in benchmarks/BENCHMARK.md.

See it run

Real photos through the actual CLI, input next to the colorized depth (turbo):

More plots: load time, model size, quantization tradeoff.

Build

git clone --recursive https://github.com/mudler/depth-anything.cpp
cd depth-anything.cpp
cmake -B build -DDA_BUILD_CLI=ON
cmake --build build -j
# -> build/examples/cli/da3-cli

CMake options

GPU example: cmake -B build -DDA_GGML_CUDA=ON && cmake --build build -j. See docs/GPU.md.

Python environment setup (conversion only)

Conversion and parity checks need a Python env; inference does not.

python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
python scripts/download_model.py --repo depth-anything/DA3-BASE --out models/DA3-BASE

Converting a model

# depth + pose models (small/base/large/giant)
python scripts/convert_da3_to_gguf.py --model models/DA3-BASE --output models/depth-anything-base-f32.gguf

# monocular (depth + sky) - DA3MONO-LARGE
python scripts/convert_mono_to_gguf.py --model models/DA3MONO-LARGE --output models/depth-anything-mono-large-f32.gguf

# nested two-branch metric model
python scripts/convert_nested_to_gguf.py --model models/DA3NESTED-GIANT-LARGE --output-prefix models/depth-anything-nested

# Depth Anything V2 — relative (encoder vits/vitb/vitl)
python scripts/convert_da2_to_gguf.py --encoder vitl --ckpt models/depth_anything_v2_vitl.pth \
    --output models/depth-anything2-large-f32.gguf --name Depth-Anything-V2-Large

# Depth Anything V2 — metric (add --max-depth: 20 for Hypersim/indoor, 80 for VKITTI/outdoor)
python scripts/convert_da2_to_gguf.py --encoder vits --ckpt models/depth_anything_v2_metric_hypersim_vits.pth \
    --output models/depth-anything2-metric-hypersim-small-f32.gguf \
    --name Depth-Anything-V2-Metric-Hypersim-Small --max-depth 20
python scripts/convert_da2_to_gguf.py --encoder vitl --ckpt models/depth_anything_v2_metric_vkitti_vitl.pth \
    --output models/depth-anything2-metric-vkitti-large-f32.gguf \
    --name Depth-Anything-V2-Metric-VKITTI-Large --max-depth 80

Quantization

build/examples/cli/da3-cli quantize models/depth-anything-base-f32.gguf models/depth-anything-base-q4_k.gguf q4_k
# types: f16 | q8_0 | q6_k | q5_k | q4_k

Running inference

CLI=build/examples/cli/da3-cli
M=models/depth-anything-base-f32.gguf

# Depth -> PFM (lossless float) + a colorizable grayscale PNG
$CLI depth --model $M --input photo.jpg --pfm depth.pfm --png depth.png

# Depth + camera pose (extrinsics 3x4 / intrinsics 3x3) as JSON
$CLI depth --model $M --input photo.jpg --pose pose.json

# Ray-based pose (solved from the auxiliary ray field; needs a --with-aux GGUF)
$CLI depth --model models/depth-anything-base-aux-f32.gguf --input photo.jpg --ray-pose --pose pose.json

# Monocular model: depth + sky mask
$CLI depth --model models/depth-anything-mono-large-f32.gguf --input photo.jpg --pfm depth.pfm --sky sky.pfm

# Nested metric-scale depth (two GGUFs)
$CLI depth --model nested-anyview.gguf --metric-model nested-metric.gguf --input photo.jpg --pfm metric.pfm

# Multi-view depth + pose
$CLI depth --model $M --input a.jpg --input b.jpg --out-prefix scene

# 3D export from a single image
$CLI depth --model $M --input photo.jpg --glb scene.glb --colmap colmap_out/
$CLI reconstruct --model models/depth-anything-giant-f32.gguf --input photo.jpg --ply cloud.ply

# Model metadata (arch, dims, head config, quant)
$CLI info --model $M

The .glb (glTF 2.0) and COLMAP cameras/images/points3D writers are dependency-free (no trimesh / pycolmap) and parity-checked against the reference exporters. See docs/EXPORT.md.

Use it from LocalAI

depth-anything.cpp ships as a native LocalAI backend (Go gRPC + purego, the .so static-links ggml - no external libggml). It exposes a typed Depth gRPC RPC and a POST /v1/depth REST endpoint returning the full output - per-pixel depth, confidence, sky, extrinsics/intrinsics, the 3D point cloud, and glb/COLMAP exports - not just a normalized PNG.

# once the backend + models are in the gallery:
local-ai run depth-anything-3-base

# REST: full depth field + pose + points in one typed response
curl http://localhost:8080/v1/depth -H 'Content-Type: application/json' -d '{
  "model": "depth-anything-3-base",
  "src": "photo.jpg",
  "include_depth": true, "include_pose": true, "include_points": true
}'

Gallery entries cover base (q4_k/q8_0/f16/f32), small, large, giant, and mono-large. The LocalAI backend lives in the LocalAI repo under backend/go/depth-anything-cpp/.

C API

A flat C ABI (include/da_capi.h, abi_version 4) over libdepthanything.so:

da_ctx* ctx = da_capi_load("model.gguf", /*threads*/ 8);
// nested metric model (two branches): da_capi_load_nested(anyview, metric, threads)
int h, w, is_metric;
float *depth, *conf, *sky, ext[12], intr[9];
da_capi_depth_dense(ctx, "photo.jpg", &h, &w, &depth, &conf, &sky, ext, intr, &is_metric);
// ... use depth[h*w], conf, pose ...
da_capi_free_floats(depth);

int n; float *xyz; unsigned char *rgb;
da_capi_points(ctx, "photo.jpg", /*conf_thresh*/ 1.0f, &n, &xyz, &rgb);   // 3D cloud
da_capi_export_glb(ctx, "photo.jpg", "scene.glb");
da_capi_free(ctx);

Opaque handles, C types only, da_capi_last_error for diagnostics. Build it with -DDA_SHARED=ON.

Parity & tests

The port is verified numerically equal to the reference, per component, not just end-to-end. The ctest suite (-DDA_BUILD_TESTS=ON, 37 tests) gates the preprocessing, backbone, attention, DPT head, depth, pose, ray head, ray→pose solver, exporters, and the C API against PyTorch-dumped tensors. End-to-end depth correlates 1.0 with the real DA3 forward; the full verification and parity numbers are in docs/VERIFICATION.md.

cmake -B build -DDA_BUILD_TESTS=ON && cmake --build build -j && ctest --test-dir build

Why depth-anything.cpp

Depth Anything 3 is a great model, but running it for inference drags in a heavy Python/PyTorch/CUDA stack. This is a clean C++17/ggml port focused purely on inference:

• No Python at inference. A single libdepthanything.so behind a flat C API, easy to embed from C, C++, Go, or Rust.
• Faster than PyTorch on CPU (1.20x f32, 1.31x q8_0), in half the memory, with ~6.7x faster load and bit-exact output.
• Small and portable. Self-contained GGUF with f16 / q8_0 / K-quants, on CPU and any ggml GPU backend.
• The whole model. Depth, confidence, pose, sky, ray-pose, 3D point cloud, and glb/COLMAP/PLY export, all parity-checked, across the full DA3 model family.

Citation

If you use depth-anything.cpp, please cite this repository and the original model:

@software{depth_anything_cpp,
  title  = {depth-anything.cpp: a C++/ggml inference engine for Depth Anything 3},
  author = {Di Giacinto, Ettore and Palethorpe, Richard},
  url    = {https://github.com/mudler/depth-anything.cpp},
  year   = {2026}
}

Depth Anything 3 is by ByteDance Seed (bytedance-seed/depth-anything-3).

Author

Ettore Di Giacinto (@mudler).

License

depth-anything.cpp is released under the MIT License. Model weights retain the license of their original Depth Anything checkpoint; converting them to GGUF does not change that license. For the models supported by the demo:

Always check the official model table and the individual Hugging Face model card before distributing or commercially deploying a checkpoint, since upstream terms may change or differ for other model variants.

Built by the LocalAI team. If you want to run depth (and LLMs, vision, voice, image, and video models) locally on any hardware with an OpenAI-compatible API, give LocalAI a star.