Lemna Knowledge Base

3 min read

title: Infisical Secrets Integration tags: [infisical, secrets, infrastructure, session] created: 2026-05-25 updated: 2026-05-25 status: active related:

Session: Infisical Secrets Integration

Summary

Integrated self-hosted Infisical as the centralized secrets manager for the entire Lemna platform. Replaced scattered .env files with a single root .env containing 4 bootstrap variables that unlock all other secrets from the Infisical server.

What Changed

Core SDK (sdk/environ.py)

  • Added load_infisical() — authenticates with Infisical using 4 bootstrap vars, fetches all project secrets, injects into os.environ. Lenient: warns and continues if Infisical is unreachable.
  • Added load_env() — convenience function that calls load_dotenv() then load_infisical().
  • Kept load_dotenv() for backward compatibility.
  • All call sites updated: load_dotenv → load_env in client.py, backends.py, cas.py, database.py, auth.py, conftest.py.
  • Pantry CLI: added _load_infisical() alongside _load_dotenv().

Infisical Server

  • Deployed at pipeline/backend/secrets/ (Docker Compose: infisical-backend, postgres, redis).
  • Dashboard: http://localhost:8192
  • Project: lemna-platform (ID: 29f05bd6-5ef2-4557-b8f6-d952861f59ad)
  • Environments: dev (localhost), staging (empty), prod (production)
  • 9 secrets per environment: API_URL, API_KEY, R2_ENDPOINT_URL, R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY, RUNPOD_API_KEY, OLLAMA_API_KEY, LINEAR_API_KEY, GITHUB_TOKEN

Docker Compose

  • All 9 endpoint compose files: 5 env passthroughs → 4 Infisical bootstrap vars
  • database/docker-compose.yml: removed env_file: .env, added Infisical passthrough + hardcoded MongoDB config
  • LocalBackend.submit(): 5 hardcoded env vars → 4 Infisical bootstrap vars

Dependencies

  • infisicalsdk>=1.0.16 added to lemna-sdk, database, and pantry
  • python-dotenv removed from database (no longer needed)

Files Deleted

  • pipeline/backend/refactor/.env — obsolete
  • pipeline/backend/tests/.env — regenerated by load_env()
  • pipeline/backend/bucket/.env — dev RustFS creds now in Infisical dev environment
  • Root .env.old — backup, then deleted

Root .env

Trimmed from 12 secrets to 4 bootstrap vars:

INFISICAL_CLIENT_ID=01fd2bad-f309-4ce0-8fe3-fffa69a1fa05
INFISICAL_CLIENT_SECRET=00a7c93eb9de7c6aed8c51342fde785a1ea41415949365cff8989cc491a619c3
INFISICAL_API_URL=http://localhost:8192
INFISICAL_ENV=dev

Not Secrets (Hardcoded in Docker Compose)

  • DATABASE_URL=mongodb://mongodb:27017/?retryWrites=false — container networking
  • DATABASE_NAME=lemna — just a name
  • ME_CONFIG_MONGODB_URL, ME_CONFIG_BASICAUTH_USERNAME/PASSWORD — dev-only trivial creds
  • MongoDB has no auth enabled currently

Secrets Skill

  • Created .opencode/skills/secrets/SKILL.md — comprehensive documentation
  • Updated AGENTS.md — added to skills table, key terms, initialization list
  • Updated essentials skill — fixed endpoint env gotcha, added Secrets gotchas section
  • Updated map skill — updated backend directory structure, key terms, added secrets/ directory

Key Decisions

  1. Lenient over strict — load_infisical() warns and continues if Infisical is unreachable. Allows offline work and graceful degradation.
  2. Pattern 3 (host injection) — load_env() in code, not infisical run CLI wrapper. No CLI dependency needed, works inside Docker containers via env passthrough.
  3. bucket/.env deleted — dev RustFS creds moved to Infisical dev environment.
  4. DATABASE_URL hardcoded — container networking, not a secret. Avoids circular dependency on Infisical for database startup.
  5. URL-safe PostgreSQL password — openssl rand -base64 24 | tr '+/' '-_' to avoid / chars breaking DB_CONNECTION_URI.

Gotchas Hit During Implementation

  1. POSTGRES_PASSWORD with / chars broke DB_CONNECTION_URI. Fixed by using URL-safe base64.
  2. infisicalsdk method names differ from docs: create_secret_by_name not create_secret, list_secrets returns ListSecretsResponse.secrets not a plain list.
  3. Machine identity creation — must be scoped to the project, not org level. Org-level identities return 401.

Architecture

Root .env (4 bootstrap vars)
  │
  ├─ load_dotenv()     → reads .env, puts 4 vars into os.environ
  ├─ load_infisical()  → authenticates → fetches 9+ secrets → injects
  └─ load_env()        → calls both in order

All services use load_env() — backward compatible with old .env files if Infisical is unavailable.

Graph View

Recent Notes

Backlinks