Building self hosted lambda

Building a functions as a service platform in Go from first principles

Hello 👋🏻

I'm Ashu

I do go/rust stuff for infra, building for the craft #trad coder

Day job: Sr. software - compute infra @ Rippling

The problem

I wanted Lambda-like developer experience, but on my own cluster, with my own control plane.

what's it doing.. you write functions; you get apis — same “lambda” contract, on your cluster.

helm install litefunctions oci://registry-1.docker.io/ashupednekar535/litefunctions

what we got

curl -sS -X POST "https://lambda.us-east-1.amazonaws.com/2015-03-31/functions/Spacelift_Test_Lambda_Function/invocations" \
  -H "Content-Type: application/json" \
  -d '{"key1":"world"}'

What does Lambda-like really mean?

At a high level it’s the layer between traffic and your code: it tracks what’s deployed, queues and routes work, and scales compute with traffic—so you ship functions and let the platform own inventory, concurrency, and runners instead of wiring that yourself.

Lambda-style architecture: control plane, invoke path, and runtimes

The real question

What actually has to happen when someone invokes a function?

edit

→

build

→

spawn

→

route

→

run

Bottom line, we version function code, build artifacts and spin up pods in response to traffic For dynamic languages, hot reload the code into pre-provisioned runtimes Kubernetes and node pools are where scaling and capacity land—scheduling, autoscaling, elasticity under those pods

Why Go for this?

It fits right in to the cloud native ecosystem, ideal for building components like ingestor, operator, runtime, the ui to the ecosystem from istio, gitea, nats, cruncy, and kubernetes itself are written in go

Components we had to build

Operator

Reconciles functions and wiring in the cluster from CRDs.

Gateway

TLS and routing from clients into invokes.

Ingestor

HTTP entry: handles sync invokes; publishes async work to the broker.

Runtimes

Runs your functions—Go, Rust, Python, TypeScript, Lua—behind one invoke contract.

Dependencies & state

Crunchy Postgres

Durable state and metadata

Valkey

Cache, queues, fast key/value

Broker (NATS)

Pub/sub and event streams

Portal: control plane

portal handlers

Portal is where platform intent starts: users, projects, and VCS automation.

Identity

WebAuthn auth, session boundaries, project access.

Projects

Create project, set roles, sync functions from repo.

VCS + Actions

Create repo, add webhook, push vendor workflow, track runs.

WebAuthn contracts

auth/spec.go

Webauthn, where security and convinience don't have to be at odds

Challenge state, credential writes, and user session are explicit interfaces.

Pseudocode

begin registration -> store challenge session

finish registration -> verify attestation -> persist credential

// internal/auth/spec.go
type PasskeyUser interface {
  webauthn.User
  AddCredential(*webauthn.Credential) error
  UpdateCredential(*webauthn.Credential) error
}

// internal/auth/spec.go
type PasskeyStore interface {
  GetOrCreateUser(userName string) (PasskeyUser, error)
  SaveCredential(user PasskeyUser, cred *webauthn.Credential) error
  GetSession(token string) (webauthn.SessionData, bool)
  SaveSession(username, token string, data webauthn.SessionData) error
  SessionStore
}

DB schema + sqlc

query.sql

SQLc, the perfect database abstraction

DDL

-- migrations/00001_init_schema.sql
-- +goose Up
-- +goose StatementBegin
CREATE TABLE projects (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name TEXT NOT NULL UNIQUE,
description TEXT,
created_by BYTEA NOT NULL
);
-- +goose StatementEnd

          goose up

DML

-- internal/project/adaptors/query.sql
-- name: CreateProject :one
INSERT INTO projects (name, description, created_by)
VALUES ($1, $2, $3)
RETURNING *;

-- name: GetProjectByID :one
SELECT * FROM projects WHERE id = $1;

          sqlc generate

Typed code

// generated shape (query.sql.go)
type CreateProjectParams struct {
Name string
Description pgtype.Text
CreatedBy []byte
}
func (q *Queries) CreateProject(ctx context.Context, arg CreateProjectParams) (Project, error)

Let AI draft SQL, generate type safe adaptors.

Git plumbing (billy/memfs)

repo/spec.go

Git plumbing in-memory: fast, isolated, and no host FS coupling.

// internal/project/repo/spec.go
type GitRepo struct {
  Storage *memory.Storage
  Fs      billy.Filesystem
  Repo    *git.Repository
}

func NewGitRepo(project string, branch *string) (*GitRepo, error) {
  fs := memfs.New()               // in-memory billy FS
  r := GitRepo{
    Fs:      fs,                  // no disk dependency
    Storage: memory.NewStorage(), // in-memory git object db
    Options: &git.CloneOptions{ URL: ... },
  }
  // SetupAuth -> Clone -> cache per project
}

// pseudocode path
repo := NewGitRepo(project)
repo.Fs.Create(path)       // write function file in memfs
repo.Commit(path)          // git commit signature
repo.Push()                // vendor remote

wins stateless, no file IO | risks process local, redundancy / drift

Vendor interfaces (GitHub/Gitea)

vendors/spec.go

Apart from standard git plumbing, we need to interface with the vendor directly for certain stuff like createRepo, addWebhook, etc

standard issue strategy pattern to abstract the integration from the callers

// internal/project/vendors/spec.go
type VendorClient interface {
  CreateRepo(ctx context.Context, opts CreateRepoOptions) (*Repository, error)
  DeleteRepo(ctx context.Context, owner, repo string) error
  AddWebhook(ctx context.Context, owner, repo string, opts WebhookOptions) (*Webhook, error)
  AddWorkflow(project string) error
  GetActionsProgress(ctx context.Context, owner, repo string, opts ActionsProgressOptions) (*ActionsProgress, error)
}

func NewVendorClient() (VendorClient, error) {
  switch pkg.Cfg.VcsVendor {
  case "github": return NewGitHubClient(pkg.Cfg.VcsToken), nil
  case "gitea":  return NewGiteaClient(pkg.Cfg.VcsBaseUrl, pkg.Cfg.VcsToken), nil
  default:       return nil, fmt.Errorf("unsupported vendor")
  }
}

Why this matters: handler logic stays stable while vendor backends swap behind interfaces.

Ingestor: decouple accept from execution

Clients hit the ingestor first—sync calls map to a straight HTTP request/response path.

Async work goes over the broker, the ingestor stays fast and dumb, publishes off the HTTP hot path, and runtimes subscribe and do the work

The ingestor can also request the operator to warm a function, basically set is_active on the Function CRD to true so reconcile runs.

ingestor/pkg/broker/request.go

WebSocket → NATS · Req + Produce · subject …exec.lang.reqId

package broker

type Req struct {
	Project, Name, Lang, ReqId string
}

func Produce(
	nc *nats.Conn,
	w http.ResponseWriter,
	r *http.Request,
	lang string,
) (*websocket.Conn, *Req, error) {
	// Upgrade; goroutine: ReadMessage → Publish
}

Sync invocation flow

Read r.Body, then write headers and the response body on w in one goroutine—no channels; the “plate” is what you Encode or Write.

package pkg

import (
	"encoding/json"
	"net/http"
)

func Handle(w http.ResponseWriter, r *http.Request) {
	var req map[string]any
	_ = json.NewDecoder(r.Body).Decode(&req)
	resp := map[string]any{"ok": true, "echo": req}

	w.Header().Set("Content-Type", "application/json")
	w.WriteHeader(http.StatusOK)
	_ = json.NewEncoder(w).Encode(resp) // response body
}

There u go

we got ourselves an api

Demo

Function + route + language in the editor — save, and commit. Vim motions btw

Async invocation flow

Chunks arrive on input; you spin a goroutine that sends on out—like plates from the belt.

Runtime / WS can drain out to the wire—decoupled from one HTTP read.

package pkg

func StreamHandler(input <-chan []byte) <-chan []byte {
	out := make(chan []byte)
	go func() {
		defer close(out)
		for raw := range input {
			c := transform(raw)
			out <- c // may block (backpressure)
		}
	}()
	return out
}

func transform(b []byte) []byte { return b }

Demo

Same stack — async path over NATS; WS / stream where the runtime exposes it.

Static runtimes

go consumer

rust consumer

Dockerfile · sample

FROM scratch
COPY consumer /consumer
USER 65532:65532
ENTRYPOINT ["/consumer"]

docker pull · extract

✓ layer 1/2 pulling 12 MB

✓ layer 2/2 pulling 21 MB

≈ 34 MB · 2 layers

Ship small static binaries in distroless or scratch—images stay tiny (often tens of MB, few layers). Each function gets its own dedicated consumer pod that starts fast.

Scale each function up or down independently without dragging a shared interpreter along.

Gotcha · runtime

Since Go 1.22, default GOMAXPROCS follows Linux cgroup CPU limits—great in containers, but it breaks old assumptions that GOMAXPROCS tracked the whole host. Re-check load tests and pprof if you sized workers as “all CPUs.”

Gotcha · scratch & deps

FROM scratch means no libc—for HTTPS you need pure Go TLS (crypto/tls) and bundle CA certs if you verify remotes. Avoid CGO wrappers (e.g. libgit2): we used go-git with a billy-style filesystem so Git stays pure Go and the binary stays static-friendly. Same idea for any dep: prefer stdlib / pure Go over .so shims.

Dynamic runtimes

py consumer

Dockerfile · sample

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY runtime/ ./runtime/
COPY functions/ ./functions/
USER 65532:65532
CMD ["python", "-m", "runtime"]

docker pull · extract

✓ layer 1/8 pulling 52 MB

✓ layer 2/8 pulling 41 MB

✓ layer 3/8 pulling 38 MB

✓ layer 4/8 pulling 29 MB

✓ layer 5/8 pulling 47 MB

✓ layer 6/8 pulling 31 MB

✓ layer 7/8 pulling 35 MB

✓ layer 8/8 pulling 22 MB

≈ 295 MB · 8 layers

Python / JS share a long-lived runtime per project: pull sources from VCS, then hot-reload into the running process—iteration without a full image rebuild every time.

Tradeoff: larger images (hundreds of MB, many layers) and slower cold paths than static binaries—balanced by that reload loop.

pseudo · git event → reload
# Python
async def on_git_update(event):
    await sync_repo()# fetch / checkout
    importlib.invalidate_caches()
    mod = importlib.reload(sys.modules["functions." + name])
    # register handler; next request uses new code

// Node
async function onGitUpdate() {
  await git.pull();
  delete require.cache[require.resolve("./functions/" + name)];
  handlers.set(name, require("./functions/" + name));
}

Gotcha

Python: importlib + sys.modules—stale modules after hot swap; invalidate deliberately. GIL doesn’t fix races on shared globals across concurrent requests. JS (Node): require cache—bust or version modules on reload; one event loop, but shared mutable state still bites under load.

Operator: desired state -> running state

An operator basically lets us have Kubernetes understand a custom YAML and do what we need to on the cluster

The reconcile loop

Recover cleanly from partial failures without duplicating state.

internal/controller/function_controller.go

Reconcile is the heart: read desired state from the API (backed by etcd), then create/update/delete Deployments and Services to match your Function CRD.

package controller

type FunctionReconciler struct {
	client.Client
	Scheme *runtime.Scheme
}

func (r *FunctionReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
	// Get Function; early-exit on NotFound
	// If !IsActive: maybe delete Deployment+Service (skip if shared runtime still in use)
	// If active: NewDeployment / NewService; SetControllerReference; Create or Update
	// Patch Function.Spec.DeProvisionTime (keep-warm)
}

Reconcile path function activation spin up deploy / HPA map services set deprovision time cleanup

Packaging it all

version control Git

orchestration Kubernetes

pubsub

state

traffic

runtime

Compute + traffic + state packaged as one

helm install litefunctions oci://registry-1.docker.io/ashupednekar535/litefunctions

What's next

Explore stronger isolation for multi-tenant function execution

Inspiration: NanoVMs Go unikernel tutorial and Firecracker-style microVM deployments as future runtime experiments.

Contributions welcome

github.com/ashupednekar/litefunctions

function packages
gateway ingress policies
HTTP over pubsub
alternative runtimes like Firecracker / microVMs / unikernels
…and much more

issues / PRs welcome 👋

Thank you

That was it folks… questions?

come hang with me over at…

👋