LiteFunctions

What - serverless on your cluster: Lambda-shaped, without the vendor

Why - building from first principles — cuz we can 😉

Hello 👋🏻

I'm Ashu

I do go/rust stuff for infra, building for the craft #trad coder

Day job: Sr. software - compute infra @ Rippling

buzzword - Serverless

Yeah, the popular kid before AI.. in a gist, serverless is just "delegated infra"

#good, pay as you go #bad, vendor/stack lock in, can be pricey

So we want self hosted lambda

what's it doing.. you write functions; you get apis — same “lambda” contract, on your cluster.

helm install litefunctions oci://registry-1.docker.io/ashupednekar535/litefunctions

what we got

curl -sS -X POST "https://lambda.us-east-1.amazonaws.com/2015-03-31/functions/Spacelift_Test_Lambda_Function/invocations" \
  -H "Content-Type: application/json" \
  -d '{"key1":"world"}'

So what's infra, in the most generic sense

version control Git

orchestration Kubernetes

pubsub

storage

cache

traffic

So this is neither a dependency list nor undermining us infra folks ;) It’s just what’s in our toolbox for typical backends — storage, events, and the compute & traffic that runs them.

LiteFunctions is supposed to package all of this so it’s abstracted away.

Let's log in to see what it's got

WebAuthn into the Portal — pick a project, see functions and runs in one place.

Let's write an API

Function + route + language in the editor — save, and commit. Vim motions btw

There you go

we got ourselves an api

How about some websockets

Same stack — async path over NATS; WS / stream where the runtime exposes it.

What does lambda look like

At a high level it’s the layer between traffic and your code: it tracks what’s deployed, queues and routes work, and scales compute with traffic—so you ship functions and let the platform own inventory, concurrency, and runners instead of wiring that yourself.

Lambda-style architecture: control plane, invoke path, and runtimes

Let’s understand what needs to happen

edit

→

build

→

spawn

→

route

→

run

Bottom line, we version function code, build artifacts and spin up pods in response to traffic For dynamic languages, hot reload the code into pre-provisioned runtimes Kubernetes and node pools are where scaling and capacity land—scheduling, autoscaling, elasticity under those pods

Components

Operator

Reconciles functions and wiring in the cluster from CRDs.

Gateway

TLS and routing from clients into invokes.

Ingestor

HTTP entry: handles sync invokes; publishes async work to the broker.

Runtimes

Runs your functions—Go, Rust, Python, TypeScript, Lua—behind one invoke contract.

Dependencies & state

Crunchy Postgres

Durable state and metadata

Valkey

Cache, queues, fast key/value

Broker (NATS)

Pub/sub and event streams

Where does go come in

Well… everywhere! Right from litefunctions components like ingestor, operator, runtime, the ui to istio, nats, cruncy, and kubernetes itself...

litefunctions/ingestor

Clients hit the ingestor first—sync calls map to a straight HTTP request/response path.

Async work goes over NATS (our pub/sub): the ingestor stays fast and dumb, publishes off the HTTP hot path, and runtimes subscribe and do the work—so the transport stays abstracted from your function code, same idea as in the lambda FaaS write-up.

The ingestor can also request the operator to warm a function, basically set is_active on the Function CRD to true so reconcile runs.

ingestor/pkg/broker/request.go

WebSocket → NATS · Req + Produce · subject …exec.lang.reqId

package broker

type Req struct {
	Project, Name, Lang, ReqId string
}

func Produce(
	nc *nats.Conn,
	w http.ResponseWriter,
	r *http.Request,
	lang string,
) (*websocket.Conn, *Req, error) {
	// Upgrade; goroutine: ReadMessage → Publish
}

Sync handler

Read r.Body, then write headers and the response body on w in one goroutine—no channels; the “plate” is what you Encode or Write.

package pkg

import (
	"encoding/json"
	"net/http"
)

func Handle(w http.ResponseWriter, r *http.Request) {
	var req map[string]any
	_ = json.NewDecoder(r.Body).Decode(&req)
	resp := map[string]any{"ok": true, "echo": req}

	w.Header().Set("Content-Type", "application/json")
	w.WriteHeader(http.StatusOK)
	_ = json.NewEncoder(w).Encode(resp) // response body
}

Async consumer

Chunks arrive on input; you spin a goroutine that sends on out—like plates from the belt.

Runtime / WS can drain out to the wire—decoupled from one HTTP read.

package pkg

func StreamHandler(input <-chan []byte) <-chan []byte {
	out := make(chan []byte)
	go func() {
		defer close(out)
		for raw := range input {
			c := transform(raw)
			out <- c // may block (backpressure)
		}
	}()
	return out
}

func transform(b []byte) []byte { return b }

Static runtimes

go consumer

rust consumer

Dockerfile · sample

FROM scratch
COPY consumer /consumer
USER 65532:65532
ENTRYPOINT ["/consumer"]

docker pull · extract

✓ layer 1/2 pulling 12 MB

✓ layer 2/2 pulling 21 MB

≈ 34 MB · 2 layers

Ship small static binaries in distroless or scratch—images stay tiny (often tens of MB, few layers). Each function gets its own dedicated consumer pod that starts fast.

Go wants a single static Linux binary by default—drop-in for scratch. Rust can do the same (--target …-musl / static link), or emit a cdylib (.so) for plugins—then you’re in dynamic-linking / loader land. For these pods we stick to a static bin in both languages so the deploy story matches Go’s “one artifact.”

Scale each function up or down independently without dragging a shared interpreter along.

Gotcha · runtime

Since Go 1.22, default GOMAXPROCS follows Linux cgroup CPU limits—great in containers, but it breaks old assumptions that GOMAXPROCS tracked the whole host. Re-check load tests and pprof if you sized workers as “all CPUs.”

Gotcha · scratch & deps

FROM scratch means no libc—for HTTPS you need pure Go TLS (crypto/tls) and bundle CA certs if you verify remotes. Avoid CGO wrappers (e.g. libgit2): we used go-git with a billy-style filesystem so Git stays pure Go and the binary stays static-friendly. Same idea for any dep: prefer stdlib / pure Go over .so shims.

Dynamic runtimes

py consumer

Dockerfile · sample

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY runtime/ ./runtime/
COPY functions/ ./functions/
USER 65532:65532
CMD ["python", "-m", "runtime"]

docker pull · extract

✓ layer 1/8 pulling 52 MB

✓ layer 2/8 pulling 41 MB

✓ layer 3/8 pulling 38 MB

✓ layer 4/8 pulling 29 MB

✓ layer 5/8 pulling 47 MB

✓ layer 6/8 pulling 31 MB

✓ layer 7/8 pulling 35 MB

✓ layer 8/8 pulling 22 MB

≈ 295 MB · 8 layers

Python / JS share a long-lived runtime per project: pull sources from VCS, then hot-reload into the running process—iteration without a full image rebuild every time.

Tradeoff: larger images (hundreds of MB, many layers) and slower cold paths than static binaries—balanced by that reload loop.

pseudo · git event → reload
# Python
async def on_git_update(event):
    await sync_repo()# fetch / checkout
    importlib.invalidate_caches()
    mod = importlib.reload(sys.modules["functions." + name])
    # register handler; next request uses new code

// Node
async function onGitUpdate() {
  await git.pull();
  delete require.cache[require.resolve("./functions/" + name)];
  handlers.set(name, require("./functions/" + name));
}

Gotcha

Python: importlib + sys.modules—stale modules after hot swap; invalidate deliberately. GIL doesn’t fix races on shared globals across concurrent requests. JS (Node): require cache—bust or version modules on reload; one event loop, but shared mutable state still bites under load.

LiteFunctions Operator

An operator basically lets us have Kubernetes understand a custom YAML and do what we need to on the cluster

$ litefunctions/operator

internal/controller/function_controller.go

Reconcile is the heart: read desired state from the API (backed by etcd), then create/update/delete Deployments and Services to match your Function CRD.

package controller

type FunctionReconciler struct {
	client.Client
	Scheme *runtime.Scheme
}

func (r *FunctionReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
	// Get Function; early-exit on NotFound
	// If !IsActive: maybe delete Deployment+Service (skip if shared runtime still in use)
	// If active: NewDeployment / NewService; SetControllerReference; Create or Update
	// Patch Function.Spec.DeProvisionTime (keep-warm)
}