A self-hosted RAG application, stood up on managed cloud infrastructure, verified end to end, and torn down again — for thirteen cents.
LocalChat lets people upload documents and ask questions about them, with retrieval over a vector database and a locally hosted language model. This proof of concept answers one question: can it run on managed cloud infrastructure without changing what the product is?
It can. The stack is a Serverless Container running the application, a Serverless SQL Database with pgvector for retrieval, and a small Instance serving the embedding model over a private network. No component was rewritten to fit the platform.
| Component | Service | Notes |
|---|---|---|
| Application | Serverless Container | 3 GB, one instance, no autoscaling |
| Database | Serverless SQL, PostgreSQL 16 | pgvector 0.8.2, scales to zero when idle |
| Embeddings | CPU Instance running Ollama | Reachable only over a private network |
| Text generation | Not deployed | Needs a GPU — the next decision |
Four commands. All of them are idempotent: running one twice does nothing the second time. They were written after deploying once by hand, so they encode what actually worked rather than what the documentation predicted.
bash scripts/scaleway/provision.sh # project, database, scoped credentials
bash scripts/scaleway/deploy_container.sh # the application
python scripts/scaleway/verify_deployment.py <endpoint>
bash scripts/scaleway/deploy_embeddings.sh # the embedding model hostThe third command is a gate rather than a checklist. It signs in, confirms the deployed image is the version you think it is, ingests a document and retrieves it with a query that shares no words with the text — so a passing result can only come from the vector search actually working. It exits non-zero on failure.
The platform has no spend cap at any level, so the teardown is a script rather than a habit:
PROJECT_ID=<project> bash scripts/scaleway/panic_teardown.sh # shows what would go
PROJECT_ID=<project> CONFIRM=DESTROY bash scripts/scaleway/panic_teardown.sh # stops itIt deletes rather than stops, because a stopped machine keeps billing for its disks and its reserved address. It refuses to touch anything outside the project it was given.
Three things the plan believed turned out to be wrong, and finding them is most of the value of running a proof of concept at all.
The plan assumed a first pass without any language model would still validate document upload and search. It does not: every document chunk is embedded through the model host, so with none reachable nothing is stored at all. The fix was not an expensive GPU — embeddings alone run acceptably on a small CPU machine, at roughly two cents an hour.
The managed database selects which database you reach from the encrypted handshake itself. An unencrypted connection cannot name its target, so it fails with a confusing error rather than a security warning. The application had never made this setting explicit; it does now.
The application's health endpoint reported the database as healthy for the life of the process, because it echoed a flag recorded at startup rather than asking the database anything. On a server that is always running, this behaves like correct code indefinitely. On a database that scales to zero, it reports green while every request fails.
This was a latent defect, not a platform requirement — the cloud only made it visible. That is the most transferable lesson here: when moving to managed infrastructure, look for the assumptions an always-present environment was quietly covering.
Thirteen cents for the entire exercise — database, application container, embedding host, private networking, stood up and torn down. The figure matters less than the shape behind it:
The sequencing that made this cheap: prove the plumbing on a two-cent machine, so that when an expensive one is finally switched on it has exactly one question left to answer.
The document path — upload, embed, store, retrieve — works on this infrastructure and has been verified. Text generation has not been deployed and is the open decision: a GPU instance, or moving to a hosted model API, which is cheaper to run and a change to what the product is rather than to how it is deployed.
The stack is currently torn down. It rebuilds from the four commands above.