I have been recently working on my basic INCUS setup. The actual system is finally done. I won’t talk about that too much, but basically its a staged ansibel build that gives you:
- A fully functional Model Serving Platform
- Separated “Experimental and Production” projects
- Snapshot for ‘rollback’
- Gated “model” upgrades (Since I can’t use the thing that upgrades the thing)
- COMPLETLY reproducible if it blows itself up WITHOUT an llm
I really wanted to talk about my new workflow. So I have what I call an “RFC” workflow that I adapted from a nice paper. It forces the LLM to make a staged,human gated ‘blueprint’ that doesn’t specify EVERYTHING but the GOALS. So its MUCH different than just a ‘tell it everything in small detail’. It is an LLM created, and Thomas guided process where I converse with the llm about a particular project, do research, and once we have enough data ‘create’ the thing. Then I have it create an EPUB file which I upload to my ereader, and then mark up.
I discuss the disagreements etc. with a Frontier model, and out of its butt comes the design.
I then have opencode (which is sandboxed with DROP) look at it, and validate the design, tell me what it wants to do, and perform ‘discovery’ on the remote platform. Here are the basics of how I start a new ‘project’
- I create a new base os in incus in the experimental group
- I add drops’ ssh key (the sandboxed one)
- I add a few requiremetns such as python, ansible, fish shell (Since bash is evil)
- Then I instruct the local opencode session (usually in a new project) to do some discovery of the platform
- We fill in the necessary details (ip address / dns name/ sudo access for the llm etc)
- I snapshot the ‘bare’ container in case the world ends
- Then I have the LLM execute the RFC and stop at gates and make sure that the context hasn’t been overflowed
- I perform manual tests after its done, and see if we need to do repair
- Perform repair and security analisys
- YEA done!
The most important thing is that I also watch what the llm does at least at first because it often gets in ruts, but MUCH less so with qwen 3.8 flash next. That and halogen are REALLY amazing approaching, IMHO some of the quality and MOST of the utility of frontier models WITHOUT giving up my privacy or data to evil horsepeople of the data apocalypse.
I could go more into it and sometime I will publish (so the evil corps can steal it) but I wanted to put something on the new grav site (Which was updated to new grav via ai & me). This was NOT written by AI just thomas.