Migrating from Proxmox to NixOS and Incus

Incus is great. I've been trying to revive an unmaintained ansible collection to manage incus resources https://github.com/sbstp/ansible-collection-incus

This seems very cool and I will probably try it, but I think I’m missing something. I run Proxmox so that I can have multiple VMs running on my NUC. This doesn’t really solve that right? I cant spin up a windows 11 vm one weekend for a random experiment.

I've been using incus for a while now and actually run it on a side project in production for the better part of a year. Rock solid performance.

I'm also considering migrating from Proxmox to Incus, but I'd look into IncusOS rather than having to manage the host OS myself.

NixOS has transformed my use of Proxmox. I configure, build, and deploy everything from my nix workstation. I don't need to use the PVE gui at all. Proxmox is just a target, and I've abstracted things enough to where I can deploy the same machines to libvirt on a local machine too. Why would I need to let my agent into my PVE box? I haven't looked at incus, but if I wanted to run the full stack declaratively, nixos and LLMs are so powerful now that I would probably just say to run libvirt and ZFS on nixos natively.

I haven't abandoned Proxmox yet, but the take here resonates with me. I do not like configuring appliances. I prefer defining infra as code, having that diffable, assertable, etc. I have had pretty good luck managing Proxmox clusters with the Proxmox API (https://pve.proxmox.com/pve-docs/api-viewer/index.html) or just letting the agent shell in as root (lol). I built a very simple provisioning tool called vmfactory that takes some really somple config on disk, bakes a fresh qcow image, pushes it to proxmox and then configures networking and boots it. It's extremely rudimentary but has been working well for me.

I did abandon TrueNAS, however. It really is a locked-down appliance. Good luck installing custom software on the base OS. I have a domain-joined Ubuntu/ZFS box that inherits a lot of policy from FreeIPA and/or Ansible config that is all backed by files on disk. It's been really easy to orchestrate what many would consider overkill in my homelab because literally everything is represented in a single Github repo.

I yanked vmfactory out and into a standalone repo if anyone is interested: https://github.com/whalesalad/vmfactory

> But fundamentally, Proxmox is built around clicking buttons. It is a GUI-first paradigm.

Uhh, whut? It provides a button-y interface, but you can do everything via config files and `pct` on the command line if you prefer. I know that’s not full nix-style declarative, but you don’t have to mislead to sell me on the advantages of declarative infra.