Proxmox VE Custom Templates #

cloud-init Template Prep #

From Official Generic Images #

First steps:

Download a generic (as opposed to cloud provider specific) image from official mirrors (Debian, Fedora, etc.)
Convert to qcow2 (some images are already in qcow2, but other could in a raw format)
Resize to the desired minimum size (by default, the templates images contain almost full filesystems, also, template drive sizes are not always nice round numbers in integer GiB, so resizing them at this stage is helpful for future cluster management)

The option preallocation=ff (alternatively preallocation=metadata) allows saving space on the template storage.

# Check the source image format
qemu-img info SOURCE_FILE.img

# Check the resulting file's info
qemu-img info OUTPUT_NAME-8g.qcow2

# Convert from raw (.iso or .raw) to (.qcow2)
# replace '-f raw' to '-f qcow2' if the original file was already qcow2.
qemu-img convert -o preallocation=off -f raw -O qcow2 SOURCE_FILE.img OUTPUT_NAME-8g.qcow2

# Resize to 8 GiB
qemu-img OUTPUT_NAME-8g.qcow2 8G

# Check the resulting file's info
qemu-img info OUTPUT_NAME-8g.qcow2

Create a VM in CLI:

MY_BASE_IMAGE=/var/lib/vz/template/iso/ubuntu2404/noble-server-cloudimg-amd64-4g.qcow2
MY_VMID=9012
MY_HOSTNAME="debian13template45"
MY_NETWORK_BRIDGE="vmbr512"

MY_PVE_STORAGE="cluster0-drives-ceph"

# Optional format for the backing file (this can be set only on file-backed storage)
# We cannot set this on Ceph block storage
#MY_PVE_DRIVE_FORMAT="--format qcow2"

# Create the VM
qm create $MY_VMID --name $MY_HOSTNAME --memory 4096 --cores 4 --net0 virtio,bridge=$MY_NETWORK_BRIDGE --ostype l26 --cpu x86-64-v2-AES

# Import cloud-init image as VM disk
qm importdisk $MY_VMID $MY_BASE_IMAGE $MY_PVE_STORAGE $MY_PVE_DRIVE_FORMAT

Note that the above VM creation line does not specify all the HW details. The quickest approach for now is to adjust some HW options in the web UI:

SCSI Controller: switch to VirtIO SCSI single (this allows setting the io_thread option)
Network Device: enable firewall (creation through web UI set it to enabled)

We can also configure the created VM in CLI:

# Add cloud-init drive
qm set $MY_VMID --ide2 $MY_PVE_STORAGE:cloudinit

# Configure boot options
qm set $MY_VMID --boot c --bootdisk scsi0

# Add serial console for cloud-init output (enables text-based console)
qm set $MY_VMID --serial0 socket --vga serial0

# Configure the networking
qm set $MY_VMID --ipconfig0 ip=dhcp

# Enable the qemu guest agent(recommend)
qm set $MY_VMID --agent enabled=1

Refs:

Allow SSH password auth #

If we want cloud-init to enable password auth in the sshd servie, place the following config in /etc/cloud/cloud.cfg.d/70_allow_ssh_pwauth.cfg:

ssh_pwauth: true

Refs:

Preserve package sources #

Complete this step if the custom template contains preconfigured package repository mirrors and we want to preserve them.

Place the following config in /etc/cloud/cloud.cfg.d/80_preserve_custom_sources.cfg:

# Use preconfigured custom apt sources in the image.
apt_preserve_sources_list: true

Cleanup Before Shutdown #

# Clean up cached packages
# Note that `apt autoclean` removed only outdated / not available packages
sudo apt autoremove
sudo apt clean

# Clear the current session's history
history -c 

# Unset the environment variable that points to the history file
unset HISTFILE 

# Remove the physical history file from the disk
rm ~/.bash_history

# Run cloud-init cleanup commands (optional, but standard practice for templates)
sudo cloud-init clean
sudo cloud-init clean --logs

# Trim (discard) free FS blocks.
# Not necessary if the filesystem is mounted with the discard option.
# Ref:
#   * https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/deduplicating_and_compressing_storage/discarding-unused-blocks_deduplicating-and-compressing-storage
sudo fstrim -v /

# to trim all mounted filesystems
sudo fstrim -v --all

# Power off the VM immediately
sudo poweroff

Recreate SSH Host Keys #

Cloud-init does this step automatically on first boot, so we only have complete this step on non-cloud-init templates or if we want to repair the VM.

Switch to sudo:
```
sudo su
```

List existing (old) keys with

ls /etc/ssh/ssh_host_*

It should display:

/etc/ssh/ssh_host_ecdsa_key
/etc/ssh/ssh_host_ecdsa_key.pub
/etc/ssh/ssh_host_ed25519_key
/etc/ssh/ssh_host_ed25519_key.pub
/etc/ssh/ssh_host_rsa_key
/etc/ssh/ssh_host_rsa_key.pub

Remove existing host keys:
```
rm /etc/ssh/ssh_host_*
```
Regenerate new keys:
```
dpkg-reconfigure openssh-server
```
Restart the VM or run:
```
systemctl restart ssh
```