Planning a Proxmox Build
If you’re just getting started with proxmox, it’s going to take a considerable time setting everything up right and some of the decisions made early on will impact a lot of what you can do. Changing these is going to require more effort and time. So it’s best to take some pointers under consideration before you begin.
A lot of these decisions will be impacted by how you want to use your build, but this guide should help you ask the right questions before you start.
Install
The first decision you will face is during the install, particularly where you decide to put the proxmox (host) itself. A server that runs 24/7 is going to take a toll on the storage drives and if the drive containing proxmox fails, you will be left scrambling for a new installation.
Now there are multiple considerations here, the first one being that with the amount of logs generated by linux, it is not ideal to use an SSD for the host itself as it will eat up the write cycles, although personally I have used an SSD as my usecase requires it.
The second is redundancy: I have the host installed on an SSD with a ZFS mirror. In case one of my SSD fails, I have the second SSD containing a mirror of the host install.
You could achieve the same redundancy using a HDD ZFS mirror instead of an SSD.
Storage / Filesystem
The second most important aspect is storage and how you decide to lay it all out. But before that I want to introduce some concepts here
RAID / ZFS
RAID is simply a way to get multiple drives to work together in different configurations for different use cases. For example, if the data is important, you want the data available on multiple drives, so when one drive fails, you still have the data available on another drive. However, if using ZFS, your RAM requirements will go up.
Also depending on the configuration, if a ZFS rebuild fails due to another drive failing during rebuild. you lose your data.
You’re also limited to combining multiple drives of the same size and type.
Snapraid
A similar concept to RAID is snapraid, but whereas RAID works as write-on-write (meaning writing to one drive will write to the other), snapraid requires manual sync between drives, a process that can be automated. This comes with the benefit of not requiring as much memory as ZFS.
MergerFS
This allows you to use multiple drives (that may or may not be the same size) to be used as one volume.
Say you have 3 drives, 1TB, 2TB, 3TB which are labeled disk0, disk1, disk2.
With MergerFS, you can combine the drives into one, mounted as /mnt/storage that will have 6TB in total. All the data you write will be to /mnt/storage and it will automatically be distributed among the drives. Any one drive failing will not affect the other ones.
For simple media storage, this is ideal as it gives you one big drive to point your storage to without the hardware requirements or reduced capacity to ZFS.
It also gives you the flexibility to add or remove drives after the fact, while doing the same in ZFS is not as straightforward.
MergerFS + Snapraid
You can combine these two concepts above to get the redundancy as you require but it will obviously require a little more setup.
It is very important to note here that RAID is not backup. It is redundancy in case of drive failure.
So a very generic config would be
- 512GB/1TB — SSD/HDD ZFS Mirror for the Host
- X-TB — RaidZ1/2/3 for important data that you want redundancy on
- X-TB — MergerFS pool for media storage / unimportant data.
Network
Highly dependent on the use case, but personally I have separated my proxmox host on a different subnet than the rest of the house while only a few services like Plex and Pihole are available on the main network. This would require very basic networking know-how and a capable router.
Backups
With the steep learning curve and the amount of time it takes to set everything up, you need a backup strategy.
One option is to have local backup to the drives already connected to the proxmox server. This is fine but ideally you want redundant backups that are on a separate machine.
Setting up Proxmox Backup Server is quite straight forward. I have it setup on VirtualBox in another machine that is usually also on. This gives me multiple backups in case something fails in the first one and the backups have come in quite handy in a few instances, so its an absolute must to have a good backup strategy.
Hardware
Also highly dependent on you but if you plan to use it as a Plex Media Server, you can not go wrong with a GPU. The GPU can be passed through to your VM / LXC and allow better transcoding performance.
LXCs have a very small memory footprint but even the simplest debian desktop VM will eat up RAM and a windows VM will not be very happy with anything less than 8GB. So it is ideal to have adequate memory.
DNS + Reverse Proxy
If you would like to not remember all the IPs in the world along with their port numbers, a combination of Pihole and Traefik can give you the ability to locate your services using your own domain name with SSL.
Instead of having to access plex using 192.168.0.100:32400, you can use plex.yourdomain.com to access services hosted on your local network. Quite nice tbh.
Templates
You’ll probably need to setup VMs/LXCs quite a few times in case something goes wrong, so its best to create a barebones LXC/VM with the least amount of resources and convert it to a template.
This renders it unusable but it allows you to make full clones that will have all the necessary setup already done and get you up and running instantly.
Unprivilged Containers
You want to use unpriviliged LXCs for everything unless a VM is absolutely necessary. Unprivileged LXCs are isolated from the Proxmox host with access only to resources you make available to it. Your PiHole service, for example, has no business having access to the documents stored on your hard drive.
UPS Backup
With the cost that comes for the hardware and the time it takes to set everything up, you want your hardware, especially your hard drives to have the best chance at survival.
A UPS Backup is ideal, especially when you use a Smart UPS that you can connect to your host and program a graceful shutdown in case battery is running low. Look into NUPS/NUT for this and research on the capabilities of the UPS model before you buy one.
The learning curve is quite steep here but the end result is worth it, especially if you deal with a lot of data. This list is nowhere exhaustive but it should help you ask the right questions before you get started and guide you towards a path that saves you time and money.