You don't want to double encrypt the traffic (PCoIP or Blast) using a VPN. You should deploy a UAG or at a minimum security server and connect through that.
I'd also recommend right sizing your connection server and VM. Are these on the same host and do any other VMs exist on the host (This could be a ready time issue)? I have two connection servers handling thousands of connections a day and they only have 2 vCPU and 10 GB of memory. For your VM I would start with 2 vCPU and scale up as needed.