Archive for

Maximum supported vSAN/vSphere/VCF Cluster

So lets talk about “maximum supported” and why it’s just a weird thing to focus on. Example building the biggest vSAN cluster possibly supported. It is today ONLY 64 nodes, but lets unpack what I can run in 64 nodes.

vSAN 7 supports 32TB Capacity devices. 5 disk groups *7 devices that’s ~1.12PB per host of raw capacity (Please don’t do this without calling me, but hey it is there).

With AMD 2 socket servers 128 Cores currently. Intel 56 core that’s 224 threads (yes, I’m aware the 9200 may require it’s own nuke plant to run and cool) I’m going to ignore quad socket for this conversation but yes, we support quad sockets like Synergy 660 Gen10 for SAP HANA.

Maximum GPU’s served for a vSAN Cluster is actually a fun one. 16GPU per host, is going to need interesting cooling solutions…. or will it? https://configmax.vmware.com/guest?vmwareproduct=vSphere&release=vSphere%207.0&categories=2-0

BitFusion allows for remote CUDA calls to be served from remote hosts (and ones not even in the cluster). So GPU workload scaling potentially could get pretty nuts and I’m going to leave others to speculate on how many GPU cores could serve a cluster.

Maximum Memory is 16TB DRAM per host, and 12TB of PMEM per host. So a PB of DRAM, and 768TB of PMEM per cluster.

Now, addressable memory gets more fun as TPS, Memory ballooning, and DRS (with new cool capabilities in 7) mean that the actually allocated could be a bit higher. And when you are spending the price of what I assume a G5 Jet costs, that you are going to use these features. Should you design to these maximums? In general no. Most people have other reasons to split a cluster up (Remember we can do shared nothing migrations between clusters always). People will want to limit the blast zone of a management domain etc.

Part of the benefit of HCI is you can easily scale it down to 2 node even… Also remember Hosts can be reclaimed and moved to other workload domains in VMware Cloud Foundation.

Lastly, just because you can, doesn’t mean you should. Most sane people don’t need or want 40 drives in a host, or want an HA event to result in 6TB of memory worth of VM’s rebooting at once. Respect operational reasons to limit blast zones and sizing. As VMware makes it easier to migrate or share resources between clusters these kinds of limits matter less and less.

Audio Quality Testing – initial tests

This one’s a long time coming, and there will be more posts on this topic. Go ahead and start playing this playlist to hear the, good the bad and the ugly (It will only take a minute).

So I took some time today to do some testing using some microphones laying around.

Testing methodology:

1. Zoom (It’s common enough) and they use pretty good codecs that are rarely a bottleneck.
2. Local recording (I’ll do WAN impact simulations at another time).
3. I did use the Original microphone sound mode (Not all conference systems have noise suppression and it’s worth noting that noise suppression while good for many cases does reduce quality). I’ll test these capabilities with actual noise another day (Crying baby and firetruck simulator?).
4. I haven’t tested every microphone in this house yet (I’ve got a sure SM58, a steel series headset my wife uses, and i’m sure some other bluetooth devices).
5. I tried to not adjust the gain on any devices, or the audio input volume (Which shows on the analog earbuds which are way to quiet). Simulating someone joining a call late and in a hurry.
6. The AC came on at one point and I tried to note it, but it’s hard to hear it too much. There is a NAS and desktop (fairly quiet fans) and the laptop fan. I plan to do another test with music, firetruck/baby in background to simulate some of the COVID WFH lifestyles.
7. I ate dairy (tends to give me some sinus congestion) and drank a lot of diet Dr. pepper. Both of these always negatively impact my throat etc for speaking. Drinking water, standing, and avoiding dairy would make me speak better, but I was going for something more realistic.
8. I use the following test phrases:

Oak is strong and also gives shade.
Cats and dogs each hate the other.
The pipe began to rust while new.
Open the crate but don’t break the glass

Realistically it woulds be better to read something a lot more mundane, but these are known test phrases that have a diversity of sounds.

About what I tested:

1. The Heil PR-40 microphone was attached to a Blu USB-> XLR adapter with no XLR cable used. Gain was a bit high.
2. Razer Nari (Note I don’t have their crazy audio tools installed, like most software built by hardware companies I find it to be an absolute nightmare to use). These headphones use proprietary 2.4Ghz wireless. and not bluetooth.
3. Apple AirPod Pros. Note while they are connected to a Macbook Pro, I had 2 other bluetooth devices (keyboard, touch pad connected) so a non-optimal codec was likely used. Again, trying to simulate minimal effort real world, what a worse case Bluetooth codec would sound like. Remember the audio quality on Bluetooth is often 10x worse for the microphone than the sound input so just because you hear “good” sound doesn’t mean your transmitting it.
4. PSTN bridge. I dialed into the zoom call (not the app) using an iPhone 11 Pro. I tested this twice, once holding up to my ear and another on speaker phone laying on my desk. I should probably test using the app, but I wanted to simulate the ugly truth of what it sounds like when people use the dial in code under semi-optimal conditions. (I get good phone service, and wasn’t in a car with the AC blasting).

I’ll post some more information later (gotta run to the bank), but here’s a playlist of my initial tests.

Do you own tests:

1. By no means accept my testing as authoritative. Do your own test and customize for
2. The room you use
3. The devices you want to use
4. Try adjusting with volume, gain and other settings.
5. Try standing (it helps some people speak more clearly).
6. Have someone else listen to them.

Conclusions:

Audio quality is often the result of many factors that are out of our control like:

1. Bad wifi
2. cheap gear
3. background noise
4. People who still use Lync/Skype4Buisness.
5. Accents (Myles, if I have 2 trees, and add 1 tree how many trees do I have?)

Still despite this it’s worth knowing what you sound like and doing some quick tests to see if you can make yourself heard more readily. Being clearer on calls leads to less repeating things, more understanding and hopefully shorter/faster and more productive phone calls.

Lastly to the Managers out there. Talk to your reports about audio quality. Get people gear, get people an Ero WIFI bundle, reimburse better internet. A team who’s well heard is a team that is productive.

vSAN with the PERC H740 or H740P

This is going to be a quick blog post, as someone asked about if the PERC 740 is going to be certified, or if VMware is going to try to certify it. A few things to consider….

1. Dell already sells a lower power, small form factor HBA330 (13Gen) and HBA 330+ (14 Gen servers) that has ultra deep queue depths, is simple to configure (no configuration), is cheap (less than 1/2 the street price of the PERC 740), and more importantly is brilliantly stable.
2. VMware does not unilaterally certify devices. It would require the OEM (Dell) often working with the ODM for the ASIC (In this case Broadcom/Avago) to submit the device to the ReadyLabs for testing. This has not happened, and it is my understanding is likely never to happen for a device that is frankly inferior in every way (Cost, Stability of pass through, performance, heat, power) for the use case of a pass through device.
3. NVMe devices do not need HBAs (the controller is built into the drive). Longer term as All Flash vSAN evolves, I expect low cost NVMe to “beat” the price of SAS/SATA Read Optimized flash drives plus the overhead of even a $250 cheap HBA.

But John, what about RAID for my boot devices?

I’m glad you asked! VMware has updated our boot device requirements with vSphere 7 and for Dell the BOSS mirrored M.2 solution provides a great blend of endurance, affordability, and fault isolation for boot, crash dumps, and log placement.

But what if my VAR rep said it’s ok to use?

Well beyond them being wrong, if you try using it you will encounter quite a few issues. vSAN health alarm checks will detect a non-supported controller and throw angry alarms. It’s not exactly easy to “sneak” into production with the wrong controller, vSAN Health will light up like a Christmas tree at you. On top of this you will not be able to life cycle the controller to a supported driver/firmware version using vLCM. VMware will not support the configuration (obviously). It’s worth noting that buying “ReadyNode” SKU’s from Dell (Chassis personality codes that end in -RN) will block this configuration entirely from being built in the factory.

If this happens to you feel free to reach out and I’ll happily introduce you to your vSAN account team, and the Dell ReadyNode teams who can help set the record straight.