I was interested in buying the latest Zhaoxin KX-7000 CPU for quite some time, which I see as a modified variation of the last unreleased Centaur CNS CPU. It is not exactly the same, but both were definitely designed in some kind of collaborative fashion. It will likely be the last one, since the Centaur team has now been disbanded.
There were a few options in the past, mainly desktop-oriented motherboards, but their prices were too high for me to consider (and a desktop motherboard is not what I was looking for). Luckily, more options appeared recently, and I got lucky enough to buy Aiostar AOS-MMZX700D (AOS-MMZX7000D on the motherboard sticker) in the Mini-ITX form-factor for a pretty decent price. My variant is very slightly customized compared to the default configuration, having one VGA connector and one HDMI connector instead of two HDMI ports.
Performance-wise, the KX-7000 CPU is not very impressive compared to the competition, and even the motherboard manufacturer admits this in the product description, but it is much more capable than anything before it and pretty capable overall (based on the resources available online). The motherboard is positioned more toward embedded/industrial usage (for example, it doesn’t have an actual PCIe slot, uses a DC-IN power adapter, and provides a SIM slot). However, I personally am less interested in its positioning and performance. The strongest reason was to continue improving Centaur support in NetBSD itself, and probably utilize the motherboard for internal purposes, such as upgrading my NAS server, or having a decent and compact AMD64 testing platform if successful.
And indeed, this platform has provided quite a number of various things to fix and improve; some easy, some still unresolved. I will go through some of them in this article!
Desired number of cache colors is not even
The first issue surfaced on the first boot of the NetBSD operating system. I hit a relatively early panic in the boot process that was saying, “desired number of cache colors 21 is > 1, but not even!” The error message was pointing to this line.
Page coloring is how the kernel groups physical pages by which cache sets they map to, so pages with the same “color” compete for the same cache lines. The kernel calculates the number of colors as cache size / (page size × associativity). So something was off in this calculation.
Fortunately, this mystery was relatively easy to crack and revealed two problems in the VIA/Zhaoxin CPU cache identification. The first one is a fairly longstanding issue, and in theory it affected all VIA CPUs starting with Nehemiah. The other one is more recent and is related to the way cache parameters should be decoded in general.
The uneven number was caused by the first issue. Historically, L2 cache parameters are decoded using the 0x80000006 CPUID function. Certain bits encode the cache size, line size, and associativity. This matches the AMD way of doing the same thing for the L2 cache.
However, the NetBSD implementation had a subtle difference between AMD and Centaur when decoding cache associativity. AMD was using a lookup to resolve the value extracted from the bits, while Centaur was using the raw value directly. This difference was likely backed up by VIA datasheets, which never mentioned any lookup table and described the bits as containing the actual value.
In practice, it turned out that they also store the same value as AMD, meaning that a lookup table is required to obtain the actual value!
This issue had never been noticed because the number of colors happened to still be a power of 2, even if it was calculated incorrectly for certain models. It is the KX-7000, which has 8-way L2 associativity, that finally exposed the issue, since 6 is the raw value and the calculation therefore failed to pass the power-of-2 check.
The second issue affected newer VIA (CNQ+) and most, if not all, Zhaoxin CPUs. It appeared that, at some point, it was decided to switch to Intel’s way of encoding cache parameters through CPUID leaf 4 instead. The parameters mostly match between the two methods (though there were cases where they disagreed), but it became the preferred way of obtaining this information. Additionally, the L3 cache is encoded only this way in the KX-7000 CPU, so CPUID leaf 4 needs to be used to obtain its L3 cache parameters.
The fix can be described in two steps:
- Use the same cache-associativity lookup table as AMD.
The raw associativity value reported by the CPU needs to be translated using the lookup table instead of being treated as the final value. - Parse cache information from CPUID leaf 4 when available.
Newer VIA and Zhaoxin CPUs use the Intel-style CPUID leaf 4 to report cache parameters, including the L3 cache on the KX-7000. The code therefore uses leaf 4 to obtain the cache information when it is available.
Same changes were applied to cpuctl(8) utility as well.
SMAP and PMAP
KX-7000 supports SMAP feature designed to set user-space mapping to prevent supervisor access to those mapping would cause a trap. Once I fixed the issue above, the boot process still failed and hit page fault trap in pmap_enter_ma() somewhere before login screen was supposed to appear. Quite quickly I identified that SMAP feature is causing this, thus I committed a workaround to disable it for the Zhaoxin CPUs (it is currently supported at least by KX-7000 and KX-6000G, and maybe KX-6900). I was hoping to quickly identify the root cause, however I haven’t reached this stage yet. Later I identified that it affects only newly installed pmap pages and PTE entries. I found that invlpg instruction (invalidating TLB caches) in couple places allows the system boot properly, but those places can affect performance, and not necessarily correct, thus I couldn’t apply them. There are few more things which affect the outcome, but none solve the problem. So far, this issue is not fully resolved, but disabling SMAP helps and this is currently applied. For whatever reason i386 is not affected in the same way and boots successfully (either SMAP is not used or recursive mapping is more shallow than the one in amd64). OpenBSD seems to be affected by this problem too. FreeBSD boots successfully though.
WARNING: no TOD clock present
After disabling SMAP, the system finally booted fully! However, I immediately noticed a warning at the end of the boot process: “WARNING: no TOD clock present”, followed by a couple more related warnings. After some investigation, it turned out to be related to a quite recent change for virtualized environments, specifically QEMU microvm boot, which needed a check that the RTC (real-time clock) is enabled, as otherwise the boot would hang. However, that check was not valid for real hardware. It used the MC146818 Register D bits to see whether bits 0-6 were zero, but real hardware may use those bits for the date alarm, so they are not necessarily zero. This is not specific to the KX-7000: any machine with non-zero values there could have its RTC mistakenly not attached. The KX-7000 system just happened to be one such case. The fix restricts the check to VM guest environments.
Legacy boot issue
The motherboard has an option to use either EFI boot or legacy BIOS boot. Oddly, it doesn’t allow both: one must be chosen in the BIOS setup (other motherboards can typically support both without a specific switch). Nevertheless, I tried to boot NetBSD using BIOS boot (specifically the i386 and amd64 BIOS-only images). Both failed to load the kernel, ending in “read section: input/output error” or “inappropriate file type or format” errors, or even an abrupt reboot. Experience from the VIA C7 boot issue investigation helped me get into bootloader debugging fairly quickly. After several sessions I found that forcing the INT13 extensions resolves the problem (#define FORCE_INT13EXT can be used to build a bootloader that enforces this). The INT13 extensions provide LBA-based disk access, while NetBSD uses CHS (Cylinder, Head, Sector) addressing for reads up to a certain sector and only switches to LBA beyond what CHS can address. For some reason, the CHS approach doesn’t work properly on the KX-7000. I proposed a fix to always use the INT13 extensions if the BIOS reports supporting them, but the developers rejected it, because some other legacy systems may have the opposite problem (support is reported but doesn’t work, or a SCSI ROM doesn’t support it at all). I plan to continue with a different approach, probably reading a certain sector and checking whether LBA can be used, but it is not ready yet. For now, the only option is to enforce the extensions, which requires rebuilding the bootloader. OpenBSD and FreeBSD don’t have this issue, since they don’t do CHS reads if INT13 extensions are supported.
HDMI output issue on BIOS boot
The motherboard has both VGA and HDMI outputs, and I typically use VGA for testing. However, once I managed to boot into the system using BIOS boot, I tried switching to HDMI output instead, and in that case the boot hangs in a not fully deterministic way, meaning not always on the same line. I tried disabling various devices and disabling vesa, which lets the boot move further, but it never completes. OpenBSD is potentially affected too: its kernel boots fully, but it hangs while starting userland services (which doesn’t happen with VGA output). It looks like a memory, framebuffer, or timeout issue, but I have little idea how to investigate it. EFI boot is not affected in the same way and boots without problems even when HDMI output is used.
Other smaller improvements
Besides the improvements above, I also committed a couple more changes related to VIA/Zhaoxin CPUs. Specifically, I added the Centaur vendor ID and family 7 to the invariant TSC check (needed for the KX-7000), which is performed through the CPUID 0x80000007 function. Additionally, I added a check for using lfence to serialize rdtsc on all CPUs from VIA Nano onwards, including all Zhaoxin CPUs. I also updated the pcidevs and hdaudiodevs files to include the PCI IDs and HD audio device IDs from the KX-7000 (and other VIA/Zhaoxin) CPUs. Finally, I added an extended model check for family 7 CPUs, which are used only by Zhaoxin. This allows more accurate model reporting in cpuctl(8) and in kernel checks.
One more issue with BIOS boot is that consdev com0 / consdev com1 does not redirect the console to the serial port. I haven’t investigated this yet.
I will continue working on the platform improvements, but the pace will likely slow down now. The initial support is in place, and the workarounds needed to boot the system have been identified or fixed, which makes for a stable and usable system as is. I am requesting, or planning to request, pull-ups of most of these changes to at least NetBSD 11, and some of them have already been applied.