Strange parallel behavior noted on Linux kernel sources wrt CPU threading
Hello, I have been trying to figure out why I see an odd behavior building Linux kernel sources that I don't see when building gcc sources. It's not a hardware issue, I've ruled that out with various testing and bios settings and monitoring clock speeds, etc.
Basically, on a 6-core/12-thread processor, building openSUSE Tumbleweed 4.17.2 kernel on a bleeding edge Tumbleweed Linux system:
1) HT on, make -j12, 13 minutes. (good)
2) HT off, make -j12, 15 minutes. (good)
3) HT off, make -j6, 17 minutes. (good)
4) HT on, make -j6, 42-51 minutes. (whoa! problem!)
Watching top, it seems to me that make simply isn't issuing the same number of compile jobs in parallel for the last case. I'm observing this both on Intel 8700K and Ryzen 1500X (with -j4). I can either boot with HT on or off, or switch the cpu threads online or offline via /sys, with the same results. Somehow make is not behaving the same depending on how many cpu threads are available, despite -j flag.
Here's the proof that the problem is make. I can launch make -j4 with half the threads offline (case 3). *Then* immediately bring the other 4 cpu threads back online after make is running (case 4). Then the kernel build is just as fast as case 3. So make 4.2.1 has to be doing something different depending on nprocs during make initialization. Launching with -j6 with 12 hardware threads available results in abysmal make performance, with Linux kernel sources.
How to debug why make is going off into the weeds like this?