Help with linux task schuduler at 4.18, please
#1
Hi all

Is there anyone here who understands how to "tune" the Linux task scheduler? My problem is that at 4.18.9 it does not obviously favour the big cores for CPU heavy jobs which is crazy.

I wrote a little script to demonstrate what I mean - it uses an additional package stress as an example that can spawn n jobs for t seconds. In my case I run the load for 10 seconds, and after 5 seconds have a look which CPUs are in use:

Code:
stress -c 1 -t 10 &
sleep 5
ps -L -o pid,lwp,pcpu,cpuid,time
sleep 6
stress -c 2 -t 10 &
sleep 5
ps -L -o pid,lwp,pcpu,cpuid,time
sleep 6
stress -c 3 -t 10 &
sleep 5
ps -o pid,lwp,pcpu,cpuid,time
sleep 6
stress -c 4 -t 10 &
sleep 5
ps -L -o pid,lwp,pcpu,cpuid,time

On the 4.4.138 (Ayufan) kernel things are pretty much to be expected - CPUs 4 & 5 (the big A72 cores) are always used:
Code:
$ ./testcpu
stress: info: [1043] dispatching hogs: 1 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
  741   741  0.0     4 00:00:00
 1042  1042  0.0     0 00:00:00
 1043  1043  0.0     4 00:00:00
 1045  1045  100     4 00:00:05
 1047  1047  0.0     5 00:00:00
stress: info: [1043] successful run completed in 10s
stress: info: [1050] dispatching hogs: 2 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
  741   741  0.0     4 00:00:00
 1042  1042  0.0     5 00:00:00
 1050  1050  0.0     5 00:00:00
 1052  1052 98.6     5 00:00:04
 1053  1053  100     4 00:00:05
 1055  1055  0.0     0 00:00:00
stress: info: [1050] successful run completed in 10s
stress: info: [1058] dispatching hogs: 3 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
  741   741  0.0     4 00:00:00
 1042  1042  0.0     5 00:00:00
 1058  1058  0.0     4 00:00:00
 1060  1060 99.8     0 00:00:04
 1061  1061  100     5 00:00:05
 1062  1062  100     4 00:00:05
 1064  1064  0.0     3 00:00:00
stress: info: [1058] successful run completed in 10s
stress: info: [1067] dispatching hogs: 4 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
  741   741  0.0     4 00:00:00
 1042  1042  0.0     4 00:00:00
 1067  1067  0.0     2 00:00:00
 1069  1069  100     1 00:00:05
 1070  1070  100     4 00:00:05
 1071  1071  100     3 00:00:05
 1072  1072  100     5 00:00:05
 1074  1074  0.0     0 00:00:00
stress: info: [1067] successful run completed in 10s

On 4.18.9 (despite all my efforts configuring the kernel) it is more luck whether CPUs 4 or 5 do any work:

Code:
$ ./testcpu
stress: info: [2375] dispatching hogs: 1 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 2354  2354  0.1     3 00:00:00
 2374  2374  0.0     0 00:00:00
 2375  2375  0.0     4 00:00:00
 2377  2377 83.5     2 00:00:05
 2379  2379  1.0     5 00:00:00
stress: info: [2375] successful run completed in 10s
stress: info: [2382] dispatching hogs: 2 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 2354  2354  0.0     3 00:00:00
 2374  2374  0.0     0 00:00:00
 2382  2382  0.0     1 00:00:00
 2384  2384  100     3 00:00:05
 2385  2385  100     4 00:00:05
 2387  2387  0.0     0 00:00:00
stress: info: [2382] successful run completed in 10s
stress: info: [2390] dispatching hogs: 3 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 2354  2354  0.0     3 00:00:00
 2374  2374  0.0     1 00:00:00
 2390  2390  0.0     4 00:00:00
 2392  2392  100     3 00:00:05
 2393  2393  100     5 00:00:05
 2394  2394  100     2 00:00:05
 2396  2396  0.0     0 00:00:00
stress: info: [2390] successful run completed in 10s
stress: info: [2399] dispatching hogs: 4 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 2354  2354  0.0     3 00:00:00
 2374  2374  0.0     3 00:00:00
 2399  2399  0.0     5 00:00:00
 2401  2401  100     0 00:00:05
 2402  2402  100     4 00:00:05
 2403  2403  100     5 00:00:05
 2404  2404  100     2 00:00:05
 2406  2406  0.0     1 00:00:00
stress: info: [2399] successful run completed in 10s

There does seem to be a kernel CONFIG option at 4.4, ARM_ROCKCHIP_CPUFREQ, that could explain the 4.4 behaviour. And is not available at 4.18.9. But in reality I would expect stock/mainline 4.18 scheduling to cope properly with big.LITTLE CPUs?

Any thoughts/suggestions welcomed.
  • ROCKPro64 v2.1 2GB, 16Gb eMMC for rootfs, SX8200Pro 512GB NVMe for /home, HDMI video & sound, Bluetooth keyboard & mouse. Arch (6.2 kernel, Openbox desktop) for general purpose daily PC.
  • PinePhone Pro Explorer Edition, daily driver, rk2aw & U-boot on SPI, Arch/SXMO & Arch/phosh on eMMC
  • PinePhone BraveHeart now v1.2b 3/32Gb, Tow-boot with Arch/SXMO on eMMC
#2
Same problem with 4.19.0-rc4, with 1 hog it runs on an A53 core (CPU 1), with 3 hogs two are on A53s and only 1 on an A72 (CPU 4):

Code:
$ uname -a
Linux rpro64.dukla.net 4.19.0-rc4-1071-ayufan-g10a63ec6c2a2 #1 SMP PREEMPT Mon Oct 1 07:33:40 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux
chris@rpro64:~/bin/Rpro64$ ./testcpu
stress: info: [3067] dispatching hogs: 1 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 2736  2736  0.0     2 00:00:00
 3066  3066  0.0     3 00:00:00
 3067  3067  0.0     4 00:00:00
 3069  3069  100     1 00:00:05
 3071  3071  0.0     4 00:00:00
stress: info: [3067] successful run completed in 10s
stress: info: [3075] dispatching hogs: 2 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 2736  2736  0.0     2 00:00:00
 3066  3066  0.0     1 00:00:00
 3075  3075  0.0     4 00:00:00
 3077  3077  100     4 00:00:05
 3078  3078  100     0 00:00:05
 3081  3081  0.0     1 00:00:00
stress: info: [3075] successful run completed in 10s
stress: info: [3085] dispatching hogs: 3 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 2736  2736  0.0     2 00:00:00
 3066  3066  0.0     2 00:00:00
 3085  3085  0.0     4 00:00:00
 3087  3087 98.4     3 00:00:04
 3088  3088 98.4     4 00:00:04
 3089  3089 98.4     0 00:00:04
 3091  3091  0.0     2 00:00:00
stress: info: [3085] successful run completed in 10s
stress: info: [3094] dispatching hogs: 4 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 2736  2736  0.0     2 00:00:00
 3066  3066  0.0     4 00:00:00
 3094  3094  0.0     2 00:00:00
 3096  3096 98.8     4 00:00:04
 3097  3097 98.6     5 00:00:04
 3098  3098 98.8     1 00:00:04
 3099  3099 99.0     2 00:00:04
 3101  3101  0.0     3 00:00:00
  • ROCKPro64 v2.1 2GB, 16Gb eMMC for rootfs, SX8200Pro 512GB NVMe for /home, HDMI video & sound, Bluetooth keyboard & mouse. Arch (6.2 kernel, Openbox desktop) for general purpose daily PC.
  • PinePhone Pro Explorer Edition, daily driver, rk2aw & U-boot on SPI, Arch/SXMO & Arch/phosh on eMMC
  • PinePhone BraveHeart now v1.2b 3/32Gb, Tow-boot with Arch/SXMO on eMMC
#3
on 4.4 when turning off CONFIG_ARM_ROCKCHIP_CPUFREQ and
setting CONFIG_ARM_BIG_LITTLE_CPUFREQ

I get the same cpu-core preference for  cpu4 and cpu5 (A72)

Is this set on your 4.19?


Code:
#
# CPU frequency scaling drivers
#
CONFIG_CPUFREQ_DT=y
CONFIG_ARM_BIG_LITTLE_CPUFREQ=y
CONFIG_ARM_DT_BL_CPUFREQ=y
# CONFIG_ARM_KIRKWOOD_CPUFREQ is not set
# CONFIG_ARM_ROCKCHIP_CPUFREQ is not set



Code:
$ ./testcpu
stress: info: [1561] dispatching hogs: 1 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 1475  1475  0.0     4 00:00:00
 1560  1560  0.0     4 00:00:00
 1561  1561  0.0     1 00:00:00
 1563  1563  100     5 00:00:05
 1564  1564  0.0     4 00:00:00
stress: info: [1561] successful run completed in 10s
stress: info: [1566] dispatching hogs: 2 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 1475  1475  0.0     4 00:00:00
 1560  1560  0.0     0 00:00:00
 1566  1566  0.0     4 00:00:00
 1568  1568 99.6     5 00:00:04
 1569  1569 99.8     4 00:00:04
 1570  1570  0.0     1 00:00:00
stress: info: [1566] successful run completed in 10s
stress: info: [1572] dispatching hogs: 3 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 1475  1475  0.0     4 00:00:00
 1560  1560  0.0     0 00:00:00
 1572  1572  0.0     1 00:00:00
 1574  1574  100     4 00:00:05
 1575  1575  100     3 00:00:05
 1576  1576  100     5 00:00:05
 1577  1577  0.0     1 00:00:00
stress: info: [1572] successful run completed in 10s
stress: info: [1580] dispatching hogs: 4 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 1475  1475  0.0     4 00:00:00
 1560  1560  0.0     3 00:00:00
 1580  1580  0.0     4 00:00:00
 1582  1582  100     5 00:00:05
 1583  1583 99.8     1 00:00:04
 1584  1584  100     4 00:00:05
 1585  1585  100     2 00:00:05
 1586  1586  0.0     0 00:00:00
stress: info: [1580] successful run completed in 10s
#4
(10-05-2018, 03:20 AM)hunderteins Wrote: on 4.4 when turning off CONFIG_ARM_ROCKCHIP_CPUFREQ and
setting CONFIG_ARM_BIG_LITTLE_CPUFREQ

I get the same cpu-core preference for  cpu4 and cpu5 (A72)

Is this set on your 4.19?

That is really interesting, thanks.

On both 4.18 and 4.18 I have tried both BIG_LITTLE_CPUFREQ and BL_CPUFREQ 

Code:
#
# CPU frequency scaling drivers
#
CONFIG_CPUFREQ_DT=y
CONFIG_CPUFREQ_DT_PLATDEV=y
# CONFIG_ACPI_CPPC_CPUFREQ is not set
CONFIG_ARM_BIG_LITTLE_CPUFREQ=y
CONFIG_ARM_DT_BL_CPUFREQ=y
# CONFIG_ARM_SCPI_CPUFREQ is not set
# CONFIG_QORIQ_CPUFREQ is not set
CONFIG_NET=y
CONFIG_NET_INGRESS=y

with no success. My thinking is that by now big.LITTLE is old enough that mainline support should be there. But I still cannot discover what I have "wrong".
  • ROCKPro64 v2.1 2GB, 16Gb eMMC for rootfs, SX8200Pro 512GB NVMe for /home, HDMI video & sound, Bluetooth keyboard & mouse. Arch (6.2 kernel, Openbox desktop) for general purpose daily PC.
  • PinePhone Pro Explorer Edition, daily driver, rk2aw & U-boot on SPI, Arch/SXMO & Arch/phosh on eMMC
  • PinePhone BraveHeart now v1.2b 3/32Gb, Tow-boot with Arch/SXMO on eMMC


Possibly Related Threads…
Thread Author Replies Views Last Post
  irradium (based on crux linux) RockPro64 riscv64, aarch64 mara 7 1,876 11-20-2024, 03:53 PM
Last Post: mara
Exclamation Ethernet regression on Linux Kernel 6.5.4? Deathcrow 3 1,497 09-22-2023, 04:27 AM
Last Post: diederik
  Installing CH431SER on Ayufan 0.9.14: gitlab-ci-linux-build-159 Thisone 4 2,287 07-14-2023, 04:22 AM
Last Post: hunderteins
  Linux laptop does not detect the board when plugged in via USB soupy 1 4,490 04-13-2023, 03:01 AM
Last Post: Reynold Grady
  RockPro64 linux console video mode callegar 0 1,135 09-06-2022, 02:32 PM
Last Post: callegar
Brick Maintained Linux booting from eMMC ootoovak 10 9,912 04-30-2022, 03:57 PM
Last Post: TRS-80
  How I Got Fedora Linux to Boot From eMMC (or microSD, for that matter) whitecat23 4 5,372 01-03-2022, 10:32 AM
Last Post: whitecat23
  Other boot options than Sdcard for linux 5.x kernel? MisterA 2 3,528 07-14-2021, 02:37 PM
Last Post: TRS-80
  Booting Linux/Debian from the eMMC linuxha 4 5,943 03-02-2021, 07:01 PM
Last Post: linuxha
  Xen EFI with Alpine Linux dom0 sir.lami 0 2,751 08-26-2020, 12:25 PM
Last Post: sir.lami

Forum Jump:


Users browsing this thread: 1 Guest(s)