Help with linux task schuduler at 4.18, please
#1
Hi all

Is there anyone here who understands how to "tune" the Linux task scheduler? My problem is that at 4.18.9 it does not obviously favour the big cores for CPU heavy jobs which is crazy.

I wrote a little script to demonstrate what I mean - it uses an additional package stress as an example that can spawn n jobs for t seconds. In my case I run the load for 10 seconds, and after 5 seconds have a look which CPUs are in use:

Code:
stress -c 1 -t 10 &
sleep 5
ps -L -o pid,lwp,pcpu,cpuid,time
sleep 6
stress -c 2 -t 10 &
sleep 5
ps -L -o pid,lwp,pcpu,cpuid,time
sleep 6
stress -c 3 -t 10 &
sleep 5
ps -o pid,lwp,pcpu,cpuid,time
sleep 6
stress -c 4 -t 10 &
sleep 5
ps -L -o pid,lwp,pcpu,cpuid,time

On the 4.4.138 (Ayufan) kernel things are pretty much to be expected - CPUs 4 & 5 (the big A72 cores) are always used:
Code:
$ ./testcpu
stress: info: [1043] dispatching hogs: 1 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
  741   741  0.0     4 00:00:00
 1042  1042  0.0     0 00:00:00
 1043  1043  0.0     4 00:00:00
 1045  1045  100     4 00:00:05
 1047  1047  0.0     5 00:00:00
stress: info: [1043] successful run completed in 10s
stress: info: [1050] dispatching hogs: 2 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
  741   741  0.0     4 00:00:00
 1042  1042  0.0     5 00:00:00
 1050  1050  0.0     5 00:00:00
 1052  1052 98.6     5 00:00:04
 1053  1053  100     4 00:00:05
 1055  1055  0.0     0 00:00:00
stress: info: [1050] successful run completed in 10s
stress: info: [1058] dispatching hogs: 3 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
  741   741  0.0     4 00:00:00
 1042  1042  0.0     5 00:00:00
 1058  1058  0.0     4 00:00:00
 1060  1060 99.8     0 00:00:04
 1061  1061  100     5 00:00:05
 1062  1062  100     4 00:00:05
 1064  1064  0.0     3 00:00:00
stress: info: [1058] successful run completed in 10s
stress: info: [1067] dispatching hogs: 4 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
  741   741  0.0     4 00:00:00
 1042  1042  0.0     4 00:00:00
 1067  1067  0.0     2 00:00:00
 1069  1069  100     1 00:00:05
 1070  1070  100     4 00:00:05
 1071  1071  100     3 00:00:05
 1072  1072  100     5 00:00:05
 1074  1074  0.0     0 00:00:00
stress: info: [1067] successful run completed in 10s

On 4.18.9 (despite all my efforts configuring the kernel) it is more luck whether CPUs 4 or 5 do any work:

Code:
$ ./testcpu
stress: info: [2375] dispatching hogs: 1 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 2354  2354  0.1     3 00:00:00
 2374  2374  0.0     0 00:00:00
 2375  2375  0.0     4 00:00:00
 2377  2377 83.5     2 00:00:05
 2379  2379  1.0     5 00:00:00
stress: info: [2375] successful run completed in 10s
stress: info: [2382] dispatching hogs: 2 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 2354  2354  0.0     3 00:00:00
 2374  2374  0.0     0 00:00:00
 2382  2382  0.0     1 00:00:00
 2384  2384  100     3 00:00:05
 2385  2385  100     4 00:00:05
 2387  2387  0.0     0 00:00:00
stress: info: [2382] successful run completed in 10s
stress: info: [2390] dispatching hogs: 3 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 2354  2354  0.0     3 00:00:00
 2374  2374  0.0     1 00:00:00
 2390  2390  0.0     4 00:00:00
 2392  2392  100     3 00:00:05
 2393  2393  100     5 00:00:05
 2394  2394  100     2 00:00:05
 2396  2396  0.0     0 00:00:00
stress: info: [2390] successful run completed in 10s
stress: info: [2399] dispatching hogs: 4 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 2354  2354  0.0     3 00:00:00
 2374  2374  0.0     3 00:00:00
 2399  2399  0.0     5 00:00:00
 2401  2401  100     0 00:00:05
 2402  2402  100     4 00:00:05
 2403  2403  100     5 00:00:05
 2404  2404  100     2 00:00:05
 2406  2406  0.0     1 00:00:00
stress: info: [2399] successful run completed in 10s

There does seem to be a kernel CONFIG option at 4.4, ARM_ROCKCHIP_CPUFREQ, that could explain the 4.4 behaviour. And is not available at 4.18.9. But in reality I would expect stock/mainline 4.18 scheduling to cope properly with big.LITTLE CPUs?

Any thoughts/suggestions welcomed.
ROCKPro64 v2.1 2GB, SM961 128GB NVMe for rootfs, HDMI video & sound, Bluetooth keyboard & mouse
Started Bionic minimal - now Cosmic, Openbox desktop for general purpose daily PC.
  Reply
#2
Same problem with 4.19.0-rc4, with 1 hog it runs on an A53 core (CPU 1), with 3 hogs two are on A53s and only 1 on an A72 (CPU 4):

Code:
$ uname -a
Linux rpro64.dukla.net 4.19.0-rc4-1071-ayufan-g10a63ec6c2a2 #1 SMP PREEMPT Mon Oct 1 07:33:40 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux
[email protected]:~/bin/Rpro64$ ./testcpu
stress: info: [3067] dispatching hogs: 1 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 2736  2736  0.0     2 00:00:00
 3066  3066  0.0     3 00:00:00
 3067  3067  0.0     4 00:00:00
 3069  3069  100     1 00:00:05
 3071  3071  0.0     4 00:00:00
stress: info: [3067] successful run completed in 10s
stress: info: [3075] dispatching hogs: 2 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 2736  2736  0.0     2 00:00:00
 3066  3066  0.0     1 00:00:00
 3075  3075  0.0     4 00:00:00
 3077  3077  100     4 00:00:05
 3078  3078  100     0 00:00:05
 3081  3081  0.0     1 00:00:00
stress: info: [3075] successful run completed in 10s
stress: info: [3085] dispatching hogs: 3 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 2736  2736  0.0     2 00:00:00
 3066  3066  0.0     2 00:00:00
 3085  3085  0.0     4 00:00:00
 3087  3087 98.4     3 00:00:04
 3088  3088 98.4     4 00:00:04
 3089  3089 98.4     0 00:00:04
 3091  3091  0.0     2 00:00:00
stress: info: [3085] successful run completed in 10s
stress: info: [3094] dispatching hogs: 4 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 2736  2736  0.0     2 00:00:00
 3066  3066  0.0     4 00:00:00
 3094  3094  0.0     2 00:00:00
 3096  3096 98.8     4 00:00:04
 3097  3097 98.6     5 00:00:04
 3098  3098 98.8     1 00:00:04
 3099  3099 99.0     2 00:00:04
 3101  3101  0.0     3 00:00:00
ROCKPro64 v2.1 2GB, SM961 128GB NVMe for rootfs, HDMI video & sound, Bluetooth keyboard & mouse
Started Bionic minimal - now Cosmic, Openbox desktop for general purpose daily PC.
  Reply
#3
on 4.4 when turning off CONFIG_ARM_ROCKCHIP_CPUFREQ and
setting CONFIG_ARM_BIG_LITTLE_CPUFREQ

I get the same cpu-core preference for  cpu4 and cpu5 (A72)

Is this set on your 4.19?


Code:
#
# CPU frequency scaling drivers
#
CONFIG_CPUFREQ_DT=y
CONFIG_ARM_BIG_LITTLE_CPUFREQ=y
CONFIG_ARM_DT_BL_CPUFREQ=y
# CONFIG_ARM_KIRKWOOD_CPUFREQ is not set
# CONFIG_ARM_ROCKCHIP_CPUFREQ is not set



Code:
$ ./testcpu
stress: info: [1561] dispatching hogs: 1 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 1475  1475  0.0     4 00:00:00
 1560  1560  0.0     4 00:00:00
 1561  1561  0.0     1 00:00:00
 1563  1563  100     5 00:00:05
 1564  1564  0.0     4 00:00:00
stress: info: [1561] successful run completed in 10s
stress: info: [1566] dispatching hogs: 2 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 1475  1475  0.0     4 00:00:00
 1560  1560  0.0     0 00:00:00
 1566  1566  0.0     4 00:00:00
 1568  1568 99.6     5 00:00:04
 1569  1569 99.8     4 00:00:04
 1570  1570  0.0     1 00:00:00
stress: info: [1566] successful run completed in 10s
stress: info: [1572] dispatching hogs: 3 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 1475  1475  0.0     4 00:00:00
 1560  1560  0.0     0 00:00:00
 1572  1572  0.0     1 00:00:00
 1574  1574  100     4 00:00:05
 1575  1575  100     3 00:00:05
 1576  1576  100     5 00:00:05
 1577  1577  0.0     1 00:00:00
stress: info: [1572] successful run completed in 10s
stress: info: [1580] dispatching hogs: 4 cpu, 0 io, 0 vm, 0 hdd
  PID   LWP %CPU CPUID     TIME
 1475  1475  0.0     4 00:00:00
 1560  1560  0.0     3 00:00:00
 1580  1580  0.0     4 00:00:00
 1582  1582  100     5 00:00:05
 1583  1583 99.8     1 00:00:04
 1584  1584  100     4 00:00:05
 1585  1585  100     2 00:00:05
 1586  1586  0.0     0 00:00:00
stress: info: [1580] successful run completed in 10s
  Reply
#4
(10-05-2018, 03:20 AM)hunderteins Wrote: on 4.4 when turning off CONFIG_ARM_ROCKCHIP_CPUFREQ and
setting CONFIG_ARM_BIG_LITTLE_CPUFREQ

I get the same cpu-core preference for  cpu4 and cpu5 (A72)

Is this set on your 4.19?

That is really interesting, thanks.

On both 4.18 and 4.18 I have tried both BIG_LITTLE_CPUFREQ and BL_CPUFREQ 

Code:
#
# CPU frequency scaling drivers
#
CONFIG_CPUFREQ_DT=y
CONFIG_CPUFREQ_DT_PLATDEV=y
# CONFIG_ACPI_CPPC_CPUFREQ is not set
CONFIG_ARM_BIG_LITTLE_CPUFREQ=y
CONFIG_ARM_DT_BL_CPUFREQ=y
# CONFIG_ARM_SCPI_CPUFREQ is not set
# CONFIG_QORIQ_CPUFREQ is not set
CONFIG_NET=y
CONFIG_NET_INGRESS=y

with no success. My thinking is that by now big.LITTLE is old enough that mainline support should be there. But I still cannot discover what I have "wrong".
ROCKPro64 v2.1 2GB, SM961 128GB NVMe for rootfs, HDMI video & sound, Bluetooth keyboard & mouse
Started Bionic minimal - now Cosmic, Openbox desktop for general purpose daily PC.
  Reply


Possibly Related Threads...
Thread Author Replies Views Last Post
  Linux Build Help: Stuck at "Starting kernel ..." ihamilton 2 110 10-10-2019, 12:51 PM
Last Post: ihamilton
  Void Linux WIP losinggeneration 0 63 10-04-2019, 03:23 AM
Last Post: losinggeneration
  Arch Linux on RockPro64 mmatyas 27 8,406 09-28-2019, 04:30 AM
Last Post: TaborFife
  Rockpro64 and stock arch linux TaborFife 0 97 09-21-2019, 04:29 PM
Last Post: TaborFife
  Linux distro that will work with Kodi? SBCraok 9 216 09-18-2019, 02:12 AM
Last Post: mtrcycllvr
  u-boot for Arch Linux Arm prw 3 185 07-20-2019, 12:38 PM
Last Post: stuartiannaylor
  NEMS Linux for RockPro64 pineadmin 0 375 05-09-2019, 05:51 PM
Last Post: pineadmin
Star 0.7.9 Linux release from ayufan Luke 48 16,917 02-25-2019, 02:55 PM
Last Post: Luke
  IMX214 Linux driver development anykey 0 227 01-26-2019, 07:37 AM
Last Post: anykey
  Linux Server OS Eldin 3 347 01-22-2019, 10:02 PM
Last Post: evilbunny

Forum Jump:


Users browsing this thread: 1 Guest(s)