12-27-2020, 06:22 PM
(This post was last modified: 01-07-2021, 07:07 AM by dsimic.
Edit Reason: Small wording improvements
)
Hello,
I have observed generation of spurios headphone detection interrupts on my second-batch PineBook Pro. The interrupt count increases with no headphones being plugged in or out, which is obviously not the expected behavior. I have also verified that music playback through internal speakers does not cause the tnterrupt count to increase.
You can check the above-described interrupt count by running something like this:
Code: cat /proc/interrupts | grep Headphone
This is what I get as the output for the command above, after about six hours of uptime:
Code: 73: 6 0 0 0 0 0 rockchip_gpio_irq 8 Edge Headphone detection
Of course, there should be a total of zero interrupts generated, instead of six as visible above.
I already looked at the PineBook Pro schematic, board DTS file, and the ES8316 driver in the Linux kernel. However, I'd refrain from spending a lot of time digging into this, until we can establish it as a known, non-isolated issue.
Could anyone, please, confirm the above-described issue?
Anyone, please? @ Arwen, @ KC9UDX?
One thought is if you have the headphone jack setup for serial console. Their is a switch inside to change function. The details are on the Wiki. Most of the time the Pinebook Pros come from the factory setup for headphones, not serial console. But, their have been other minor issues like this, (WiFi being disabled via the privacy switch...)
Can you play music through the headphones?
--
Arwen Evenstar
Princess of Rivendale
01-07-2021, 03:14 PM
(This post was last modified: 01-07-2021, 03:17 PM by dsimic.
Edit Reason: Clarified a bit
)
Yes, music playback works as expected through the headphones on my PineBook Pro. The issue is that the hardware sees headphone plug-in events (more precisely, the hardware generates the associated interrupts) that actually do not exist.
Could you, please, use the command from my original post to check the interrupt count on your PineBook Pro?
01-07-2021, 06:11 PM
(This post was last modified: 01-07-2021, 06:15 PM by KC9UDX.)
I think it's odd that all 6 are on CPU0. If you leave it go longer, are they always on CPU0?
Frankly though, I don't know anything about interrupt control on this SOC, so maybe there's a reasonable explanation for this.
Here's what I get. I see something completely different. I don't see an IRQ 73 at all. Please upload your full results.
Code: pbp$ intrctl list
interrupt id CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 device name(s)
gicv3 irq 0 32015* 31202* 27700* 31835* 181838* 148091* IPI ast
gicv3 irq 1 364* 385* 342* 361* 305* 178* IPI xcall
gicv3 irq 2 0* 0* 0* 0* 0* 0* IPI nop
gicv3 irq 3 0* 0* 0* 0* 0* 0* IPI shootdown
gicv3 irq 4 0* 0* 0* 0* 0* 0* IPI ddb
gicv3 irq 5 64038* 254* 303* 406* 376* 273* IPI generic
gicv3 irq 23 0* 0* 0* 0* 0* 0*
gicv3 irq 27 42431* 42375* 42375* 42376* 42371* 42372*
gicv3 irq 43 37760* 0 0 0 0 0
gicv3 irq 58 0* 2* 0* 0* 0* 0*
gicv3 irq 60 253* 311* 269* 240* 239* 223*
gicv3 irq 62 9* 10* 5* 6* 5* 8*
gicv3 irq 64 0* 0* 0* 0* 0* 0*
gicv3 irq 72 0* 0* 0* 0* 0* 0*
gicv3 irq 96 6* 0 0 0 0 0
gicv3 irq 97 0* 0 0 0 0 0
gicv3 irq 129 0* 0* 0* 0* 0* 0*
gicv3 irq 131 0* 1* 0* 0* 0* 0*
gicv3 irq 132 0* 0* 0* 0* 0* 0*
gicv3 irq 137 0* 0* 0* 0* 0* 0*
gicv3 irq 142 21885* 25409* 23693* 22990* 22501* 22202*
Incidentally, this is after about ten minutes of uptime.
I have no "irqbalance" daemon running, that's why the interrupts aren't distributed among CPU cores.
Below is the full contents of "/proc/interrupts" from my PineBook Pro, which is running Manjaro ARM, after about 12 hours of uptime and only one actual headphones plug-in event. The remaining 22 are spurious.
Code: $ cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5
18: 0 0 0 0 0 0 GICv3 25 Level vgic
20: 0 0 0 0 0 0 GICv3 27 Level kvm guest vtimer
23: 3904518 3377900 3290159 3225236 2565583 2373591 GICv3 30 Level arch_timer
25: 1864621 1612662 1191521 959221 222972 223531 GICv3 113 Level rk_timer
26: 0 0 0 0 0 0 GICv3-23 0 Level arm-pmu
27: 0 0 0 0 0 0 GICv3-23 1 Level arm-pmu
28: 0 0 0 0 0 0 GICv3 37 Level ff6d0000.dma-controller
29: 592173 0 0 0 0 0 GICv3 38 Level ff6d0000.dma-controller
30: 0 0 0 0 0 0 GICv3 39 Level ff6e0000.dma-controller
31: 1 0 0 0 0 0 GICv3 40 Level ff6e0000.dma-controller
35: 227676 0 0 0 0 0 GICv3 96 Level dw-mci
36: 0 0 0 0 0 0 GICv3 97 Level dw-mci
37: 270459 0 0 0 0 0 GICv3 43 Level mmc2
38: 2 0 0 0 0 0 GICv3 58 Level ehci_hcd:usb1
39: 528071 0 0 0 0 0 GICv3 60 Level ohci_hcd:usb3
40: 154 0 0 0 0 0 GICv3 62 Level ehci_hcd:usb2
41: 0 0 0 0 0 0 GICv3 64 Level ohci_hcd:usb4
43: 0 0 0 0 0 0 GICv3 94 Level ff100000.saradc
44: 1383 0 0 0 0 0 GICv3 91 Level ff110000.i2c
45: 0 0 0 0 0 0 GICv3 66 Level ff130000.i2c
46: 1078 0 0 0 0 0 GICv3 131 Level ttyS0
48: 15 0 0 0 0 0 GICv3 85 Level ff1d0000.spi
49: 0 0 0 0 0 0 GICv3 129 Level rockchip_thermal
50: 13812483 0 0 0 0 0 GICv3 89 Level ff3c0000.i2c
51: 155002 0 0 0 0 0 GICv3 88 Level ff3d0000.i2c
52: 0 0 0 0 0 0 GICv3 146 Level ff650000.video-codec
53: 0 0 0 0 0 0 GICv3 145 Level ff650000.video-codec
54: 0 0 0 0 0 0 GICv3 147 Level ff650800.iommu
55: 0 0 0 0 0 0 GICv3 148 Level ff660000.video-codec
56: 0 0 0 0 0 0 GICv3 149 Level ff660480.iommu
57: 0 0 0 0 0 0 GICv3 87 Level ff680000.rga
58: 0 0 0 0 0 0 GICv3 152 Edge ff848000.watchdog
60: 3294 0 0 0 0 0 GICv3 151 Level ff8f3f00.iommu, ff8f0000.vop
61: 0 0 0 0 0 0 GICv3 150 Level ff903f00.iommu, ff900000.vop
62: 0 0 0 0 0 0 GICv3 75 Level ff914000.iommu
63: 0 0 0 0 0 0 GICv3 76 Level ff924000.iommu
64: 0 0 0 0 0 0 GICv3 42 Level analogix-dp
65: 0 0 0 0 0 0 GICv3 52 Level panfrost-job
66: 0 0 0 0 0 0 GICv3 53 Level panfrost-mmu
67: 6 0 0 0 0 0 GICv3 51 Level panfrost-gpu
73: 1 0 0 0 0 0 GICv3 59 Level rockchip_usb2phy
74: 1 0 0 0 0 0 GICv3 63 Level rockchip_usb2phy
75: 0 0 0 0 0 0 GICv3 137 Level xhci-hcd:usb7
76: 4494953 0 0 0 0 0 GICv3 142 Level xhci-hcd:usb5
77: 0 0 0 0 0 0 rockchip_gpio_irq 10 Level rk808
83: 0 0 0 0 0 0 rk808 5 Edge RTC alarm
87: 0 0 0 0 0 0 rockchip_gpio_irq 2 Level fsc_interrupt_int_n
88: 0 0 0 0 0 0 rockchip_gpio_irq 7 Edge fe320000.mmc cd
89: 2 0 0 0 0 0 rockchip_gpio_irq 1 Edge Lid
90: 0 0 0 0 0 0 rockchip_gpio_irq 5 Edge Power
91: 0 0 0 0 0 0 rockchip_gpio_irq 24 Edge dc-charger
92: 23 0 0 0 0 0 rockchip_gpio_irq 8 Edge Headphone detection
93: 3 0 0 0 0 0 rockchip_gpio_irq 4 Edge host_wake
IPI0: 11682934 11200633 14275308 12625722 1059817 913992 Rescheduling interrupts
IPI1: 4229416 3802990 3781007 3766402 8341669 5314531 Function call interrupts
IPI2: 0 0 0 0 0 0 CPU stop interrupts
IPI3: 0 0 0 0 0 0 CPU stop (for crash dump) interrupts
IPI4: 390290 536615 563909 549713 234300 203538 Timer broadcast interrupts
IPI5: 419956 325416 265493 265074 219822 210541 IRQ work interrupts
IPI6: 0 0 0 0 0 0 CPU wake-up interrupts
Err: 0
My initial research points into the direction of a possible hardware issue, but I'll stay away from spending a lot of time and digging deeper until we actually have it confirmed as an issue.
By the way, what does the "intrctl" utility actually do? I'm not familiar with it and a web search returns only NetBSD references?
01-07-2021, 07:05 PM
(This post was last modified: 01-07-2021, 07:30 PM by KC9UDX.)
I'm running NetBSD. NetBSD doesn't have /proc/interrupts. We should be seeing the same thing though; your /proc/interrupts should be the same as my intrctl list. The hardware is the same. But again, I don't know how interrupts work on the PBP, at all. So, I can only surmise that the IRQ numbers are software-generated, not hardware-generated. Else we should have similar IRQ numbers. I suppose by comparing occurrences we could guess at which ones align between Manjaro and NetBSD.
Code: pbp$ uptime
1:02AM up 1:02, 4 users, load averages: 1.42, 1.30, 1.32
pbp$ intrctl list
interrupt id CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 device name(s)
gicv3 irq 0 956881* 337991* 382453* 456935* 1556493* 1380410* IPI ast
gicv3 irq 1 505* 509* 449* 488* 440* 279* IPI xcall
gicv3 irq 2 0* 0* 0* 0* 0* 0* IPI nop
gicv3 irq 3 0* 0* 0* 0* 0* 0* IPI shootdown
gicv3 irq 4 0* 0* 0* 0* 0* 0* IPI ddb
gicv3 irq 5 181484* 5899* 7603* 10334* 6726* 5807* IPI generic
gicv3 irq 23 0* 0* 0* 0* 0* 0*
gicv3 irq 27 374141* 374099* 374099* 374100* 374085* 374086*
gicv3 irq 43 50788* 0 0 0 0 0
gicv3 irq 58 0* 2* 0* 0* 0* 0*
gicv3 irq 60 3152* 3280* 3205* 3242* 3197* 3127*
gicv3 irq 62 9* 10* 5* 6* 5* 8*
gicv3 irq 64 0* 0* 0* 0* 0* 0*
gicv3 irq 72 0* 0* 0* 0* 0* 0*
gicv3 irq 96 6* 0 0 0 0 0
gicv3 irq 97 0* 0 0 0 0 0
gicv3 irq 129 0* 0* 0* 0* 0* 0*
gicv3 irq 131 0* 1* 0* 0* 0* 0*
gicv3 irq 132 0* 0* 0* 0* 0* 0*
gicv3 irq 137 0* 0* 0* 0* 0* 0*
gicv3 irq 142 78732* 83358* 81044* 80077* 79508* 79194*
Very odd; I booted at 0:00 UTC, completely by accident!
Since I don't know anything about interrupts on these systems, I kind of wonder if they could be called by software. I suppose so; but I wonder if that would show up in the occurrence count; and I wonder why Manjaro would call that interrupt.
I checking my wife's PBP running Manjaro. It has an uptime of over 12 days, and has 0 occurrences of the IRQ associated with headphone detection. I do notice that the IRQ numbers don't match yours or mine, and I also notice that yours have changed. So clearly the IRQ number is software generated. Mine (NetBSD) don't appear to change from boot to boot.
Also significant: My PBP has the serial console enabled. My wife's does not.
01-07-2021, 07:49 PM
(This post was last modified: 01-07-2021, 07:52 PM by dsimic.
Edit Reason: Minor clarification
)
Thumbs up for running NetBSD on your PineBook Pro! Having an accidental midnight boot is quite funny. 
As you have already concluded, interrupt numbers are not fixed on Linux, and there's no need to have the same interrupts present (i.e., visible or handled) on Linux and NetBSD. For example, the headphones detection IRQ is received by a GPIO line on the RK3399 SoC, which is further mapped to the "simple-audio-card" Linux kernel driver, using the device tree file. On the other hand, there's no urgent need for NetBSD to handle this particular interrupt, it may safely go unhandled.
Having zero headphone detection interrupts on your wife's PinBook Pro points into this being an isolated hardware issue on my PineBook Pro, but I'd like to have at least a couple more PineBook Pros checked, as different batches may behave differently.
Could you, please, also try to connect headphones to your wife's PineBook Pro, just to make sure that the interrupts are actually generated and counted as expected? It should be worth checking.
Thank you for checking!
01-07-2021, 08:07 PM
(This post was last modified: 01-07-2021, 08:41 PM by KC9UDX.)
Prior to enabling the serial console, I did use the headphone jack for audio output to my Hi-Fi. Assuming that headphone/speaker switching is done in software (it must be) then NetBSD must be handling the interrupt; it just isn't annotated. What's interesting with intrctl, is that you can watch interrupts as they happen in real time.
My wife does use headphones very often, so I am surprised that this interrupt hasn't occurred. But maybe she hasn't in the past 12 days. I'll see what happens when I plug them in.
I checked my wife's PBP, and the interrupt count does go up one, on every plug and every unplug of the headphones. I must have made a mistake earlier because before I plugged in the headphones, I checked the count again and it was 5. So either she had the headphones plugged in when booting 12 days ago (maybe?) or it also has a spurious source of interrupts.
Then I stupidly tried to see what would happen if I plugged the headphones into mine. Of course, I ended up at the DDB prompt with the system halted and no recourse, as I had no serial console handy.
Given the number of spurious interrupts that you have over the time, I don't know that I'd be too concerned, although it is a mystery worth solving. I don't know how hard it is to find the interrupt line to check it with a scope though. The trouble with noise on an interrupt line though is that hooking up the scope will influence the noise, or lack thereof. I wonder if Wi-Fi or bluetooth activity affects it.
Also food for thought: It may just be that you have a loose contact in the headphone jack. I wonder if tapping the case changes the count. I also wonder if you usually see even numbers.
01-08-2021, 04:29 AM
(This post was last modified: 01-08-2021, 04:53 AM by dsimic.
Edit Reason: Also replied to the latest responses
)
Thank you for checking it further! I'm sorry that you effectively panicked your laptop in the process.
The remark on even numbers as the interrupt counts is a good one, but, for example, I currently have 33 as the interrupt count, which for some reason hasn't changed in the last few hours. I'll try to keep an eye on that. By the way, I don't use WiFi or Bluetooth, which consequently shouldn't be the root cause.
I've just finished checking the PineBook Pro schematic, the DTS file and the source code of related Linux kernel drivers, and you're right, the disabling and enabling of the internal speakers is performed by the drivers, which use the headphones detection to turn off and on the amplifiers that power the internal speakers.
Speaking of that, I do hear some faint popping/clicking noises coming randomly out of the internal speakers, which might be the result of the amplifiers being turned off and on, as a result of spurios headphones detection interrupts. I'll try to debug that further.
I doubt that any percussion maintenance would actually help because the source of the headphones detection interrupt comes from the audio codec (which actually isn't documented exactly how), and the Linux kernel driver that handles the RK3399's associated GPIO line has also debouncing enabled on it.
Anyway, I tried tapping the laptop case lightly near the headphones jack, and no new interrupts were generated. I'll also try inserting and removing the headphones jack a few more times, as the cause might be related to a not-so-great auxiliary connection inside the headphones jack, which is used to signal the absence of the headphones to the audio codec.
Edit: I have also replied to the latest responses.
|
