PCIe Inconsistency Problems
#1
Hi,

I've posted this in another thread:
(10-20-2018, 05:36 AM)jja2000 Wrote:
(10-15-2018, 05:28 AM)jja2000 Wrote: I've been having trouble getting my Pine bought PCIe SATA card to work aswell.

It used to show up before but then totally disappear. The HDD spins up but does not show up in lsblk or in dmesg. There's no LED burning on the card. The HDD works aswell.

Doing a dmesg | grep pci gives me this:

Code:
[email protected]:~$ dmesg | grep pci
[    0.498071] of_get_named_gpiod_flags: parsed 'gpio' property of node '/vcc3v3-pcie-regulator[0]' - status (0)
[    0.498129] reg-fixed-voltage vcc3v3-pcie-regulator: Looking up vin-supply from device tree
[    0.498167] vcc3v3_pcie: supplied by dc_12v
[    0.498232] vcc3v3_pcie: 3300 mV
[    0.498386] reg-fixed-voltage vcc3v3-pcie-regulator: vcc3v3_pcie supplying 3300000uV
[    1.177078] ehci-pci: EHCI PCI platform driver
[    2.656712] vcc3v3_pcie: disabling
[    3.114214] phy phy-pcie-phy.9: Looking up phy-supply from device tree
[    3.114218] phy phy-pcie-phy.9: Looking up phy-supply property in node /pcie-phy failed
[    3.292837] rockchip-pcie f8000000.pcie: GPIO lookup for consumer ep
[    3.292852] rockchip-pcie f8000000.pcie: using device tree for GPIO lookup
[    3.292902] of_get_named_gpiod_flags: parsed 'ep-gpios' property of node '/[email protected][0]' - status (0)
[    3.293278] rockchip-pcie f8000000.pcie: Looking up vpcie3v3-supply from device tree
[    3.293470] rockchip-pcie f8000000.pcie: Looking up vpcie1v8-supply from device tree
[    3.293483] rockchip-pcie f8000000.pcie: Looking up vpcie1v8-supply property in node /[email protected] failed
[    3.293518] rockchip-pcie f8000000.pcie: no vpcie1v8 regulator found
[    3.302175] rockchip-pcie f8000000.pcie: Looking up vpcie0v9-supply from device tree
[    3.302195] rockchip-pcie f8000000.pcie: Looking up vpcie0v9-supply property in node /[email protected] failed
[    3.302230] rockchip-pcie f8000000.pcie: no vpcie0v9 regulator found
[    3.313324] rockchip-pcie f8000000.pcie: invalid power supply
[    3.820353] rockchip-pcie f8000000.pcie: PCIe link training gen1 timeout!
[    3.830861] rockchip-pcie: probe of f8000000.pcie failed with error -110

Okay so I installed mainline and I get a similar error:

Code:
[email protected]:~$ dmesg | grep pci
[    1.709157] vcc3v3_pcie: supplied by dc_12v
[    2.994902] ehci-pci: EHCI PCI platform driver
[    3.051118] ohci-pci: OHCI PCI platform driver
[    3.468181] vcc3v3_pcie: disabling
[    6.753355] rockchip-pcie f8000000.pcie: no vpcie12v regulator found
[    6.753416] rockchip-pcie f8000000.pcie: Linked as a consumer to regulator.3
[    6.753431] rockchip-pcie f8000000.pcie: no vpcie1v8 regulator found
[    6.753441] rockchip-pcie f8000000.pcie: no vpcie0v9 regulator found
[    7.308970] rockchip-pcie f8000000.pcie: PCIe link training gen1 timeout!
[    7.309707] rockchip-pcie f8000000.pcie: Dropping the link to regulator.3
[    7.309778] rockchip-pcie: probe of f8000000.pcie failed with error -110

But it happens a lot less consistently, mostly reboots and a bit of reseating of the cables makes it function again.

When it does work and I try to access the harddrive (either by mounting or by opening a partition manager via ssh) the board (presumeably, because I have no display output on Mainline and no way to read the GPIO pins for serial) kernel panics.

It kinda sucks that I can't use this device as intended. Anyone have a suggestion to make this work?
  Reply
#2
No problems here with two 2,5'' HDD. Works fine for me.

Code:
Linux rockpro64v_2_1 4.19.0-rc4-1071-ayufan-g10a63ec6c2a2 #1 SMP PREEMPT Mon Oct 1 07:33:40 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux

Ok, now your problem. Whats your setup?
  • Image / Kernel
  • which hdd's
  • power supply
  • sd-card / eMMC (boot medium)
And please give an output from


Code:
sudo lspci -vvv
Sorry for any mistakes. English is not my native language

1. RP64 v2.0 / PCIe NVMe as root / sd-card as boot / 2,5 Zoll HDD 1TB (USB3) using as Webserver .... (Kernel 4.4.167-1140-rockchip-ayufan-g6f266fb5d677)
2. RP64 v2.1 / PCIe SATA Marvell 88SE9230 Chipsatz / sd-card / 2 * 2,5 Zoll 2TB HDD (raid1) / using as NAS / Kernel 5.0.0-1101-ayufan-g41eeb7cd789e
3. RP64 v2.1 / testing.....testing....testing

https://forum.frank-mankel.org/category/14/rockpro64



  Reply
#3
(10-21-2018, 11:29 AM)Bullet64 Wrote: No problems here with two 2,5'' HDD. Works fine for me.

Code:
Linux rockpro64v_2_1 4.19.0-rc4-1071-ayufan-g10a63ec6c2a2 #1 SMP PREEMPT Mon Oct 1 07:33:40 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux

Ok, now your problem. Whats your setup?
  • Image / Kernel
  • which hdd's
  • power supply
  • sd-card / eMMC (boot medium)
And please give an output from


Code:
sudo lspci -vvv

Okay so I'm running bionic-lxde-rockpro64-0.7.10-1072-arm64 with the mainline kernel (so the same one as you). I have a single 3.5" HDD from Seagate (Seagate IronWolf ST4000VN008). The powersupply is the 12V 5A model from the store. I'm booting off a 32GB SD card from sandisk.

Just like mounting/partitioning the HDD, lspci makes the board hang. If it's a kernel panic or not I don't know. I can't seem to get display to work on HDMI.
  Reply
#4
I am using this release -> https://github.com/ayufan-rock64/linux-b...m64.img.xz

Only minimal, its enough for an server Smile Please use an uart adapter so we can find the error. We need logs from lspci -vvv and dmesg.
Sorry for any mistakes. English is not my native language

1. RP64 v2.0 / PCIe NVMe as root / sd-card as boot / 2,5 Zoll HDD 1TB (USB3) using as Webserver .... (Kernel 4.4.167-1140-rockchip-ayufan-g6f266fb5d677)
2. RP64 v2.1 / PCIe SATA Marvell 88SE9230 Chipsatz / sd-card / 2 * 2,5 Zoll 2TB HDD (raid1) / using as NAS / Kernel 5.0.0-1101-ayufan-g41eeb7cd789e
3. RP64 v2.1 / testing.....testing....testing

https://forum.frank-mankel.org/category/14/rockpro64



  Reply
#5
(10-22-2018, 11:06 AM)Bullet64 Wrote: I am using this release -> https://github.com/ayufan-rock64/linux-b...m64.img.xz

Only minimal, its enough for an server Smile Please use an uart adapter so we can find the error. We need logs from lspci -vvv and dmesg.

Any idea where I can find the pin layout for serial? Is there anything else I need to enable before it works?
  Reply
#6
I think this post help you. https://forum.pine64.org/showthread.php?tid=6387
Sorry for any mistakes. English is not my native language

1. RP64 v2.0 / PCIe NVMe as root / sd-card as boot / 2,5 Zoll HDD 1TB (USB3) using as Webserver .... (Kernel 4.4.167-1140-rockchip-ayufan-g6f266fb5d677)
2. RP64 v2.1 / PCIe SATA Marvell 88SE9230 Chipsatz / sd-card / 2 * 2,5 Zoll 2TB HDD (raid1) / using as NAS / Kernel 5.0.0-1101-ayufan-g41eeb7cd789e
3. RP64 v2.1 / testing.....testing....testing

https://forum.frank-mankel.org/category/14/rockpro64



  Reply
#7
(10-22-2018, 11:23 AM)Bullet64 Wrote: I think this post help you. https://forum.pine64.org/showthread.php?tid=6387

Code:
[email protected]:~$ sudo lspci -vvv
[sudo] password for rock64:
[   79.585964] SError Interrupt on CPU4, code 0xbf000002 -- SError
[   79.585966] CPU: 4 PID: 2743 Comm: lspci Not tainted 4.19.0-rc4-1071-ayufan-g10a63ec6c2a2 #1
[   79.585967] Hardware name: Pine64 RockPro64 (DT)
[   79.585968] pstate: 60000085 (nZCv daIf -PAN -UAO)
[   79.585969] pc : rockchip_pcie_rd_conf+0x1a4/0x210 [pcie_rockchip_host]
[   79.585970] lr : rockchip_pcie_rd_conf+0x190/0x210 [pcie_rockchip_host]
[   79.585970] sp : ffff00000ee63bf0
[   79.585971] x29: ffff00000ee63bf0 x28: ffff8000efb26400
[   79.585973] x27: 0000000000000000 x26: ffff8000efb26400
[   79.585975] x25: 000000000000000f x24: 0000000000000040
[   79.585977] x23: ffff00000ee63ce4 x22: ffff00000ee63c64
[   79.585979] x21: ffff8000ef186b98 x20: 0000000000000004
[   79.585980] x19: 0000000000000000 x18: 0000000000000000
[   79.585982] x17: 0000000000000000 x16: 0000000000000000
[   79.585984] x15: 0000000000000000 x14: 0000000000000000
[   79.585985] x13: 0000000000000000 x12: 0000000000000000
[   79.585987] x11: 0000000000000000 x10: 0000000000000000
[   79.585989] x9 : 0000000000000000 x8 : 0000000000000000
[   79.585990] x7 : 0000000000000000 x6 : 0000000000000000
[   79.585992] x5 : 0000000000000000 x4 : 0000000000c00008
[   79.585994] x3 : ffff000013000000 x2 : 000000000080000a
[   79.585995] x1 : ffff000013c00008 x0 : ffff000010000000
[   79.585997] Kernel panic - not syncing: Asynchronous SError Interrupt
[   79.585998] CPU: 4 PID: 2743 Comm: lspci Not tainted 4.19.0-rc4-1071-ayufan-g10a63ec6c2a2 #1
[   79.585999] Hardware name: Pine64 RockPro64 (DT)
[   79.586000] Call trace:
[   79.586000]  dump_backtrace+0x0/0x180
[   79.586001]  show_stack+0x14/0x20
[   79.586002]  dump_stack+0x90/0xb4
[   79.586002]  panic+0x120/0x294
[   79.586003]  nmi_panic+0x6c/0x70
[   79.586004]  arm64_serror_panic+0x74/0x88
[   79.586004]  do_serror+0xbc/0xc0
[   79.586005]  el1_error+0x7c/0xdc
[   79.586006]  rockchip_pcie_rd_conf+0x1a4/0x210 [pcie_rockchip_host]
[   79.586006]  pci_user_read_config_dword+0x7c/0x110
[   79.586007]  pci_read_config+0xe8/0x258
[   79.586008]  sysfs_kf_bin_read+0x50/0x90
[   79.586008]  kernfs_fop_read+0x98/0x1d8
[   79.586009]  __vfs_read+0x30/0x158
[   79.586010]  vfs_read+0x90/0x168
[   79.586010]  ksys_pread64+0x70/0xb0
[   79.586011]  __arm64_sys_pread64+0x1c/0x28
[   79.586012]  el0_svc_common+0x60/0xe8
[   79.586012]  el0_svc_handler+0x2c/0x80
[   79.586013]  el0_svc+0x8/0xc
[   79.586027] SMP: stopping secondary CPUs
[   79.586027] Kernel Offset: disabled
[   79.586028] CPU features: 0x0,2180600c
[   79.586029] Memory Limit: none

As I thought, kernel panic :/

Edit: I tried it again (just a regular "lspci")

Now I don't get a kernel panic:
Code:
[ 2914.597297] rk3x-i2c ff3c0000.i2c: timeout, ipd: 0x10, state: 1
[ 2915.621292] rk3x-i2c ff3c0000.i2c: timeout, ipd: 0x10, state: 1
[ 2916.645295] rk3x-i2c ff3c0000.i2c: timeout, ipd: 0x10, state: 1
[ 2917.669304] rk3x-i2c ff3c0000.i2c: timeout, ipd: 0x10, state: 1
[ 2918.693301] rk3x-i2c ff3c0000.i2c: timeout, ipd: 0x10, state: 1
[ 2919.717306] rk3x-i2c ff3c0000.i2c: timeout, ipd: 0x10, state: 1
[ 2919.717986] cpu cpu4: _set_opp_voltage: failed to set voltage (1200000 1200000 1200000 mV): -110
[ 2919.718956] cpufreq: __target_index: Failed to change cpu frequency: -110
[ 2920.741313] rk3x-i2c ff3c0000.i2c: timeout, ipd: 0x10, state: 1
[ 2921.765310] rk3x-i2c ff3c0000.i2c: timeout, ipd: 0x10, state: 1
[ 2922.789315] rk3x-i2c ff3c0000.i2c: timeout, ipd: 0x10, state: 1
[ 2923.813320] rk3x-i2c ff3c0000.i2c: timeout, ipd: 0x10, state: 1
[ 2924.837320] rk3x-i2c ff3c0000.i2c: timeout, ipd: 0x10, state: 1
[ 2925.861324] rk3x-i2c ff3c0000.i2c: timeout, ipd: 0x10, state: 1
[ 2925.862002] cpu cpu4: _set_opp_voltage: failed to set voltage (1200000 1200000 1200000 mV): -110
[ 2925.862970] cpufreq: __target_index: Failed to change cpu frequency: -110
[ 2926.885328] rk3x-i2c ff3c0000.i2c: timeout, ipd: 0x10, state: 1
[ 2927.909329] rk3x-i2c ff3c0000.i2c: timeout, ipd: 0x10, state: 1
[ 2928.933332] rk3x-i2c ff3c0000.i2c: timeout, ipd: 0x10, state: 1
[ 2929.957336] rk3x-i2c ff3c0000.i2c: timeout, ipd: 0x10, state: 1
[ 2930.325296] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 2930.326003] rcu:     3-...0: (0 ticks this GP) idle=24e/1/0x4000000000000000 softirq=73507/73507 fqs=2626
[ 2930.327029] rcu:     (detected by 4, t=5252 jiffies, g=538765, q=1554)
[ 2930.981340] rk3x-i2c ff3c0000.i2c: timeout, ipd: 0x10, state: 1
[ 2931.765306] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 2931.766005] rcu:     3-...0: (1 GPs behind) idle=24e/1/0x4000000000000000 softirq=73506/73507 fqs=2625
[ 2931.767006] rcu:     (detected by 5, t=5253 jiffies, g=19657, q=2)
[ 2940.753399] rk3x-i2c ff3c0000.i2c: timeout, ipd: 0x10, state: 1
[ 2940.754081] cpu cpu4: _set_opp_voltage: failed to set voltage (1200000 1200000 1200000 mV): -110
[ 2940.755048] cpufreq: __target_index: Failed to change cpu frequency: -110
[ 2941.761377] rk3x-i2c ff3c0000.i2c: timeout, ipd: 0x10, state: 1
[ 2942.785381] rk3x-i2c ff3c0000.i2c: timeout, ipd: 0x10, state: 1
  Reply
#8
Exactly the same problem (I have post same symptoms on another topic).
Reboot the device
DONT LOGIN
press ctrl+alt+f1 to bring the terminal.
Login and mount your hdd. You can also check your hdd with a smartctl to see if you can read its properties.

You should be ok.
After that if you try to login to lxde the board will hang again. But at least you will know that the PCIe and the harddisk works ok.
It's a software problem for sure.
  Reply
#9
And now please use bionic-minimal. I think LXDE is the problem.
Sorry for any mistakes. English is not my native language

1. RP64 v2.0 / PCIe NVMe as root / sd-card as boot / 2,5 Zoll HDD 1TB (USB3) using as Webserver .... (Kernel 4.4.167-1140-rockchip-ayufan-g6f266fb5d677)
2. RP64 v2.1 / PCIe SATA Marvell 88SE9230 Chipsatz / sd-card / 2 * 2,5 Zoll 2TB HDD (raid1) / using as NAS / Kernel 5.0.0-1101-ayufan-g41eeb7cd789e
3. RP64 v2.1 / testing.....testing....testing

https://forum.frank-mankel.org/category/14/rockpro64



  Reply
#10
Installed the minimal version, tried mainline and stable. First boot it showed up but it hung after doing lspci. After the next couple of reboots I got the same error as before about "pcie link training".

@ayufan can't help?
  Reply


Possibly Related Threads...
Thread Author Replies Views Last Post
  My PCIe disk controller works with Linux 5.3.6 guannais 1 109 10-30-2019, 07:11 PM
Last Post: guannais
  Frequent kernel panic, possibly due to PCIe SSD? meier 3 184 10-29-2019, 10:28 AM
Last Post: Happybuilder
Information Update 22.02.2019 Feature-complete Debian build - SDIO WiFi/BT / PCIe / 3D / 2D Luke 100 14,195 10-04-2019, 08:57 PM
Last Post: tuxd3v
  PCIe ath10k on RockPro64 dasfranky 5 198 08-31-2019, 11:36 AM
Last Post: tuxd3v
  PCIe Sata II adapter and ssd TRIM bavay 0 92 08-16-2019, 07:49 AM
Last Post: bavay
  Major issues with 0.8.3 bionic mate and PCIe TheSupercomputer 2 209 07-03-2019, 03:38 PM
Last Post: ayufan
  Problems flashing u-boot on RockPRO64 eMMC mau-tux 1 303 05-28-2019, 07:15 AM
Last Post: CrystalGamma
  Image for ZFS on PCIe SATA drives with OpenVPN and WiFi caillou 0 205 02-26-2019, 05:14 PM
Last Post: caillou
  Is PCIE x4 bootable for NVMe SSD under Linux jzhang18 48 6,941 12-25-2018, 03:05 PM
Last Post: Atch
Question rockpro64: Enable sdio0, disable pcie, on ayufan 4.4.132 ? fosf0r 3 621 12-12-2018, 04:33 PM
Last Post: Luke

Forum Jump:


Users browsing this thread: 1 Guest(s)