PCIe x4 mode - Lane 3 failure

Has anyone had any problems with PCIe working at x4 link width? I have two Rockpro64 boards, let's call them Board A and Board B. And I have an I350-T2 PCI Express card, which has x4 Lane support. Board A only ever achieves a x2 Lane configuration after link training. Board B usually manages x4 Lane (but not always) and quite often fails on the first Configuration Register access with an External Data Abort fault.

To diagnose this further, I have modified the PCIe PHY driver, so that at startup only one Lane is turned on, the other three remaining off (by configuring GRF_SOC_CON_5_PCIE accordingly). I then boot the board with just this one Lane enabled and observe the results - obviously the system should run successfully with a x1 Lane configuration after link training. Trying each board, with each lane enabled in turn individually, I find that:

With Board A, each of Lanes 0, 1 and 2 always achieve successful link training and subsequent access to the card never fails. However, link training on Lane 3 always fails, so the card cannot be accessed with just this lane enabled - and hence explains why it always falls back to x2 mode when all four lanes are enabled.

With Board B, again each of Lanes 0, 1 and 2 always work successfully. With just Lane 3 enabled, link training succeeds (nearly always, anyway). However,  after successful link training, the first access to a register on the card fails around 50% of the time, with an External Data Abort fault, implying that the signal over this lane is not reliable. On one occasion, the first few accesses succeeded and then it failed. The frequency of the failure does diminish if you increase the bus-scan-delay-ms period, but even with this set at 3 seconds, failures still occur.

I notice on the Rockpro64 schematic (Sheet 27) that the PCIE_RX3P / PCIE_RX3N lines have additional circuitry. It appears to consist of a potential divider to hold the DC bias of the Lane 3 output from the PCIe card at 350mV, followed by a 50ohm impedance termination and AC coupling capacitor, before the signal is fed into to RK3399. I'm no hardware engineer, so I don't quite understand what the purpose of this is - and I haven't managed to find any information about it on the internet. Does anyone know what this is for? Could it be affecting the reliability of Lane 3?

(I get the same results with a second I350-T2 card, but unfortunately don't have any alternative PCIe x4 cards to test with).

I look forward to anyone's comments regarding this,
@acdkyn You might be interested in the following thread where a fix for exactly this issue is being discussed:

Possibly Related Threads…
Thread Author Replies Views Last Post
  Failure to boot atcwells 1 181 05-10-2020, 03:06 PM
Last Post: atcwells
  Considering buying and question on the PCIe slot MobileJAD 25 13,023 11-07-2019, 03:10 PM
Last Post: hmuller
  RockPro64 pcie Gpu Davidos 1 640 06-09-2019, 02:09 PM
Last Post: guannais
  Non burst mode in MIPI DSI skumar 0 139 06-06-2019, 12:54 AM
Last Post: skumar
  login to GUI desktop hangs when PCIe SSD is plugged in grimace 3 324 04-24-2019, 04:24 PM
Last Post: Luke
  PCIe SSD drive still not working ?? dr_ju_ju 7 1,273 04-15-2019, 04:16 PM
Last Post: tllim
  Desktop distro and pcie nvme ssd futurejones 2 335 03-16-2019, 12:41 AM
Last Post: futurejones
  NAS Box - fdisk not showing hard-drives w/pcie sata card dkebler 0 430 10-12-2018, 12:30 PM
Last Post: dkebler
  dedicated GPU PCIe on RockPro64 ice2642 3 2,088 10-12-2018, 02:00 AM
Last Post: ice2642
  PCIe-Bus not initialized, no devices detected seiji 10 2,179 08-11-2018, 05:54 PM
Last Post: Arglebargle

Forum Jump:

Users browsing this thread: 1 Guest(s)