USB 3.0 Host Controller crashes when accessing several devices simultaneously

[h31]

Hello!

I'm trying to connect three 2.5" HDDs to the ROCK64 computer. To do this, I use an externally-powered USB 3.0 hub. The problem is that I'm receiving errors when trying to access two drives simultaneously.

I've used fio and dd utilities to access hard drives. The example command is "dd if=/dev/sda of=/dev/null bs=128k". When I'm reading from one drives, everything works well. When I'm reading from two drives in parallel, dd and fio utilities hang and ROCK64 disconnects hard drives.

Here are some samples of the errors occured. Each time I'm trying, the new error I receive 

Code:

[ 1563.840645] xhci-hcd xhci-hcd.8.auto: WARN Successful completion on short TX
[ 1563.845139] xhci-hcd xhci-hcd.8.auto: WARN Event TRB for slot 4 ep 2 with no TDs queued?
[ 1563.849678] xhci-hcd xhci-hcd.8.auto: WARN Event TRB for slot 4 ep 2 with no TDs queued?

Code:

[ 1498.598339] usb 5-1-port2: Cannot enable. Maybe the USB cable is bad?

Code:

[  500.701392] usb 5-1.2: reset SuperSpeed USB device number 4 using xhci-hcd
[  501.702379] usb 5-1.3: reset SuperSpeed USB device number 5 using xhci-hcd
[  503.633374] usb 5-1.3: reset SuperSpeed USB device number 5 using xhci-hcd
[  503.707382] usb 5-1.2: reset SuperSpeed USB device number 4 using xhci-hcd
[  505.797483] usb 5-1.3: reset SuperSpeed USB device number 5 using xhci-hcd
[  542.889165] xhci-hcd xhci-hcd.8.auto: xHCI host not responding to stop endpoint command.
[  542.890041] xhci-hcd xhci-hcd.8.auto: Assuming host is dying, halting host.
[  542.890883] xhci-hcd xhci-hcd.8.auto: HC died; cleaning up
[  542.890970] hub 5-1:1.0: hub_port_status failed (err = -22)
[  542.890979] usb 5-1-port2: cannot reset (err = -22)
[  542.890983] usb 5-1-port2: cannot reset (err = -22)
[  542.890987] usb 5-1-port2: cannot reset (err = -22)
[  542.890992] usb 5-1-port2: cannot reset (err = -22)
[  542.890995] usb 5-1-port2: cannot reset (err = -22)
[  542.890998] usb 5-1-port2: Cannot enable. Maybe the USB cable is bad?
[  542.891004] usb 5-1-port2: cannot disable (err = -22)
[  542.891011] hub 5-1:1.0: hub_port_status failed (err = -22)
[  542.891016] usb 5-1-port2: cannot reset (err = -22)
[  542.891020] usb 5-1-port2: cannot reset (err = -22)
[  542.891024] usb 5-1-port2: cannot reset (err = -22)
[  542.891028] usb 5-1-port2: cannot reset (err = -22)
[  542.891031] usb 5-1-port2: cannot reset (err = -22)
[  542.891033] usb 5-1-port2: Cannot enable. Maybe the USB cable is bad?
[  542.891037] usb 5-1-port2: cannot disable (err = -22)
[  542.891044] hub 5-1:1.0: hub_port_status failed (err = -22)
[  542.891048] usb 5-1-port2: cannot reset (err = -22)
[  542.891052] usb 5-1-port2: cannot reset (err = -22)
[  542.891056] usb 5-1-port2: cannot reset (err = -22)
[  542.891059] usb 5-1-port2: cannot reset (err = -22)
[  542.891063] usb 5-1-port2: cannot reset (err = -22)
[  542.891065] usb 5-1-port2: Cannot enable. Maybe the USB cable is bad?
[  542.891069] usb 5-1-port2: cannot disable (err = -22)
[  542.891075] hub 5-1:1.0: hub_port_status failed (err = -22)
[  542.891079] usb 5-1-port2: cannot reset (err = -22)
[  542.891083] usb 5-1-port2: cannot reset (err = -22)
[  542.891087] usb 5-1-port2: cannot reset (err = -22)
[  542.891091] usb 5-1-port2: cannot reset (err = -22)
[  542.891095] usb 5-1-port2: cannot reset (err = -22)
[  542.891096] usb 5-1-port2: Cannot enable. Maybe the USB cable is bad?
[  542.891100] usb 5-1-port2: cannot disable (err = -22)
[  542.891105] usb 5-1-port2: cannot disable (err = -22)
[  542.891235] hub 5-1:1.0: hub_port_status failed (err = -22)
[  542.942185] xhci-hcd xhci-hcd.8.auto: xHCI host not responding to stop endpoint command.
[  542.945100] xhci-hcd xhci-hcd.8.auto: Assuming host is dying, halting host.
[  542.947869] xhci-hcd xhci-hcd.8.auto: HC died; cleaning up
[  542.951904] usb 4-1: USB disconnect, device number 2
[  542.956912] usb 5-1: USB disconnect, device number 2
[  542.959802] usb 5-1.1: USB disconnect, device number 3
[  542.977962] usb 5-1.2: USB disconnect, device number 4
[  542.986287] sd 2:0:0:0: [sdc] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x07 driverbyte=0x00
[  542.989232] sd 2:0:0:0: [sdc] tag#0 CDB: opcode=0x28 28 00 00 01 f6 00 00 00 f0 00
[  542.990275] sd 1:0:0:0: [sdb] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x01 driverbyte=0x00
[  542.990284] sd 1:0:0:0: [sdb] tag#0 CDB: opcode=0x28 28 00 01 73 b6 00 00 08 00 00
[  542.990289] blk_update_request: I/O error, dev sdb, sector 24360448
[  542.990481] sd 1:0:0:0: [sdb] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x01 driverbyte=0x00
[  542.990488] sd 1:0:0:0: [sdb] tag#0 CDB: opcode=0x28 28 00 01 73 be 00 00 08 00 00
[  542.990491] blk_update_request: I/O error, dev sdb, sector 24362496
[  543.009644] blk_update_request: I/O error, dev sdc, sector 128512
[  543.043358] sd 2:0:0:0: [sdc] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x07 driverbyte=0x00
[  543.046621] sd 2:0:0:0: [sdc] tag#0 CDB: opcode=0x28 28 00 00 01 f6 f0 00 00 10 00
[  543.049930] blk_update_request: I/O error, dev sdc, sector 128752
[  543.083261] sd 2:0:0:0: [sdc] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x07 driverbyte=0x00
[  543.086773] sd 2:0:0:0: [sdc] tag#0 CDB: opcode=0x28 28 00 00 01 f7 00 00 00 f0 00
[  543.090248] blk_update_request: I/O error, dev sdc, sector 128768
[  543.124265] sd 2:0:0:0: [sdc] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x07 driverbyte=0x00
[  543.127869] sd 2:0:0:0: [sdc] tag#0 CDB: opcode=0x28 28 00 00 01 f7 f0 00 00 10 00
[  543.131518] blk_update_request: I/O error, dev sdc, sector 129008
[  543.167252] sd 2:0:0:0: [sdc] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x07 driverbyte=0x00
[  543.171131] sd 2:0:0:0: [sdc] tag#0 CDB: opcode=0x28 28 00 00 01 f6 00 00 00 08 00
[  543.175063] blk_update_request: I/O error, dev sdc, sector 128512
[  543.178874] Buffer I/O error on dev sdc, logical block 16064, async page read
[  544.306086] usb 5-1.3: USB disconnect, device number 5

Code:

[   86.613946] usb 5-1.2: reset SuperSpeed USB device number 4 using xhci-hcd
[   91.115965] usb 5-1.3: reset SuperSpeed USB device number 5 using xhci-hcd
[  106.101604] usb 5-1.2: reset SuperSpeed USB device number 4 using xhci-hcd
[  120.967035] usb 5-1.1: reset SuperSpeed USB device number 3 using xhci-hcd
[  124.326313] usb 5-1.2: reset SuperSpeed USB device number 4 using xhci-hcd
[  132.806598] xhci-hcd xhci-hcd.8.auto: ERROR Transfer event TRB DMA ptr not part of current TD ep_index 2 comp_code 1
[  132.807731] xhci-hcd xhci-hcd.8.auto: Looking for event-dma 003f000000000000 trb-start 000000004f11abe0 trb-end 000000004f11ac10 seg-start 000000004f11a000 seg-end 000000004f11aff0
[  133.727522] usb 5-1.3: reset SuperSpeed USB device number 5 using xhci-hcd
[  134.952223] xhci-hcd xhci-hcd.8.auto: ERROR Transfer event TRB DMA ptr not part of current TD ep_index 2 comp_code 1
[  134.953355] xhci-hcd xhci-hcd.8.auto: Looking for event-dma 003ef00000000000 trb-start 000000004f044b10 trb-end 000000004f044b40 seg-start 000000004f044000 seg-end 000000004f044ff0
[  135.848533] usb 5-1.3: reset SuperSpeed USB device number 5 using xhci-hcd
[  167.944738] usb 5-1.2: reset SuperSpeed USB device number 4 using xhci-hcd

Code:

[    3.221233] usb 4-1: device descriptor read/64, error -71
[    3.427214] usb 4-1: new low-speed USB device number 4 using xhci-hcd
[    3.533216] usb 4-1: device descriptor read/64, error -71
[    3.739169] usb 4-1: device descriptor read/64, error -71
[    3.945206] usb 4-1: new low-speed USB device number 5 using xhci-hcd
[    3.956377] usb 4-1: Device not responding to setup address.
[    4.168381] usb 4-1: Device not responding to setup address.
[    4.380201] usb 4-1: device not accepting address 5, error -71
[    4.493192] usb 4-1: new low-speed USB device number 6 using xhci-hcd
[    4.504378] usb 4-1: Device not responding to setup address.
[    4.716373] usb 4-1: Device not responding to setup address.
[    4.927197] usb 4-1: device not accepting address 6, error -71
[    4.937910] usb usb4-port1: unable to enumerate USB device

Code:

[  693.135416] xhci-hcd xhci-hcd.8.auto: WARN Event TRB for slot 3 ep 2 with no TDs queued?
[  724.023537] usb 5-1.1: reset SuperSpeed USB device number 3 using xhci-hcd

Code:

[  909.970759] xhci-hcd xhci-hcd.8.auto: xHCI host not responding to stop endpoint command.
[  909.983792] xhci-hcd xhci-hcd.8.auto: Assuming host is dying, halting host.
[  910.017740] xhci-hcd xhci-hcd.8.auto: Host not halted after 16000 microseconds.
[  910.022301] xhci-hcd xhci-hcd.8.auto: Non-responsive xHCI host is not halting.
[  910.026873] xhci-hcd xhci-hcd.8.auto: Completing active URBs anyway.
[  910.031396] hub 5-1:1.0: hub_port_status failed (err = -108)
[  910.031404] xhci-hcd xhci-hcd.8.auto: HC died; cleaning up
[  910.031438] xhci-hcd xhci-hcd.8.auto: xHCI host not responding to stop endpoint command.
[  910.031445] xhci-hcd xhci-hcd.8.auto: Assuming host is dying, halting host.
[  910.031466] xhci-hcd xhci-hcd.8.auto: HC died; cleaning up
[  910.032583] usb 4-1: USB disconnect, device number 2
[  910.033237] usb 5-1: USB disconnect, device number 2
[  910.033246] usb 5-1.1: USB disconnect, device number 5
[  910.061780] sd 4:0:0:0: [sdb] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x07 driverbyte=0x00
[  910.061789] sd 4:0:0:0: [sdb] tag#0 CDB: opcode=0x28 28 00 00 12 96 00 00 00 f0 00
[  910.061793] blk_update_request: I/O error, dev sdb, sector 1218048
[  910.085847] sd 5:0:0:0: [sda] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x01 driverbyte=0x00
[  910.090574] sd 5:0:0:0: [sda] tag#0 CDB: opcode=0x28 28 00 00 17 fd 00 00 00 f0 00
[  910.091851] sd 4:0:0:0: [sdb] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x07 driverbyte=0x00
[  910.091861] sd 4:0:0:0: [sdb] tag#0 CDB: opcode=0x28 28 00 00 12 96 f0 00 00 10 00
[  910.091866] blk_update_request: I/O error, dev sdb, sector 1218288
[  910.109030] blk_update_request: I/O error, dev sda, sector 1572096
[  910.113781] scsi 5:0:0:0: [sda] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x01 driverbyte=0x00
[  910.118622] scsi 5:0:0:0: [sda] tag#0 CDB: opcode=0x28 28 00 00 17 fd f0 00 00 10 00
[  910.123388] blk_update_request: I/O error, dev sda, sector 1572336
[  910.135964] usb 5-1.3: USB disconnect, device number 4

Even when I'm keeping only 2 hard drives connected, the error still occurs. But everything works well when I'm plugging an USB hub cable to a USB 2.0 port of the ROCK64. Also, everything works perfectly when I'm connecting the USB hub to my Intel laptop (using the same set of cables and a PSU).

The ROCK64 computer is powered by an official 3A PSU. The USB hub is powered by 2A PSU. Tried several PSUs - no difference. Kernel version is 4.4.77-rockchip-ayufan-136, distro is Debian 9 (stretch).

[h31]

Update: Found a workaround:

Code:

echo 64 > /sys/block/sda/device/max_sectors
echo 64 > /sys/block/sdb/device/max_sectors
echo 64 > /sys/block/sdc/device/max_sectors

The performance is significantly reduced but at least there are no disconnects and resets. So I'm still searching for a proper fix.

[jebivetar]

Ugh, this is very disappointing to hear having just ordered a unit and a whole bunch of parts expressly for this purpose.

What USB3 hub and SATA-USB3 converters were you using out of curiosity?

[xalius]

You need to make sure that you get SATA/USB3 bridges that have no quirks/bugfixes already enabled in the Linux kernel, otherwise UAS or other features might be disabled. Armbian and others did some tests and generally the JMS576/JMS578 seem to work well. Is this with BSP or mainline kernel? You could also file an issue with your description against Rockchip's kernel on github if it is a BSP based image...

[ayufan]

I'm using Pine64 SATA-USB3 and it works stable on OMV. Sometimes there are some quirks, but I don't yet have resolution for them

[dlazarus]

I have the same problem. Two USB disks cannot be used under load on rock64. One disk works fine.
USB hub is powered.

This is the beginning of the failing process:
xhci-hcd xhci-hcd.8.auto: xHCI host not responding to stop endpoint command.

[Redwid]

Got the same issue.
My rock64 (4Gb) disconnects all my drives that connected to powered usb 3.0 hub if I do copy data from one disk to another.
The same set up works well on banana pi pro with armbian. I'm doing migration right now from my old self made NAS solution based on banana pi pro to rock64.
It's a quite pity as I planned to use more than one usb 3.0 disk connected to rock64

I've found that, if I connect destination disk to usb 2.0 and do copy by rsync from source disk that connected to usb 3.0 hub so no crashes occurs.

After a googling for a while, found that thread: https://e2e.ti.com/support/dsp/c6000_mul...cessor-SDK
There is some hack to fix a quite the same issue, but on the different board.

My image is 4.4.126-rockchip-ayufan-239

Any ideas?

---

Opened issue on ayufan github page: https://github.com/ayufan-rock64/linux-build/issues/112

[jovval]

Hi.

Have you considered this thread, "Rock64 NAS project" - https://forum.pine64.org/showthread.php?tid=5132

Perhaps, its content will provide some response to issues here.

I noted the following.
(1) The OP would like a RAID of two drives via USB 3.0 /and/ a third drive via USB 2.0. Seems odd to me. Why not all three on USB 3.0?
(2) The OP plans to use a RAID controller.
(3) The desired setup is peculiarly complex, with wires heading to nowhere.


Replies include [my comments in brackets],
(1) "Reading the Armbian guys who are very OMV (open media vault) orientated it would seem Raid over USB separate drives is not a good idea [why?].
It will work [how so?] but before you get there you have to think about the USB ports max raw bandwidth then deduct overhead and look at what you can place on it. [Specifics? Anyone, anywhere?]"

(2) "Don't do RAID over USB3 hub, it will break [how?], and very baddly [ditto] Smile Use only proven to work SATA-to-USB adapter, that has UASP support, as this gives a good performance.
There's a very good and long thread from tkaiser on Armbian forum where he describes SATA bridges, NAS solutions, and Rock64 performance numbers and he has good explanations what setups will work correctly."

(3) [responding to previous citation] "I have to disagree and not because it will not break, as it prob will but for the lack of reasons to why it should.
Much is to do with power & connectors and with the kernel we have and some quirks on cheap silicon.
UASP is transparent across a hub so all a hub does is increase cable length and connector count if its a decent hub with a good chip.
Many of the problems are how host port stacks are implemented on the motherboard and not all are the same.
Kernel 4.4 is the current image 4.6 = USB 3.1 prob really with images we should already be backporting kernel 4.9 as we have hardware that is further forward than the 4.4 mainline kernel we are currently employing."

(4) "I'm using one of these with my Rock64 to turn it into a NAS, it's a 5-port SATA to USB 3.0 bridge with hardware RAID:
https://www.amazon.com/Oodelay-eSATA-Por...B00PZ7347E
There's also a (much cheaper) 2-port version, if all your need are two disks:
https://www.amazon.com/Oodelay-eSATA-6Gb...B00T22JUT4
In hardware RAID mode, the Rock64 just sees one disk, and operates without any further configuration. If you switch to JBOD mode, the Rock64 sees all disks individually, allowing for the use of software RAID."
[I did not see any way to detect HD failure if using hardware RAID. Could anyone either confirm my observations or tell me what I missed?]

(5) "Speed and stability have been great. I currently have two 2TB hard disks connected to the controller, with the controller performing hardware RAID 1 (mirroring), and have it connected to my Rock64 (running Open Media Vault) via USB 3.0. I'm sharing an NTFS formatted volume using SAMBA.
I use it as a target for daily backups of my homelab, with backups being taken of an ESXi host by Veeam Backup & Recovery. I usually see transfer speeds in excess of between 90 and 100 MB/s while running backups over the network.
I know I might get slightly better performance if I were to reformat the disks as EXT4, but I wanted to be able to plug the controller + disks into a Windows machine if need-be."

(6) "I'm using the "jessie-openmediavault-rock64-0.5.15-136-armhf" image from here [ https://github.com/ayufan-rock64/linux-b...tag/0.5.15 ] with the kernel it comes with.
Like I said in my previous post, network transfers are being done over SMB.
I didn't really do anything special to get these speeds, it just kinda worked out of the box... Connected my storage device via USB 3.0, mounted the pre-existing NTFS volume and created shares using the OpenMediaVault WebUI, good to go. Still 100% reliable, handling gigabytes of backups from Veeam, every night."

(7) [Last post included for completeness. I found it relevant but insufficiently detailed for me to make sensible.] "I am successfully running 16 drives on my 4GB Rock64, two 8x USB3/eSATA (USB3 mode) enclosures with ZFS on Bionic. ..."


Any kind soul wishing to address my comments and questions above [in brackets]?

[jovval]

TAKE-AWAY: tkaiser's findings (shown last, at the end of #3) with insufficient power may relate to the wacky results found by the OP. Has anyone reproduced the errors with a 5A PSU or powered hub with more amperage?

I elaborate on my preceding post with this supplement. I believe #2 (below) has the "very good and long thread from tkaiser on Armbian forum where he describes SATA bridges, NAS solutions, and Rock64 performance numbers and he has good explanations what setups will work correctly" mentioned in #2 of my previous post. But. The link has performance numbers yet lacks or has in underdeveloped form (for my purposes) NAS solutions and SATA bridges.

Please note my respectful differences with tkaiser, and correct me if I am underinformed or just plain wrong. Hey. It happens, you know?


(1) Rock64 Mini NAS and media center, works? - https://forum.armbian.com/topic/4958-roc...ter-works/
By stefan.z.j, August 20, 2017 in Rockchip 3288 & 3328
Pertinant posts follow.

    NAS backed by a USB3 Raid5 enclosure
[tkaiser responds]
IMO a really bad idea. Those proprietary RAID boxes are just a huge single point of failure and I would never rely on such things (but I do storage for a living and only deal with RAID any more when it failed so I'm a bit biased here).
Wrt performance: ROCK64 is limited by Gigabit Ethernet, real world throughput with Windows Explorer or macOS Finder will be around or even above 100 MB/s (megabyte) given the USB3 attached storage is fast enough.
[tkaiser's point is well-taken, but better phrased in (a). I address shortcomings in (b-e).
(a) The RAID controller is a black box, mechanisms known only to the illuminati. If it fails, who knows what befalls your data.
(b) You can have two controllers, and swap out the bad one for the good one. If the latter works, you still depend on the bad one not having corrupted your data.
© Local backups address most of the deficiencies (ideally having redundant redundancy, i.e. two RAIDs).
(d) Remote backups address the problem more completely (with a pair of remote RAIDs, naturally, each at a different location).
(e) If cost concerns you, you can still consider non-RAID backups: (i) on and off-line, (ii) local and remote.
(f) I have no statistics to support my statements. Be sure to get a second and third opinion of (a-e).]

[tkaiser continues]
I would suggest not using RAID at home since it sucks (it works as long as you won't need it but once a disk fails funny things happen like the whole array disappearing). [These "funny things" are exactly the reason why I need to have a controller tell me when a disk is failing, or about to fail. If the controller will not do so, software RAID with JBOD should suffice, and may be more readily transferrable to different equipment.] Also the redudant RAID modes are not about data protection/safety but only about availability (who needs this at home?). [Those with home offices. So does anyone who does not want to come home from work to find more work. Maybe even you.]


(2) Some storage benchmarks on SBCs - https://forum.armbian.com/topic/1925-som...s-on-sbcs/
By tkaiser, August 30, 2016 in Research guides & tutorials [USB 3.0 mentioned 6x; Rock64, 17x beginning June 22, 2017. Last post, 5/10/2018]
Posted June 22, 2017 (edited)
All tests done with Samsung EVO SSDs (my EVO840 used for all tests [begin strike-out]except of the ODROID-XU4 results which were made with a much faster EVO850 instead and[end strike-out] Pine64 numbers with a slower EVO750 so random IO numbers should be multiplied with 1.3 or even 1.4):                     
Random IO in IOPS     Sequential IO in MB/sec
                        4K read/write           1M read/write
Pine64 (USB2/UAS)         2260/1948                42 /  41
Banana Pi Pro (SATA)      3943/3478               122 /  37
Wandboard Quad (SATA)     4141/5073               100 / 100
ODROID-XU4 (USB3/UAS)     4637/5126               262 / 282
ROCK64 (USB3/UAS)         4619/5972               311 / 297
Clearfog Pro (SATA)      10148/19184              507 / 448
     USB3 SBC like ODROID-XU4/HC1 or ROCK64 show way better performance. You only have to take care of the USB downsides (XU4 with an internal USB hub and receptacles problems is in a bad position here) and ensure that you're using good USB-to-SATA bridges to connect SSD or HDD

If we also take price/performance ratio into account then ROCK64 or other RK3328 boards that will follow are really hard to beat. The 1GB ROCK64 variant will most probably stay below the $25 margin and you should keep in mind that more DRAM is pretty useless if you think about (NAS) performance.

tkaiser,  Posted March 19, Early 2018 update [Contains and expands upon the data above. I include only new outperformers of Rock64.]
                       Random IO in IOPS     Sequential IO in MB/sec
                        4K read/write           1M read/write
RK3399 (USB3/UAS)         5994/6308               330 / 340
ASM1061 powersave         6010/7900               320 / 325
ASM1061 performance       9820/16230              330 / 330
RK3399-Q7 (NVMe)         11640/36900             1070 / 1150


(3) [Mini review] ROCK64 SATA cable - https://forum.armbian.com/topic/5098-min...ata-cable/
[All of you know or /should know/ the content of this link.
Check. it. out.
It's concluding paragraph puzzles me.]
 The 5V power requirements for JMS578 + 3.5" HDD disk were as follows:
    USB2: 6.4W vs. 3.3W (full load vs. idle). Numbers with 5V PSU included so we're talking about needed power provided on an USB2 port of approximately ~3W which fits perfectly in the power budget of ROCK64's USB2: 650mA * 5V == 3250mW
    USB3: 7.9W vs. 3.3W (full load vs. idle). Numbers again with 5V PSU included so we're talking about needed power provided on an USB3 port of approximately ~4W which fits perfectly in the power budget of ROCK64's USB3: 950mA * 5V == 4750mW

At least with my 3.5" HDD it worked pretty well to let it operate on both USB2 and USB3 ports when board powering was appropriate (with insufficient powering all weird sorts of issues popped up. My favourite was a kernel panic when issueing 'lsusb' after 30 seconds. Once I powered ROCK64 reliably all these 'software issues' were gone immediately -- again and again: insufficient powering is one of the most severe problem sources)


End of forums and their excepts. My concluding thoughts.

The clause "with insufficient powering all weird sorts of issues popped up" begs the question: what's he talking about?
More succinctly, is the 3A power block from the Pine store insufficient?
Is the Rock64Pro 5A power block compatible with the Rock64? If not, what alternative do you suggest?

Also, the sputtering of several SATA drives may stem from a need for a more powerful USB hub. I know that even 2A /should/ cover 2 x 4W (( 2 x  4W) / 5V = 1.6A) that tkaiser attributes to his 3.5" drives' use of 5V, but I am still wondering if, in practice, 2A does. Clearly, it can not handle more than 2 3.5" SATA HDDs. And I am presuming that the powered hub is functioning as it should, but it may not be. Nor may the power supply be up to snuff. (Yes, I know the OP was using 2.5" drives. I am just being thorough.)
I would like to know the output of the hubs used by the OP and any others experiencing problems with multiple SATA drives connected via USB.

TAKE-AWAY: tkaiser's findings with insufficient power may relate to the wacky results found by the OP. Has anyone reproduced the errors with a 5A PSU or powered hub with more amperage?


Over and out.

[lybin]

Try add usbcore.autosuspend=-1 in /etc/default/extlinux and sudo update_extlinux.sh

Looks like a my problem not reproduced anymore and kernel doesn't crash.

[ 3966.656954] usb 10-3: reset SuperSpeed USB device number 3 using xhci_hcd
[ 3969.286859] usb 10-3: reset SuperSpeed USB device number 3 using xhci_hcd
[ 3971.284889] usb 10-3: reset SuperSpeed USB device number 3 using xhci_hcd
[ 3973.738677] usb 10-3: reset SuperSpeed USB device number 3 using xhci_hcd
[ 4007.029496] usb 10-3: reset SuperSpeed USB device number 3 using xhci_hcd
[ 4009.548435] xhci_hcd 0000:01:00.0: WARN Event TRB for slot 2 ep 2 with no TDs queued?
[ 4042.996762] usb 10-3: reset SuperSpeed USB device number 3 using xhci_hcd
[ 4045.515788] xhci_hcd 0000:01:00.0: WARN Event TRB for slot 2 ep 2 with no TDs queued?
[ 4079.018244] usb 10-3: reset SuperSpeed USB device number 3 using xhci_hcd
[ 4081.538333] xhci_hcd 0000:01:00.0: WARN Event TRB for slot 2 ep 2 with no TDs queued?
[ 4094.056556] EXT4-fs error (device sdf1): ext4_find_extent:910: inode #302082: comm scavenger: pblk 438336000 bad header/extent: invalid magic - magic f39, entries 17478, max 20872(0), depth 26550(0)
[ 4125.045784] usb 10-3: reset SuperSpeed USB device number 3 using xhci_hcd
[ 4158.064591] usb 10-3: reset SuperSpeed USB device number 3 using xhci_hcd
[ 4158.082063] xhci_hcd 0000:01:00.0: WARN Event TRB for slot 2 ep 2 with no TDs queued?
[ 4189.052740] usb 10-3: reset SuperSpeed USB device number 3 using xhci_hcd
[ 4222.070111] usb 10-3: reset SuperSpeed USB device number 3 using xhci_hcd
[ 4224.589439] sd 2:0:0:0: timing out command, waited 180s
[ 4224.590309] sd 2:0:0:0: [sde] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
[ 4224.590339] sd 2:0:0:0: [sde] tag#0 Sense Key : 0x4 [current]
[ 4224.590365] sd 2:0:0:0: [sde] tag#0 ASC=0x40 <<vendor>>ASCQ=0x99
[ 4224.590396] sd 2:0:0:0: [sde] tag#0 CDB: opcode=0x88 88 00 00 00 00 00 d6 45 b1 e0 00 00 00 20 00 00
[ 4224.590417] blk_update_request: I/O error, dev sde, sector 3594891744
[ 4255.043288] usb 10-3: reset SuperSpeed USB device number 3 using xhci_hcd


Nov 09 16:08:15 rockpro64 kernel: usb 8-2: reset SuperSpeed USB device number 3 using xhci_hcd
Nov 09 16:08:17 rockpro64 kernel: xhci_hcd 0000:01:00.0: WARN Event TRB for slot 1 ep 2 with no TDs queued?
Nov 09 16:08:50 rockpro64 kernel: usb 8-2: reset SuperSpeed USB device number 3 using xhci_hcd
Nov 09 16:08:53 rockpro64 kernel: xhci_hcd 0000:01:00.0: WARN Event TRB for slot 1 ep 2 with no TDs queued?
Nov 09 16:09:06 rockpro64 kernel: EXT4-fs error (device sde1): ext4_find_extent:916: inode #302082: comm scavenger: pblk 851474688 bad header/extent: invalid magic - magic c19c, entries 4923, max 17809(0), depth 19583(0)
Nov 09 16:09:36 rockpro64 kernel: usb 8-2: reset SuperSpeed USB device number 3 using xhci_hcd
Nov 09 16:10:01 rockpro64 CRON[1399]: pam_unix(cron:session): session opened for user root by (uid=0)
Nov 09 16:10:01 rockpro64 CRON[1400]: (root) CMD (/home/rock64/stat.sh | /home/rock64/ansi2html.sh > /var/www/html/index.html)
Nov 09 16:10:01 rockpro64 CRON[1399]: pam_unix(cron:session): session closed for user root
Nov 09 16:10:10 rockpro64 kernel: usb 8-2: reset SuperSpeed USB device number 3 using xhci_hcd
Nov 09 16:10:10 rockpro64 kernel: xhci_hcd 0000:01:00.0: WARN Event TRB for slot 1 ep 2 with no TDs queued?
Nov 09 16:10:41 rockpro64 kernel: usb 8-2: reset SuperSpeed USB device number 3 using xhci_hcd
Nov 09 16:11:14 rockpro64 kernel: usb 8-2: reset SuperSpeed USB device number 3 using xhci_hcd
Nov 09 16:11:47 rockpro64 kernel: usb 8-2: reset SuperSpeed USB device number 3 using xhci_hcd
Nov 09 16:11:47 rockpro64 kernel: sd 2:0:0:0: timing out command, waited 180s
Nov 09 16:11:47 rockpro64 kernel: sd 2:0:0:0: [sdd] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x07 driverbyte=0x00
Nov 09 16:11:47 rockpro64 kernel: sd 2:0:0:0: [sdd] tag#0 CDB: opcode=0x88 88 00 00 00 00 01 9d 99 11 e0 00 00 00 20 00 00
Nov 09 16:11:47 rockpro64 kernel: blk_update_request: I/O error, dev sdd, sector 6939021792
Nov 09 16:12:19 rockpro64 kernel: usb 8-2: reset SuperSpeed USB device number 3 using xhci_hcd
Nov 09 16:12:20 rockpro64 kernel: sd 2:0:0:1: [sde] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x05 driverbyte=0x00
Nov 09 16:12:20 rockpro64 kernel: sd 2:0:0:1: [sde] tag#0 CDB: opcode=0x8a 8a 00 00 00 00 00 00 00 08 00 00 00 00 08 00 00
Nov 09 16:12:20 rockpro64 kernel: blk_update_request: I/O error, dev sde, sector 2048
Nov 09 16:12:20 rockpro64 kernel: Buffer I/O error on dev sde1, logical block 0, lost sync page write
Nov 09 16:12:20 rockpro64 kernel: EXT4-fs error (device sde1): ext4_find_extent:916: inode #302082: comm scavenger: pblk 851474688 bad header/extent: invalid magic - magic c19c, entries 4923, max 17809(0), depth 19583(0)
Nov 09 16:12:20 rockpro64 kernel: EXT4-fs (sde1): previous I/O error to superblock detected
Nov 09 16:12:53 rockpro64 kernel: usb 8-2: reset SuperSpeed USB device number 3 using xhci_hcd
Nov 09 16:13:26 rockpro64 kernel: usb 8-2: reset SuperSpeed USB device number 3 using xhci_hcd
Nov 09 16:13:57 rockpro64 kernel: usb 8-2: reset SuperSpeed USB device number 3 using xhci_hcd
Nov 09 16:14:27 rockpro64 kernel: usb 8-2: reset SuperSpeed USB device number 3 using xhci_hcd