+- PINE64 (https://forum.pine64.org)
+-- Forum: ROCKPRO64 (https://forum.pine64.org/forumdisplay.php?fid=98)
+--- Forum: Linux on RockPro64 (https://forum.pine64.org/forumdisplay.php?fid=101)
+--- Thread: Is PCIE x4 bootable for NVMe SSD under Linux (/showthread.php?tid=6082)
Is PCIE x4 bootable for NVMe SSD under Linux - jzhang18 - 05-21-2018
I am not sure uBoot will support NVMe SSD to boot off from PCIE x4 .
Anyone can test and tell me?
Really appreciated.
RE: Is PCIE x4 bootable for NVMe SSD under Linux - Luke - 05-21-2018
(05-21-2018, 09:01 AM)jzhang18 Wrote: I am not sure uBoot will support NVMe SSD to boot off from PCIE x4 .
Anyone can test and tell me?
Really appreciated.
I ordered an NVMe drive today. Once Linux images are out and we can flash uboot SPI I'll test.
RE: Is PCIE x4 bootable for NVMe SSD under Linux - jzhang18 - 05-22-2018
(05-21-2018, 09:07 AM)Luke Wrote:(05-21-2018, 09:01 AM)jzhang18 Wrote: I am not sure uBoot will support NVMe SSD to boot off from PCIE x4 .
Anyone can test and tell me?
Really appreciated.
I ordered an NVMe drive today. Once Linux images are out and we can flash uboot SPI I'll test.
If that is true, it will be amazing for the performance!
RE: Is PCIE x4 bootable for NVMe SSD under Linux - Bullet64 - 06-10-2018
Ayufan Image 0.6.52
Code:
rock64@rockpro64:~$ lspci
00:00.0 PCI bridge: Rockchip Inc. RK3399 PCI Express Root Port Device 0100
01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM961/PM961Code:
Disk /dev/nvme0n1: 232.9 GiB, 250059350016 bytes, 488397168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytesCode:
rock64@rockpro64:/mnt$ sudo dd if=/dev/zero of=sd.img bs=1M count=4096 conv=fdatasync
4096+0 records in
4096+0 records out
4294967296 bytes (4.3 GB, 4.0 GiB) copied, 11.795 s, 364 MB/sOnly x2 atm. But it works
Thanks Ayufan!!
RE: Is PCIE x4 bootable for NVMe SSD under Linux - pfeerick - 06-10-2018
(06-10-2018, 02:19 PM)Bullet64 Wrote: Ayufan Image 0.6.52
Code:rock64@rockpro64:~$ lspci
00:00.0 PCI bridge: Rockchip Inc. RK3399 PCI Express Root Port Device 0100
01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM961/PM961
Only x2 atm. But it works
Plug and play? Or did you need to configure anything? I'm still waiting on my M2/NVMe drive before I can try it out. Shouldn't be too long now. I'm not getting any activity from the SATA card, but the onboard power LED isn't lit so I suspect the regulator is not switched on for it.
RE: Is PCIE x4 bootable for NVMe SSD under Linux - Bullet64 - 06-11-2018
Plug & Play
But not stable atm.When not allowed, please delete the link. More infos -> https://frank-mankel.org/topic/58/rockpro64-pcie-x4/7 (only german)
RE: Is PCIE x4 bootable for NVMe SSD under Linux - pfeerick - 06-11-2018
(06-11-2018, 07:40 AM)Bullet64 Wrote: Plug & Play
When not allowed, please delete the link. More infos -> https://frank-mankel.org/topic/58/rockpro64-pcie-x4/7 (only german)
Thanks for the link... I'll be sure to follow your progress
I'm getting some PCI-e test cards so I can check the PCI-e ports, and can hopefully get the NVM card running once the drive arrives. I'm surprised that just plugging in the SATA card with no drives attached didn't work (get detected) for me though... although from your notes it seems like there is a fault? Since AFAIK this hasn't been documented as a possible known issue on the forum yet (as it should have been by now), I'm assuming from the #rock64 IRC chat history from the 27/28th of May 2018 that removal of the resistor indicated in this picture is the solution to my problem with a pre-production rockpro64? https://app.box.com/s/wghe5gu40dliaz2014g1rj28i2j164bv If so, looks easy enough, will warm up the soldering iron and do some surgery 
RE: Is PCIE x4 bootable for NVMe SSD under Linux - Bullet64 - 06-11-2018
Yes, i have remove the resistor on preproduction rockpro64!!!! Please take care, that is not without danger!
My resistor has removed a specialist.
After this on NVMe Card the red LED is ON
And the NVMe card works.On the SATA-Card is an white(?) LED. But this LED isn't ON. So maybe wrong pcie-regulator ???
Code:
rock64@rockpro64:~$ sudo dmesg | grep -E "pci"
[ 0.502571] of_get_named_gpiod_flags: parsed 'gpio' property of node '/vcc3v3-pcie-regulator[0]' - status (0)
[ 0.502619] reg-fixed-voltage vcc3v3-pcie-regulator: Looking up vin-supply from device tree
[ 0.502657] vcc3v3_pcie: supplied by dc_12v
[ 0.502723] vcc3v3_pcie: 3300 mV
[ 0.502878] reg-fixed-voltage vcc3v3-pcie-regulator: vcc3v3_pcie supplying 3300000uV
[ 0.906753] phy phy-pcie-phy.5: Looking up phy-supply from device tree
[ 0.906762] phy phy-pcie-phy.5: Looking up phy-supply property in node /pcie-phy failed
[ 0.908221] rockchip-pcie f8000000.pcie: GPIO lookup for consumer ep
[ 0.908231] rockchip-pcie f8000000.pcie: using device tree for GPIO lookup
[ 0.908258] of_get_named_gpiod_flags: parsed 'ep-gpios' property of node '/pcie@f8000000[0]' - status (0)
[ 0.908533] rockchip-pcie f8000000.pcie: Looking up vpcie3v3-supply from device tree
[ 0.908622] rockchip-pcie f8000000.pcie: Looking up vpcie1v8-supply from device tree
[ 0.908633] rockchip-pcie f8000000.pcie: Looking up vpcie1v8-supply property in node /pcie@f8000000 failed
[ 0.908646] rockchip-pcie f8000000.pcie: no vpcie1v8 regulator found
[ 0.908671] rockchip-pcie f8000000.pcie: Looking up vpcie0v9-supply from device tree
[ 0.908681] rockchip-pcie f8000000.pcie: Looking up vpcie0v9-supply property in node /pcie@f8000000 failed
[ 0.908692] rockchip-pcie f8000000.pcie: no vpcie0v9 regulator found
[ 0.929331] rockchip-pcie f8000000.pcie: invalid power supply
[ 0.965700] PCI host bridge /pcie@f8000000 ranges:
[ 0.966078] rockchip-pcie f8000000.pcie: PCI host bridge to bus 0000:00
[ 0.966105] pci_bus 0000:00: root bus resource [bus 00-1f]
[ 0.966128] pci_bus 0000:00: root bus resource [mem 0xfa000000-0xfbdfffff]
[ 0.966152] pci_bus 0000:00: root bus resource [io 0x0000-0xfffff] (bus address [0xfbe00000-0xfbefffff])
[ 0.966211] pci 0000:00:00.0: [1d87:0100] type 01 class 0x060400
[ 0.966330] pci 0000:00:00.0: supports D1
[ 0.966339] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[ 0.966581] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 0.966733] pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-1f] (conflicts with (null) [bus 00-1f])
[ 0.966767] pci 0000:01:00.0: [1b21:0612] type 00 class 0x010601
[ 0.966810] pci 0000:01:00.0: reg 0x10: initial BAR value 0x00000000 invalid
[ 0.966834] pci 0000:01:00.0: reg 0x10: [io size 0x0008]
[ 0.966850] pci 0000:01:00.0: reg 0x14: initial BAR value 0x00000000 invalid
[ 0.966871] pci 0000:01:00.0: reg 0x14: [io size 0x0004]
[ 0.966887] pci 0000:01:00.0: reg 0x18: initial BAR value 0x00000000 invalid
[ 0.966908] pci 0000:01:00.0: reg 0x18: [io size 0x0008]
[ 0.966923] pci 0000:01:00.0: reg 0x1c: initial BAR value 0x00000000 invalid
[ 0.966945] pci 0000:01:00.0: reg 0x1c: [io size 0x0004]
[ 0.966960] pci 0000:01:00.0: reg 0x20: initial BAR value 0x00000000 invalid
[ 0.966981] pci 0000:01:00.0: reg 0x20: [io size 0x0020]
[ 0.966997] pci 0000:01:00.0: reg 0x24: [mem 0x00000000-0x000001ff]
[ 0.967013] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
[ 0.968329] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[ 0.968363] pci 0000:00:00.0: BAR 8: assigned [mem 0xfa000000-0xfa0fffff]
[ 0.968392] pci 0000:01:00.0: BAR 6: assigned [mem 0xfa000000-0xfa00ffff pref]
[ 0.968426] pci 0000:01:00.0: BAR 5: assigned [mem 0xfa010000-0xfa0101ff]
[ 0.968452] pci 0000:01:00.0: BAR 4: no space for [io size 0x0020]
[ 0.968474] pci 0000:01:00.0: BAR 4: failed to assign [io size 0x0020]
[ 0.968496] pci 0000:01:00.0: BAR 0: no space for [io size 0x0008]
[ 0.968517] pci 0000:01:00.0: BAR 0: failed to assign [io size 0x0008]
[ 0.968539] pci 0000:01:00.0: BAR 2: no space for [io size 0x0008]
[ 0.968560] pci 0000:01:00.0: BAR 2: failed to assign [io size 0x0008]
[ 0.968581] pci 0000:01:00.0: BAR 1: no space for [io size 0x0004]
[ 0.968602] pci 0000:01:00.0: BAR 1: failed to assign [io size 0x0004]
[ 0.968623] pci 0000:01:00.0: BAR 3: no space for [io size 0x0004]
[ 0.968644] pci 0000:01:00.0: BAR 3: failed to assign [io size 0x0004]
[ 0.968667] pci 0000:00:00.0: PCI bridge to [bus 01]
[ 0.968692] pci 0000:00:00.0: bridge window [mem 0xfa000000-0xfa0fffff]
[ 0.968764] pcieport 0000:00:00.0: enabling device (0000 -> 0002)
[ 0.969009] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt
[ 0.969032] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
[ 0.969057] pcie_pme 0000:00:00.0:pcie01: service driver pcie_pme loaded
[ 0.969161] aer 0000:00:00.0:pcie02: service driver aer loaded
[ 1.225528] ehci-pci: EHCI PCI platform driverCode:
rock64@rockpro64:~$ lspci
00:00.0 PCI bridge: Rockchip Inc. RK3399 PCI Express Root Port Device 0100
01:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02)RE: Is PCIE x4 bootable for NVMe SSD under Linux - Tigger - 06-13-2018
(06-11-2018, 07:40 AM)Bullet64 Wrote: Plug & Play
When not allowed, please delete the link. More infos -> https://frank-mankel.org/topic/58/rockpro64-pcie-x4/7 (only german)
Thanks for the effort you put in for posting this information (with nice photos too).
I doubt that the link will be removed. There is nothing better for sales than seeing real people buy and actually using products.
RE: Is PCIE x4 bootable for NVMe SSD under Linux - Bullet64 - 07-03-2018
with 0.6.59
Code:
root@rockpro64:/home/rock64# lspci -vvv
00:00.0 PCI bridge: Rockchip Inc. RK3399 PCI Express Root Port Device 0100 (prog-if 00 [Normal decode])
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx-
Latency: 0
Interrupt: pin A routed to IRQ 236
Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
I/O behind bridge: 00000000-00000fff
Memory behind bridge: fa000000-fa0fffff
Prefetchable memory behind bridge: 00000000-000fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [80] Power Management version 3
Flags: PMEClk- DSI- D1+ D2- AuxCurrent=0mA PME(D0+,D1+,D2-,D3hot+,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME+
Capabilities: [90] MSI: Enable+ Count=1/1 Maskable+ 64bit+
Address: 00000000fee30040 Data: 0000
Masking: 00000000 Pending: 00000000
Capabilities: [b0] MSI-X: Enable- Count=1 Masked-
Vector table: BAR=0 offset=00000000
PBA: BAR=0 offset=00000008
Capabilities: [c0] Express (v2) Root Port (Slot+), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0
ExtTag- RBE+
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L1, Exit Latency L0s <256ns, L1 <8us
ClockPM- Surprise- LLActRep- BwNot+ ASPMOptComp+
LnkCtl: ASPM L1 Enabled; RCB 64 bytes Disabled- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt+ AutBWInt+
LnkSta: Speed 5GT/s, Width x4, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
Slot #0, PowerLimit 0.000W; Interlock- NoCompl-
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Off, PwrInd Off, Power+ Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL+ CmdCplt- PresDet- Interlock-
Changed: MRL- PresDet- LinkState-
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Range B, TimeoutDis+, LTR+, OBFF Via message ARIFwd+
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
Capabilities: [274 v1] Transaction Processing Hints
Interrupt vector mode supported
Device specific mode supported
Steering table in TPH capability structure
Kernel driver in use: pcieport
01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM961/PM961 (prog-if 02 [NVM Express])
Subsystem: Samsung Electronics Co Ltd NVMe SSD Controller SM961/PM961
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin A routed to IRQ 235
Region 0: Memory at fa000000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable- Count=1/32 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [70] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
LnkCap: Port #0, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L0s unlimited, L1 <64us
ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM L1 Enabled; RCB 64 bytes Disabled- CommClk-
ExtSynch- ClockPM+ AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR+, OBFF Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [b0] MSI-X: Enable+ Count=8 Masked-
Vector table: BAR=0 offset=00003000
PBA: BAR=0 offset=00002000
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00
Capabilities: [158 v1] Power Budgeting <?>
Capabilities: [168 v1] #19
Capabilities: [188 v1] Latency Tolerance Reporting
Max snoop latency: 0ns
Max no snoop latency: 0ns
Capabilities: [190 v1] L1 PM Substates
L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
PortCommonModeRestoreTime=10us PortTPowerOnTime=10us
L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
T_CommonMode=0us LTR1.2_Threshold=0ns
L1SubCtl2: T_PwrOn=10us
Kernel driver in use: nvmeYeah
Code:
LnkSta: Speed 5GT/s, Width x4iozone with Samsung 960 EVO m.2 256GB
Code:
root@rockpro64:/mnt# sudo iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
Iozone: Performance Test of File I/O
Version $Revision: 3.429 $
Compiled for 64 bit mode.
Build: linux
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
Vangel Bojaxhi, Ben England, Vikentsi Lapa.
Run began: Wed Jul 4 03:21:54 2018
Include fsync in write timing
O_DIRECT feature enabled
Auto Mode
File size set to 102400 kB
Record Size 4 kB
Record Size 16 kB
Record Size 512 kB
Record Size 1024 kB
Record Size 16384 kB
Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
Output is in kBytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 kBytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random random bkwd record stride
kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
102400 4 69551 109290 128601 114462 25822 73054
102400 16 133640 240685 269449 273207 117705 192497
102400 512 582963 588868 458828 465076 417720 551456
102400 1024 519443 628637 490247 497807 468750 627362
102400 16384 977517 1048544 1032961 1097595 1070311 1094686 Thanks Ayufan!