Kinda upset at the lack of response to GBe issues

[androsch]

Minimum requirements for any test are 3 SSH/Terminal sessions, one running htop, the other longsleep's pine64_health.sh script (sudo watch -n1 pine64_health.sh) and the 3rd with iperf3. And the following line in /etc/rc.local prior to 'exit 0':

Code:

(sleep 30 && echo performance >/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor) &

OK, tkaiser, did exactly this as mentioned above. 3 sessions, running htop, iperf3 and the script from longsleep on its last kernel.
Also checked Rpi monitor on the pine for the time period.
Htop iperf3 process got max about 52% cpu time, mainly between 10 and 20%. pine-script showed constant 1152 MHz and 4 cores running, max temp 60 degrees, also see screenshot of RPi-monitor attached (https://dl.dropboxusercontent.com/u/1664...onitor.jpg):


Here to come the results:

Code:

1 client/connection:

[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-120.00 sec  2.75 GBytes   197 Mbits/sec  5521             sender
[  4]   0.00-120.00 sec  2.75 GBytes   197 Mbits/sec                  receiver

working as client with 4 connections

[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-120.00 sec  1.32 GBytes  94.3 Mbits/sec  4566             sender
[  4]   0.00-120.00 sec  1.32 GBytes  94.2 Mbits/sec                  receiver
[  6]   0.00-120.00 sec  1.31 GBytes  93.7 Mbits/sec  4560             sender
[  6]   0.00-120.00 sec  1.31 GBytes  93.7 Mbits/sec                  receiver
[  8]   0.00-120.00 sec  1.31 GBytes  93.5 Mbits/sec  4534             sender
[  8]   0.00-120.00 sec  1.31 GBytes  93.5 Mbits/sec                  receiver
[ 10]   0.00-120.00 sec  1.25 GBytes  89.3 Mbits/sec  4356             sender
[ 10]   0.00-120.00 sec  1.25 GBytes  89.3 Mbits/sec                  receiver
[SUM]   0.00-120.00 sec  5.18 GBytes   371 Mbits/sec  18016             sender
[SUM]   0.00-120.00 sec  5.18 GBytes   371 Mbits/sec                  receiver

not very good, but at least something towards GbE

working as server:

[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-120.06 sec  27.0 MBytes  1.89 Mbits/sec                  sender
[  5]   0.00-120.06 sec  26.9 MBytes  1.88 Mbits/sec                  receiver

so there is something wrong here...

You are welcome to receive my board like longsleep and test by yourself and i'm very happy to test your board within my network for the perfect speed you have.

Don't get this wrong please, i just wanted to make sure, that we are doing like you suggest and trying to provide information about the environment and other details you mention in your posts, but still with those suggestions there is no improvement and no GbE with the pine board i have acting as server. Maybe yours is fine and maybe i'm still doing something wrong here, but you are welcome to prove your environment and 'perfect' power/switch/network with my board also. I'm happy to sent it to you at any time!

[blairac]

Okay, from my last post I think (think) it is evident that I have a board resident "defect". I believe we are all interested in running diagnostics that can eventually point to root cause. I am more than happy to help in this effort however, this is not my primary area of expertise.

I am really good at following instructions :-)

Is there a list of potential causes that continued testing can rule out (or in)?
Is it known if we are likely looking at a single issue or are there multiple causes?

Can we formulate a logical plan that may lead us to some conclusions? We seem to be all over the place and not working together in solving a common problem. I believe if we work together, we can make some headway.

Again, I am more that willing to contribute.
What do we need to do to put together an agreeable plan?

It is most important that we work together. Collectively.

[MarkHaysHarris777]

whoo hoo, you guys are NOT going to believe this !

... I just grabbed a new Cat6 cable and opened each end with a scalple. I literally cut and removed the blue pair and the brown pair;  um, leaving intact the orange pair and the green pair.

Then I pulled the blue pair and the brown pair ( with a pair of needle nose pliers ) ; I literally removed them from the cable, forming a 2 pair (four wire) cable.

Using my ubuntu image and the 'bad card' connection to the GbE switch is now working !  It went from not working at all on the GbE switch to running at 95mb/s !

w00t w00t... !!!

---

I suspect that the reason this works is that the negotiation for GbE does not happen and by default the unit comes up  100 /full.


( we're getting closer people,  we're getting closer ! )

[pfeerick]

I suspect that the reason this works is that the negotiation for GbE does not happen and by default the unit comes up  100 /full.

Just out of curiosity, with a non-modified cable, what happens if you run "sudo ethtool -s eth0 speed 1000 duplex full" ... does it give the ethernet a kick to reboot and re-negotiation as GbE/1000Mbs? I'd try myself... but as you know... it's a non-issue for me. However, I can switch between 100Mb and 1000Mb that way.

[tkaiser]

Using my ubuntu image and the 'bad card' connection to the GbE switch is now working !  It went from not working at all on the GbE switch to running at 95mb/s !

( we're getting closer people,  we're getting closer ! )

You're not getting closer, you just forced Fast Ethernet which is something completely different as I already tried to explain to you yesterday (read through the 'zillions of words and this time try to understand). Add 'ethtool -s eth0 speed 100 duplex full' to /etc/rc.local prior to the 'exit 0' line and you get the same without fiddling around with your cable.

The real problem is still a negotiation problem in GbE mode (which is something completely different than Fast Ethernet and that's the reason I already warned exactly you yesterday to try to understand what's happening and not using iperf now and talk about performance numbers). You used a workaround that simply works since the real problem is not even touched.

[MarkHaysHarris777]

I suspect that the reason this works is that the negotiation for GbE does not happen and by default the unit comes up  100 /full.

Just out of curiosity, with a non-modified cable, what happens if you run "sudo ethtool -s eth0 speed 1000 duplex full" ... does it give the ethernet a kick to reboot and re-negotiation as GbE/1000Mbs? I'd try myself... but as you know... it's a non-issue for me. However, I can switch between 100Mb and 1000Mb that way.

@pfeerick
hi, good question... with the unmodified cable I can use the ethtool to set speed 1000 duplex full; but, it doesn't work at all !   With the modified cable the link comes up 100 /full, which is why its working so well; if I use the ethtool to set speed 1000 duplex full it DOES go into 1000 full for about 20 seconds and then the link goes down  (this is reproducible).   If I bring the linke back up manuall without reboot [ ifconfig eth0 up ]  the link comes up 100 /full.

The temporary good news here is that while we're trying to figure out what needs to be done to the card, the 100 link comes up automatically now without intervention so that (in my case only) the GbE switch is now useful. The machines (2) that can use it do, and the one that cannot use it comes up 100 by default... so my wired GbE network just had an order of magnitude improvement !

---

Using my ubuntu image and the 'bad card' connection to the GbE switch is now working !  It went from not working at all on the GbE switch to running at 95mb/s !

( we're getting closer people,  we're getting closer ! )

You're not getting closer, you just forced Fast Ethernet which is something completely different as I already tried to explain to you yesterday (read through the 'zillions of words and this time try to understand). Add 'ethtool -s eth0 speed 100 duplex full' to /etc/rc.local prior to the 'exit 0' line and you get the same without fiddling around with your cable.

The real problem is still a negotiation problem in GbE mode (which is something completely different than Fast Ethernet and that's the reason I already warned exactly you yesterday to try to understand what's happening and not using iperf now and talk about performance numbers). You used a workaround that simply works since the real problem is not even touched.

@tkaiser
Thank you tkaiser for being a student of the obvious, but unfortunately not a student who reads the history of this problem !  WE are all very aware of the point that you just posted here.  I have been recommending the "while we wait" work-around of using ethtool for setting the speed to 100/full and have been pointing people to that for weeks.

WE are also painfully aware that this is a negotiation problem... what has not been clear until recently was whether the problem was hardware or software;  it is almost certainly hardware related at the root cause.

But WE have moved forward ( from a work-around standpoint ) because now the units that do not work at 1000 GbE/full will now work at 100/full 'by default' without intervention.  While I'm working on this, my GbE switch just became useful... !   This is a big deal...  

---

For everyone else:

Take a look at  www.hardwaresecrets.com/how-gigabit-ethernet-works/

All eight wires ( all four pairs ) are required for gigabit ethernet to work. So, modifying your Cat6 cable , or using a 2 piar cable is only a temporary work-around ( albeit an automatic work-around ).

marcus

[pfeerick]

hi, good question... with the unmodified cable I can use the ethtool to set speed 1000 duplex full; but, it doesn't work at all !   With the modified cable the link comes up 100 /full, which is why its working so well; if I use the ethtool to set speed 1000 duplex full it DOES go into 1000 full for about 20 seconds and then the link goes down  (this is reproducible).   If I bring the linke back up manuall without reboot [ ifconfig eth0 up ]  the link comes up 100 /full.

The temporary good news here is that while we're trying to figure out what needs to be done to the card, the 100 link comes up automatically now without intervention so that (in my case only) the GbE switch is now useful. The machines (2) that can use it do, and the one that cannot use it comes up 100 by default... so my wired GbE network just had an order of magnitude improvement !

Interesting... certainly once again reinforces that the issue is GbE auto-negotiation related. The behaviour of ethtool forcing GbE is understandable with the modified cable, since you basically disabled GbE capability by removing the two normally unused pairs, and after some thinking, the RTL8211 realises something is wrong... tries again, and establishes the link at 100Mb. However, I certainly won't be recommending this fix to anyone as it is a waste of a perfectly good GbE capable cable. If adding 'ethtool -s eth0 speed 100 duplex full' to/etc/rc.local is all that is needed, I would consider that as the best compromise until (if) a solution is found. However, it is interesting as a data collection point, as it did force the auto-negotiation to not consider GbE, and then allowed things to work.

[MarkHaysHarris777]

 However, I certainly won't be recommending this fix to anyone as it is a waste of a perfectly good GbE capable cable. If adding 'ethtool -s eth0 speed 100 duplex full' to/etc/rc.local is all that is needed, I would consider that as the best compromise until (if) a solution is found. However, it is interesting as a data collection point, as it did force the auto-negotiation to not consider GbE, and then allowed things to work.

Yes, and no.  I agree that I disabled a perfectly good Cat6 cable (new, at that!)    I purchased an entire bag (10) new very expensive Cat6 eth cables just for testing... and I tried each and every one of them just to make sure that this was not some kind of cabling issue (its not!). So, yes I modified one of those cables, but I did not waste it (all 10/100 connections never use the pairs I pulled from the cable); in other words, this cable will work perfectly in any 10/100 network.  And modifying the cable is easy, and fast. And then, the cable just works... no futzing around with the ethtool; nor automating a script. 

So, yes, I will be recommending this work-around to folks, but, I will not be calling it a fix!  It is not a fix, it is a work-around only .

[androsch]

Dear Marcus,

calling the use of a crippled piece of hardware to do what a normal software tool at the startup can do a workaround for the GbE problem is just nonsense. Sorry, but it is.

We all should concentrate on what causes the problem and for this its necessary to collect reliable and approved data.

I just tried the suggestions e.g. tkaiser made here, because he really seems to know his part of it, and even with this sessions and checks my board wasn't capable to use GbE as sender. So it seems to be a part of the TX settings (please correct me when i'm wrong) and maybe he can sort this out by exchanging our boards and test.

I agree at the moment, there only seems to be a good workaround by fixing the ethernet port to a 100mbs speed, but this should be done with the right tools and not by seducing hardware parts.

Gesendet von meinem ME371MG mit Tapatalk

[tkaiser]

what a normal software tool at the startup can do a workaround for the GbE problem

We were at this stage since the early days and ethtool has been included into some OS images for exactly and only this reason: http://forum.pine64.org/showthread.php?t...ol#pid5947

Back at that time none of the developers was able to reproduce the issue (both 1GB developer samples that arrived here never showed any problems regarding GbE and I tried it out against 2 GbE routers and 4 different GbE switches) and unfortunately back at that time a few other Ethernet issues existed (Android never booting when Ethernet cable connected on first boot, Ethernet driver crashing in Linux on 2 GB boards and so on) and also unfortunately even moderators all the time suggested the wrong things to users ('Use the workaround! Don't help us nail the problem down!)

So what we need now is someone collecting clear descriptions of the issue ruling out the other potential issues that lead to horribly low iperf numbers first (just like you did!).

GbE works completely different at the PHY layer than Fast Ethernet, that's also the reason why auto-negotiation is mandatory with GbE since there 2 connected GbE PHYs have to establish a master <--> slave connection and sync their clocks and so on. If anything happens at this point for whatever reasons the symptoms so many are telling are normal. I experienced stuff like that maybe 10 years ago at a customer where the local admin followed advises from last century, disabled auto-negotiation on the new GbE switch and all sorts of bad things happened.

I hope that I can reproduce the issue when your board arrives. But if two minor issues are combined (instable clock here and missing negotiation there) weird things might happen and maybe your board will work here without any problems. Let's see. In the meantime quoting the IEEE standard:

All 1000BASE-T PHYs shall provide support for Auto-Negotiation
(Clause 28) and shall be capable of operating as MASTER or SLAVE.
Auto-Negotiation is performed as part of the initial set-up of the link, and
allows the PHYs at each end to advertise their capabilities (speed, PHY
type, half or full duplex) and to automatically select the operating mode
for communication on the link. Auto-negotiation signaling is used for the
following two primary purposes for 1000BASE-T:

a) To negotiate that the PHY is capable of supporting 1000BASE-T half
duplex or full duplex transmission.

b) To determine the MASTER-SLAVE relationship between the PHYs at
each end of the link. 1000BASE-T MASTER PHY is clocked from a local source.
The SLAVE PHY uses loop timing where the clock is recovered from the received data stream.