1Gbps clock sync

[stepw]

Other notes:
After executing d) above, very poor ping response from other device on network. Cable re-insertion made no difference whatsover. 

Thank you so much pfeerick!

So that confirms:
1. Internal EMAC transmit clock (on A64 SoC) is used both on boards working well at 1Gbps and those that don't
2. External EMAC transmit clock (from RTL PHY or a separate source) is resulting in severe packet loss on both too

On "good" boards EMAC and PHY appear to be passing data reliably while using their respective internal clocks, resulting in no data loss and reasonable 1Gbps performance, which could reach link capacity with some software tweaking. On "bad" boards, like mine, looks like there could be a clock/timing issue between EMAC and PHY in Pine64's configuration and board layout, resulting in major loss of data as it transitions between layer1 and layer2 or vice-versa. Perhaps clock skew is outside of acceptable range or internal 125MHz clock is not stable either on EMAC or on PHY. It may or may not be related to voltage anomalies, I don't have the skills or tools to confirm voltage as the root cause.

Per RTL8211E DS, a 125MHz clock reference can be supplied by PHY to EMAC for clock to be in sync:
1. Features:
Supports 25/50MHz external crystal or OSC
Provides 125MHz clock source for MAC
2. RGMII mode
In 1000Base-T mode (RGMII interface is selected), TXC and RXC sources are 125 MHz.
TXC will always be generated by the MAC and RXC will always be generated by the PHY.

If I'm reading Rev.C PCB layout diagrams correctly, RTL8211E CLK125 is attached to A64 GCLKIN, which, once external TX clock is enabled for EMAC (with devmem2 or in driver), should have sync'ed the clock between PHY and EMAC. However, enabling it makes things even worse, kind of counter-logical to me.

It's not clear from A64 user guide if its EMAC uses internal clock for 100Mbps communication as well, but associated 25MHz clock is external on PHY and that appears to work well/stable on all boards, regardless of power source and base input voltage.

@stepw,  if you have a scope it might be helpful to probe the PHY_VDD33 voltage while in GbE mode and be looking for a ripple in the voltage (signal on the VDD) ?

Unfortunately I don't have a scope to monitor continuously, but if PMIC AC-IN voltage drops were affecting 1Gbps communications, I'd expect them to equally affect 100Mbps communications and that is not the case. So the issue with my board doesn't seem to be caused by PHY VDD33/AVDD33 input voltage fluctuations. It could be related to 1V fluctuations off of PHY's internal regulator on PHY VDD10/AVVD10 pins, I have not measured that.

I wonder if the inductor tllim suggested could be shorted to fix 1Gbps is part of 1V voltage regulator scheme. Unfortunately he hasn't commented when I asked him in the other thread.