SOPine modules stay off at reboot?

[Dreamwalker]

I also have this issue base board is fine but in clusterboard it fails.

[Pine]

Hi all, I'm desperately working on this issue too in 2021. It's so perplexing, so much so that I've taken to document my long investigation into this: https://ericdraken.com/a64-reset-problem/

What I've got so far in a nutshell is:
- a single SOPINE reboots in the baseboard.
- a single SOPINE does not reboot in the clusterboard.
- a single SOPINE does reboot in the clusterboard if the 2xAA batteries are present.

Any ideas?

[poVoq]

No ideas, but thank you very much on trying to figure this (and other stuff) out.

I really wish Pine64 would release an updated version of the Clusterboard. As it stands, it seems to have many hardware related issue, especially the v2.2 that I have.

[dsimic]

Hi all, I'm desperately working on this issue too in 2021. It's so perplexing, so much so that I've taken to document my long investigation into this: https://ericdraken.com/a64-reset-problem/

First of all, thank you very much for writing such an extensive article!  It must have required a whole lot of elbow grease and many sleepless nights.

Your article concludes that two AA batteries in a powered-on Clusterboard will become dead in about a couple of months, as a result of the power draw from batteries being much higher when the Clusterboard is powered on.  Obviously, discharging batteries while the system is taking power from the wall makes very little sense, so I followed along and researched the relevant parts of the Clusterboard, SOPine, SOPine Baseboard and Pine A64-LTS schematic.  Below are some of my conclusions.

You've also raised a question about the missing OD4 diode in the SOPine schematic, and you've actually confirmed that the OD4 is physically present on the SOPine module.  Moreover, a complete translation of a mixed-language note in the SOPine schematic, near the circled OD4, is as follows:

Note: R9717 DELETE increases OD4

I'm not sure where the R9717 exactly is; there's one on the SOPine Baseboard, but it isn't marked as NC.  Though, it's also circled in the SOPine Baseboard schematic and labeled as "debug", so it's probably the right R9717.  Anyway, this just additionally confirms that the OD4 is populated.

Furthermore, if I'm not wrong, there's nothing that can stop the combined VCC_RTC outputs of the AXP803 PMICs on all seven SOPine modules to attempt charging the two AA batteries?  It would get even worse once the batteries become weak, so they can no longer "fight" against the VCC_RTC outputs.  Am I wrong there?

[Pine]

Hi all, I'm desperately working on this issue too in 2021. It's so perplexing, so much so that I've taken to document my long investigation into this: https://ericdraken.com/a64-reset-problem/

First of all, thank you very much for writing such an extensive article!  It must have required a whole lot of elbow grease and many sleepless nights.

Your article concludes that two AA batteries in a powered-on Clusterboard will become dead in about a couple of months, as a result of the power draw from batteries being much higher when the Clusterboard is powered on.  Obviously, discharging batteries while the system is taking power from the wall makes very little sense, so I followed along and researched the relevant parts of the Clusterboard, SOPine, SOPine Baseboard and Pine A64-LTS schematic.  Below are some of my conclusions.

You've also raised a question about the missing OD4 diode in the SOPine schematic, and you've actually confirmed that the OD4 is physically present on the SOPine module.  Moreover, a complete translation of a mixed-language note in the SOPine schematic, near the circled OD4, is as follows:

Note: R9717 DELETE increases OD4

I'm not sure where the R9717 exactly is; there's one on the SOPine Baseboard, but it isn't marked as NC.  Though, it's also circled in the SOPine Baseboard schematic and labeled as "debug", so it's probably the right R9717.  Anyway, this just additionally confirms that the OD4 is populated.

Furthermore, if I'm not wrong, there's nothing that can stop the combined VCC_RTC outputs of the AXP803 PMICs on all seven SOPine modules to attempt charging the two AA batteries?  It would get even worse once the batteries become weak, so they can no longer "fight" against the VCC_RTC outputs.  Am I wrong there?

Thank you, my friend.

Since I found the diode, I've added about 20% more to that page I made. I tried to model the schematic as well. One discovery I made is that PWROK drops to 2.25V without the RTC battery, but stays at 3.25V (+0.25V due to voltage from the 74LVT04) with an RTC battery. In the baseboard, PWROK stays at 3.00V, of course (no 74LVT04).



About the RTC battery fighting a losing battle, it looks to power all 7 SOPines, yes, then gets weaker, and then gets overpowered by the combined VCC_RTC output of the 7 SOPines. The baseboard solves this with diode OD3. I'll opt to not use batteries going forward.

I've ordered some custom hardware from Ali to edit the PCB trace lines to test fix ideas.

[dsimic]

Since I found the diode, I've added about 20% more to that page I made. I tried to model the schematic as well. One discovery I made is that PWROK drops to 2.25V without the RTC battery, but stays at 3.25V (+0.25V due to voltage from the 74LVT04) with an RTC battery. In the baseboard, PWROK stays at 3.00V, of course (no 74LVT04).

I went through the current state of your web page, of course, and your latest discoveries are great!  It is really satisfying to see both hardware (i.e. manual) and software resets working, after putting so much time and hard work into the analysis and debugging.  Once again, an awesome job!

About the RTC battery fighting a losing battle, it looks to power all 7 SOPines, yes, then gets weaker, and then gets overpowered by the combined VCC_RTC output of the 7 SOPines. The baseboard solves this with diode OD3. I'll opt to not use batteries going forward.

Exactly, the two AA batteries are connected directly to the VCC-RTC pins of the A64 SoCs on the SOPine modules on one side, and to the VCC_RTC outputs of the AXP803 PMICs on the SOPine modules on the other side, so nothing prevents the combined VCC_RTC outputs of the seven PMICs to start charging the AA batteries once they get weak, which is certainly very bad.  This power path should be always disabled, regardless of any further modifications, if you agree.

Adding an SMT Schottky diode (the one already in the SOPine schematic, XBS104S14, would be fine) would fix the above-described issue, which is actually rather easily doable due to the Clusterboard PCB layout.  If you follow the pad to which the plus pin of the battery holder is soldered, there is a very long trace on the top side of the PCB that goes along almost the entire left-hand side of the PCB.  Cutting that trace and soldering in an XBS104S14 diode is all that's needed to prevent the AA batteries from becoming charged.

With the additional SMT Schottky diode in place, two AA batteries should provide about 2.7-2.8 V (when new) to the VCC-RTC pins of the A64 SoCs on SOPine modules, but the actual RTC backup functionality (as concluded in your research, and confirmed in the A64 datasheet) should work just fine with as low as 1.8 V.  In other words, batteries would work fine (and not just while they're brand new) as the RTC backup (i.e. while the Clusterboard is powered off) with the additional SMT Schottky diode on the Clusterboard.

My desired outcome is to have the resets working properly, and to have two AA batteries in the existing battery holder as the RTC backup, without the batteries becoming discharged (or, even worse, charged) while the Clusterboard is powered on.  As a note, adding the above-described SMT Schottky diode would probably prevent the batteries from being a fix for the reset issue, but the diode must be added anyway.  Based on your latest research and my own further investigation of the SOPine and Clusterboard PCBs and schematics, there are two ways for accomplishing the desired outcome, and I'll describe them below.

An external 3.3 V source could be used, for which an input is already conveniently routed from the Clusterboard's 24-pin ATX connector to one of the R351 pads (more precisely, the right-hand one).  As we know, R351 is the large resistor that can be optionally soldered to the PCB to provide a dummy load for the connected ATX power supply, and is usually not needed.  It is rather easy to solder a thin wire that connects the right-hand pad of the R351 and any point in the above-described long trace that's after the additional SMT Schottky diode.  Of course, another through-hole Schottky diode needs to be placed "within" that jumper wire, which is also rather easy to do with some heatshrink tubing, and nearly everything can be tucked away rather neatly on the bottom side of the Clusterboard PCB.

The above-described way is particularly handy when using an ATX power supply to power the Clusterboard.  It can also be used with a completely separate 3.3 V power supply, or with a 3.3 V "tap" (i.e. a separate step-down converter) going from an already existing 5 V power supply that powers one or a few Clusterboards.

As you've already described it on your web page, another way is to tap into an already existing, internally regulated 3.3 V power source on the Clusterboard.  As you've already pointed out, DVDDIO can be used as a 3.3 V source; as a note, the U1 (RT8284) is, according to the RT8284 datasheet, configured to give 0.923 * (1 + R3 / R7) = 0.923 * (1 + 26.1 / 10) = 3.332 V at its output.  Also, the RTL8370N switch ASIC should take around 2.8 W or 0.9 A from the U1, which leaves us a lot of headroom until reaching the U1's maximum output current of 2 A. 

However, I wouldn't take 3.3 V from the cathode of D1; instead, I'd simply remove a small part of the solder mask from the thick PCB trace that the pin #3 of the U1 is soldered to, and solder a thin wire to it.  The wire can be threaded through one of the pin holes of the unpopulated J33 ("Molex") connector, routed underneath the PCB, and threaded through the left-hand pin hole of the unpopulated R373, to bring it again to the top of the PCB.  The other end of the wire needs to be soldered to any point in the above-described long trace that's after the above-described additional SMT Schottky diode.  Another through-hole Schottky diode needs to be placed "within" that jumper wire, as already described above.  Also, this approach doesn't require the battery holder to be removed, which is a plus.

I would love to hear your comments.

[poVoq]

Hmm, I am not nearly deep enough in this stuff to fully understand it, but wouldn't it also be possible to simply connect a fake 2xAA battery (3.3V or less?) to get the desired effect of a rebootable clusterboard? I don't really care about the RTC backup, and I am hesitant to start soldering additional components to it myself.

[dsimic]

If you don't need the actual RTC backup functionality and you want to avoid applying hardware modifications to your Clusterboard, you could connect an external regulated 3.3 V DC power supply to the two pins of the battery holder, with a mandatory Schottky diode "inside" the "+" wire that connects the "+" of the external power supply to the "+" pin of the battery holder.  The purpose of the additional diode is to ensure that no current will flow from the PMICs on the SOPine modules into the external power supply, which would be really bad.

Even in this case, you'd need to solder two wires to the battery holder pins, or to somehow affix the wires reliably to the battery terminals inside the battery holder.

[Pine]

Adding an SMT Schottky diode (the one already in the SOPine schematic, XBS104S14, would be fine) would fix the above-described issue, which is actually rather easily doable due to the Clusterboard PCB layout.  If you follow the pad to which the plus pin of the battery holder is soldered, there is a very long trace on the top side of the PCB that goes along almost the entire left-hand side of the PCB.  Cutting that trace and soldering in an XBS104S14 diode is all that's needed to prevent the AA batteries from becoming charged.

That is clever. This is the line you are talking about. An SMD Schottky diode would work. With a microscope, an Exacto knife, and a bit of patience, it will work. One issue I had with soldering is that the whole top layer is a heat conductor, so I had to tin my soldering iron and apply flux to the board and only transfer solder from the tip to the SMD, not heating up the pad first. About one second of touching the soldering iron should do it.



Another idea I was toying with is to uncrimp the +RTC terminal in the holder with a fine jeweler's screwdriver or Exacto knife, pop out the tiny center piece of metal (the battery terminals are rigid loops), and interpose a diode with a new + contact for the holder.


> An external 3.3 V source could be used, for which an input is already conveniently routed from the Clusterboard's 24-pin ATX connector to one of the R351 pads..

Exactly. What I'm actually going to do personally is, I'm going to use the Molex connector where I already supply 5V and add a 3.3V line on Molex pin 2 with barrier diodes to batteries. R351 (placeholder) runs near the +RTC, so another barrier diode will fit nicely there with a jump wire as you described. 


> With the additional SMT Schottky diode in place, two AA batteries should provide about 2.7-2.8 V (when new) to the VCC-RTC pins of the A64 SoCs on SOPine modules, but the actual RTC backup functionality (as concluded in your research, and confirmed in the A64 datasheet) should work just fine with as low as 1.8 V.  In other words, batteries would work fine (and not just while they're brand new) as the RTC backup (i.e. while the Clusterboard is powered off) with the additional SMT Schottky diode on the Clusterboard.

Min 1.8V supplying 0.45mA to all 7 SOPines while powered off according to the A64 specs will do it.


> Also, the RTL8370N switch ASIC should take around 2.8 W or 0.9 A from the U1, which leaves us a lot of headroom until reaching the U1's maximum output current of 2 A.

I popped in 7 SOPines and ran heavy networking and the switch didn't suffer. When powered, the 7 SOPines draw 3mA from RTC. Lots of headroom left.


> However, I wouldn't take 3.3 V from the cathode of D1; instead, I'd simply remove a small part of the solder mask from the thick PCB trace that the pin #3 of the U1 is soldered to...

That will work. You could even solder a thin wire on L1. I only chose D1 because it is after the LC filter (the U1 voltage out waveform is noisy so a big choke is needed), there is a bank of capacitors available after that L1 choke (power cycle but keep RTC powered for a few seconds), and D1 is closest to the +RTC terminal.

I'd personally remove the holder. A super-easy way is to uncrimp the terminals as described before, lift the holder straight up and out, and then clip the terminals or put the holder back if you add another diode. You don't even need to unsolder the battery holder. Pretty neat.

[dsimic]

That is clever. This is the line you are talking about. An SMD Schottky diode would work. With a microscope, an Exacto knife, and a bit of patience, it will work. One issue I had with soldering is that the whole top layer is a heat conductor, so I had to tin my soldering iron and apply flux to the board and only transfer solder from the tip to the SMD, not heating up the pad first. About one second of touching the soldering iron should do it.

Exactly, I was talking about the vertical PCB trace in your picture above (thanks for the picture, by the way).  As a note, there's something even better than an Exacto knife when it comes to removing solder mask: a fiberglass brush pencil.  With a stencil made of Kapton tape, it should be possible to use a fiberglass brush pencil to make solder pads that are almost of the same quality and precision as the factory-made ones, with virtually no risk of damaging the traces.

Another idea I was toying with is to uncrimp the +RTC terminal in the holder with a fine jeweler's screwdriver or Exacto knife, pop out the tiny center piece of metal (the battery terminals are rigid loops), and interpose a diode with a new + contact for the holder.

I had another look at the battery holder, the battery terminals and the battery terminal pins, and I'm not really sure how would that work?

The "+" pin is crimped underneath the collar of the "+" battery terminal, so you'd need to squeeze in the radial fins at the other side of the "+" terminal in order to pull the battery terminal out of the battery holder enough to free the "+" pin.  Also, the freed "+" pin would stay physically inside the battery holder.  Where would you put the additional barrier diode, what would serve the purpose of the new "+" battery terminal, and how would you connect the diode, the pin and the new "+" battery terminal?

Exactly. What I'm actually going to do personally is, I'm going to use the Molex connector where I already supply 5V and add a 3.3V line on Molex pin 2 with barrier diodes to batteries. R351 (placeholder) runs near the +RTC, so another barrier diode will fit nicely there with a jump wire as you described. 

Just to make sure, you're referring to the 24-pin ATX connector, right?  Also, if you place a barrier Schottky diode on the external 3.3 V line, i.e. between the 24-pin ATX connector and the external 3.3 V power supply, you'd need another Schottky barrier diode on a Clusterboard only if you intend to also use two AA batteries inside the battery holder, to provide the RTC backup power.

Min 1.8V supplying 0.45mA to all 7 SOPines while powered off according to the A64 specs will do it.

Exactly.  As a result, the two AA batteries can be used "down to the wire" in a properly modified Clusterboard, ensuring their long usable life.

I popped in 7 SOPines and ran heavy networking and the switch didn't suffer. When powered, the 7 SOPines draw 3mA from RTC. Lots of headroom left.

Excellent, thank you for the experimental verification.  In theory, there should even be just enough headroom to power another RTL8370N switch ASIC from the same U1 regulator (2 * ~0.9 A < 2 A), as we may pretty much not count the other components that are fed by the U1.

That will work. You could even solder a thin wire on L1. I only chose D1 because it is after the LC filter (the U1 voltage out waveform is noisy so a big choke is needed), there is a bank of capacitors available after that L1 choke (power cycle but keep RTC powered for a few seconds), and D1 is closest to the +RTC terminal.

Now I see my mistake, thank you very much for pointing it out!  Tapping into the output of the U1 before the L1 was a bad idea; actually, I'd never take that route, but the large PCB trace around the pin #3 of the U1 was, visually, so tempting that it became misleading.

After having another close look at the Clusterboard PCB, there is a pretty large PCB trace on the "good" side of the L1, on the top side of the PCB.  Even better, there's also a trace in form of a small "island" on the bottom side of the PCB that is also connected to the "good" side of the L1 and serves to accommodate three members of the capacitor bank (C7, PC6 and PC7); I suppose they had to do it that way because the capacitor bank is made of a rather large number of capacitors.

The above-described "3.3 V island" is highlighted in the attached picture.  Using that "island" would be even better than using the traces on the top side of the PCB, because the jumper wire would be visible only once on the top side of the PCB.

Moreover, using the "3.3 V island" could allow the entire jumper wire to be placed on the bottom side of the PCB, by placing the additional barrier diode on the bottom side of the PCB, on the short part (before the first via, for the BAT-RTC pin of the SOPine #0) of the horizontal continuation of the long vertical trace in your picture above.  However, I'd rather not take that route, as the jumper wire would then go across a zillion of components; the jumper wire will carry DC so it should do no harm, but it's better to stay on the safe side.

I'd personally remove the holder. A super-easy way is to uncrimp the terminals as described before, lift the holder straight up and out, and then clip the terminals or put the holder back if you add another diode. You don't even need to unsolder the battery holder. Pretty neat.

Quite frankly, I don't see that crimping back the original battery terminals would be an easy thing to do.  You could use needle-nose pliers to do that, but holding everything together while bending back the radial fins, applying pressure so the crimping makes good electrical contact with the "+" pin, and ensuring that the battery holder doesn't move at the same time, damaging the "+" pin...  It might be doable, but I doubt that the final result would be as good as new.

By the way, why would you actually need to remove the battery holder and put it back later?  What is not accessible with the holder still in place?  Also, I don't think that an additional barrier diode, for example, can be tucked underneath the battery holder.