What to do when you’re not hashing anymore, a very frustrating but often simple problem to fix. I’ve put together a quick cheat sheet of common problems that stop the hashing but for today we’ll focus on bad fans.
Bad fans (attached to the miners) are rarely a catastrophic failure, as any reputable miner will shut down the hashing automatically for these failures, however some aftermarket firmware allows you to bypass this safety mechanism. Having nearly lost a miner to overheating through thermal runaway I will advocate never bypass them. Even if you have an external duct van venting your unit I would still (and I practice as I preach) leave the stock fan in and running. They will run at a much lower speed in general, if using an external duct fan, so their noise level will be minimal and the lifespan much greater.
Now what to do if you’re not hashing? Here’s a quick and simple video showing the diagnosing of a fan issue and how to locate the bad fan.
I’ve seen a lot of posts lately, and even got an message or two, asking how I remotely monitor my L7. Since (as of 8/12/22) there isn’t any after-market firmware available, nor would I suggest using any while your system is under warranty, some hacks and work-arounds are needed.
That solves the remote login issue, but what about alerting you when it’s down? There’s scripts you can write that can verify pings every so often, and that will tell you if it’s alive. But what about if it’s hashing?
To solve that I look at it inversely, I want my miner to tell me it’s down, but if that’s not easily available, then I want it to tell me it’s working. So how can we do that? It’s not as hard as you might think, in fact it’s a quick and easy hack. First, this will depend on what pool you mine and what the payout interval is. For example, if you use Litecoinpool you can set a payout threshold to meet a time interval of roughly 4 hours. For Nicehash it just happens every 4-6 hours. Why is this important you might ask?
Now let’s look at your wallet. Pretty much every wallet on earth allows you to receive a notification when you receive a payment. I have a wallet that I use for mining that I have an alert set for any incoming payment. So, knowing I have a payment threshold of 0.1 LTC on Litecoin pool, for one L7 I expect a payout roughly every 5 hours. If I don’t receive a text alert every 5-6 hours from my wallet that a payment was received I can login with VNC and verify that I’m still hashing.
Is it perfect, no, but it works for me and was a quick and easy hack to make sure I don’t have to constantly login to monitor miners. I know there are better hacks, feel free to share, that’s what this community is all about!
How is that for a headline? Certainly nothing you ever want to read or even think about. As you can imagine, I feel very blessed and lucky that the circuit tripped before anything worse could happen, but it’s a very good lesson to learn. Mainly, always inspect you equipment, don’t buy cheap unknown knockoffs, and use circuit protection for everything.
So what happened? I was gone for a few days on a work trip so I can’t say the exact time this happened, but I can ball park it since the duct fan killing the circuit shut down the miner as well. A caveat to that is that the miner only shut down because it has programming to shut it off when it gets too hot, which is another learning point for me.
This happened on an L7 I have and I was using the duct fan as the primary cooling and manually set the fans on the L7 to be at 30% (roughly 3000RPM.) Once the duct fan controller burned out the only thing cooling the L7 was its own fans, which I had set too low to actually cool it, hence it shutting down. Lesson here is that I had no need to manually dial the L7 fans at 30%, I could have left it (and will from now on) at automatic and they simply would speed up if the duct fan ever failed. So what if I had removed my onboard fans like several people have mentioned? There’s more of a risk that you could burn up your hash boards before the software catches them heating beyond threshold (90C I believe on the L7.)
I threw a little video together on my lesson learned and what to watch out for.
We’ve all had miners fail (if you’re here that’s most likely why), it’s frustrating, maddening, but gosh darn-it, people like you. After you’re done staring in the mirror, it’s time to start figuring out what to do. The number one thing I tell everyone I’ve worked with is to save that kernel log! I mean don’t turn the miner off until you’ve gone into the settings and copied and pasted that log into a file you save. The answer to your question may be in there.
Why save the current log? Well, for one thing that’s the “black box” of what happened. All communications between the boards and the mining pool are there and you want to know what the last thing was that happened before things went awry. Most miners (at least ones I’ve worked with) don’t save the log if you power down and reboot, it will instead overwrite with a new one. There are ways to recover it however, but unless you’re a more advanced user that knows how to SSH in, that’s not something that is easy to walk through.
There are many things that the kernel log can tell us that will point directly to the issue at hand, below are just some of the more common ones (and some uncommon) that I’ve come across:
Chain X ASIC 0 !!!
0 ASICS isn’t always a death sentence.
Another very frustrating error with very little data. Luckily most of the time this is a rather simple fix for being such a vague error. This error essentially tells you that when the control board polls the hash board (via I2C bus) it gets either no response or an improper response from the hash boards PIC after it initializes. Sometimes it will even come back with a number like Chain 2 ASIC 23 !!! which points to ASIC 24 as the specific issue, however a ASIC 0 points to a few things we can try to fix.
CHECK VOLTAGES AT THE 10V BUCK CONVERTER AND 14.2V BOOST CIRCUIT.
Sometimes, more often than it should, the boost circuit on the hash board fails and subsequently asics show as missing or ‘xxxxxx’. Check out my walkthrough manual and scroll to the 14.2V Boost Circuit for more info. Furthermore insure the output of the 10V buck converter matches the voltage you’ve set in the firmware for that chain (i.e. common voltages like 9.5V, 9.8V, or stock voltage of 10.11V.)
CHECK VOLTAGES AT YOUR LDOS.
The L3 has 12 voltage domains, each controlled by a single voltage regulator. Most recently Bitmain used an LN1134 but has used an SPX5205 and SGM2202 in the past. In the past these domains have failed when the 14.2V circuit failed so it’s a good idea to check the voltage at each domain. This can be done by checking the voltage between pin 2 (middle pin on LDO) and pin 1 (input ~2.4V) and pin 5 (output ~1.8V.) Check this at each LDO (i.e. U75, U76, U77…) Check out my walkthrough manual and scroll to power domains for more info.
COLD RESTART THE UNIT.
From time to time intermittent problems like this can be solved by shutting the unit down for 30 seconds and then rebooting. This isn’t a long term fix but may get your unit back up and running for the time being.
LOWER THE FREQUENCY AND INCREASE THE VOLTAGE.
Go into your advanced settings for the problem hash board and try lowering the frequency and upping the voltage to at least 9.8V. Tuning the hash boards to run on minimal speed and power can have the board operating at the edge of its ability to function. Resetting the PIC to a more normal operating condition may solve your problem. Likewise operating at too high a frequency and power can potentially shorten the life of components or operate on the edge of functionality.
RELOAD THE FIRMWARE.
Sometimes reloading the firmware, especially with one that allows autotune, can help isoloate or even fix the problem if it’s with the PIC.
REFLOW THOSE SOLDER JOINTS!
An intermittent connection can change with environmental conditions . Heat and cold can flex cold solder joints and ultimately lead to failures. I’ve found that reheating and reflowing the joints on the temp sensor, buck converter, and PIC have resolved problems I’ve had in the past with missing components.
There are a couple errors that can be associated with fans and they’re pretty straight forward to troubleshoot. The first is below, some older versions of the L3 firmware had this error, most newer and after-market versions don’t however. This points to a failing or failed fan.
Fatal Error: Some Fan Lost or Fan Speed Low!
However as the fans do start to fail, below is the message you may get. For reference fan 1 is the fan plugged into the connector closest to the front of the unit.
Fan Err! Disable PIC! Fan1 speed is too low 390 pwm 44
Fan Err! Disable PIC! Fan1 speed is too low 0 pwm 100
Another thing that points to fan errors is no hashing as the firmware shuts down the hash board if there’s a fan failure to save a thermal runaway condition. Basically, change your fan, eBay is a gold mine for these!
Get Temp Data Failed
This is pretty straight forward and almost always points to a bad TMP451 on your hash board. This error tells you exactly which chain has the failure so you can first try and reflow the solder on the TMP451 (the quality varies) or replacing the chip. I have a link to replacement parts here. The caveat is that this doesn’t always affect hashing and some firmware versions will regularly report this error due to a firmware glitch (per Bitmain.)
Get [1]Temp Data Failed!
Network Errors
Net Err! Frustrating and generally means you’re not making $. Most of the time you may see an error like below:
Net Err! lastest job more than 2 mins! waiting …
This is most likely due to one of three common problems.
You have a poor network connection
Your network cables or router have failed
Your stratum address is entered incorrectly
Number of “x” = 4 of Chain 0 Has Exceeded Threshold (on any Chain…)
This can point to a number of issues. Generally when you see this error in the kernel log that is the point at which the hash board shuts down or, if you have a hash rate watchdog enabled, it will reboot. This is generally due to boards overheating or running too “hot” of a frequency. These issues can generally be remedied by lowering the frequency you’re operating the hash board at.
Scanreg Error
The infamous scanreg/crc5 error can be quite frustrating, but know that the secret to this generally lies in the ASIC chips themselves. The problem generally starts by seeing a series of the following in your kernel log:
bitmain_scanreg,crc5 error,should be 00, but check as 01…..
This is essentially pointing to an ASIC failing its self test. Sadly I haven’t seen a way in the kernel log to point to a specific ASIC, but following my post on CRC5 errors will help you track it down using your multimeter.
In my series of, “I know what I want to see, but what do you want to see?” videos I had someone ask about the installation of basics 120V or 240V circuits. Well, you got it. I’ve done electrical work for years off and on, that along with some University training should hopefully lead to some useful insight in the below video.
Of note, and a disclaimer, I am not a licensed Electrician, so please consult an Electrician, NEC code, or your local inspector before energizing any circuits. If you’re doing this in your home you’ll most likely need to pull a permit for it anyway, but many areas allow homeowners to do their own work as long as it meets code.
Got a question the other day about how to autotune an L3+ with aftermarket firmware. There are several firmwares out there that you can do this with, however I’ll stick to HiveOS as it has the most flexibility I’ve found, all the way down to the individual ASIC level. While HiveOS isn’t the only one out there, and I’m not saying it’s the best, it’s fairly simple and straight forward and easy to learn to use.
When I hear autotune I get a little worried. How does it know exactly what I want or what I need, or how can it change without a complete understanding of the setup. I treat it like a starting point, it’s a good feeler for what the board can handle, but at the end of the day I always go in and manually setup the boards as that will give me the maximum benefit with the longevity of the hash boards I’m looking for.
HiveOS states they attempt to make the hash rate the highest with the lowest power consumption, and they do pretty good. However what they state in the FAQ doesn’t seem to line up with what I’ve experienced it doing. First off, they state they start with a high voltage and work down, it appears the opposite works and in fact sometimes they needlessly raise the voltage to get rid of a few HW errors that don’t really affect performance. So they’re actually costing you money by using more power even though the HW errors were so minimal (#nonce still under 0.03%.)
That being said, it is a good starting point to understand what your unit is capable of.
A buddy called me up and let me know he was having trouble with his L3+. It was resetting every 5-15 minutes, “for no reason.” Well, there’s always a reason so I took this as an opportunity to knock the dust off (literally) my equipment and give it a go.
I’ve learned to take more photos and video as I work through these as it’s easier to speak to when I can just put a video or picture of what I’ve found. Of course I forgot to film as I inspected and found the problem (spoiler alert – DC power connector) but I caught about everything else in the video below.
Part of managing your crypto miner is finding the perfect balance between performance, power usage, cooling, and noise. It’s not often easy, or even fair, how the balance works out. Often times I find myself sacrificing hashing power in favor of cooling and noise (happy wife happy life…)
So let’s talk about that, cooling and noise. Is there a way to pull this off and still keep a reasonably high level of hashing? Short answer, of course yes, long answer is still to come.
I did a little research into the actual air movement with respect to fan speed, and threw in some sound data (db @ 3″) for fun. I didn’t include temperature data because all hash boards and all systems running at all different voltage levels would just add too much confusion into the data (which may already be confusing enough!)
The test data was from a stock L3+, 4 hash boards running at 384 MHz (504 MH/s), using stock 6000 RPM fans, and I ran the fans from 20% speed to 100% speed (set in the firmware.) Each measurement I ran I took 3 different times after resetting the speed in the firmware. This gave slightly different results each time. One dataset was significantly different then I realized my measuring point was off for that set so I tossed it. The CFM was measured at one corner of the intake and exhaust where I saw the greatest volume. I measured the sound (db) at 3″ since I have other miners running and much further out from that it modified the experiment too much. 3″ allows us to track the trend of sounds with respect to fan speed, but won’t give you a relative db level to explain to your family why your basement/garage/shed/man cave is so loud.
Below are graphical reports of the data I collected. What I found most interesting was the actual CFM that I measured the fans at. As you can see the CFM drops off as the RPM drops off.
This is a basic and straight forward view, but to understand the relationship of CFM to RPM I normalized it versus 100 RPM. Basically this tells us that the relationship is directly proportional, i.e. speed up the fan 10% and you get 10% more CFM.
Now what about noise? While we all care about the last two graphs, mama and the family really only care about one thing, noise (two if you count money but I didn’t collect data on happiness versus profitability yet…)
What the chart shows is that you have a significant drop in noise going from 6000 RPM (100%) to around ~3700 RPM (50%.) First off, I know 3700 isn’t 50% of 6000, but that’s up to Bitmain and how they wrote their firmware. But what it does tell me is that if I can tune my boards so that I only have to run around 3700 RPM then I’ve turned my indoor gas lawnmower into pleasant office noise (and minimized my chances at even more hearing loss.)
Also, one bonus chart, ever wonder how to relate fan speed % to RPM in the firmware, here’s what I came up with. While actual RPM did vary slightly, it didn’t seem to vary more than 1% each time I set at each specific fan speed percentage.
The big thing to remember as well, the speeds and CFM (not the db necessarily) are based off the fan manufacturer so these values are good on any miner that uses this stock 6000 RPM fan.
Bypass it, jump over it, leave it in the dust, at least that’s a path that I decided to go down. I was having problems with ASIC 28 on a hash board, and due to (most likely) a lack of skill in soldering I was unable to fix it, Before I caused more damage I decided to write it off, not the whole board, but the ASIC itself.
After looking at the schematics and understanding how the L3+ hash board works I had an epiphany, why not just skip it. The via’s are already there to run the wires from, I wonder…
The firmware doesn’t disallow a board to run without all the ASICs, it just checks that they are present and reports data from them. If it’s not there, it skips it and moves along. So how did it turn out? Spoiler alert, it works fine, just some patience, 30AWG wire, and a soldering iron is all it took.
Bypassing a bad ASIC on a Bitmain Antminer L3+ hash board
I’ve always heard the first thing that fails on an L3 hash board is the 14V boost circuit, I can say that I 100% agree with this. Of all the hash boards I’ve repaired, at least 75% have been fixed by installing an external boost circuit. Normally this problem is evident once you measure the voltage across D1 (anything less than 14V spells out a problem), however sometimes that voltage can be deceiving.
I recently worked on 3 different hash boards that came on two L3+ systems I bought off Ebay. The sellers were clear and honest about there being a bad hash board or two, so I knew what I was getting into. Two of the boards measured over 14V at D1, which normally would tell me the boost circuit was fine, however I now know it wasn’t. The other was more telling, 10V at D1, popped a boost on it and it’s been hashing away fine.
After measuring 14V at the boost I proceeded to measure all other relevant voltages on the hash board, that includes not only checking 14V, but the 10V from the buck controller (measured at L2), 12V input to the board, and the voltage across pins 2 (GND) and 5 (1.8V output) of each LDO. The first 10 domains measured correct at ~1.8V, however domains 11 and 12 came in at 1.1V and 0.5V respectively. My go to was to try and replace the LDO’s since that’s a common failure, however this time it didn’t fix them.
Note – The LDO’s at the last two domains require an LDO that takes a higher input voltage (~5V on 11 and ~4.5V on 12.) Some versions of the LDO work across a wide enough input voltage range that they can be used on all domains, however the most common (LN1134 marked usually as 4VK4) only works with the first 10 domains. This device only works up to a 3V input and will fry if you put it on the last two domains.
After failing to fix the problem my first thought was, “what if, under load, the 14V is dropping.” This would point to the common problem the on-board boost circuit has, and even the inductor (L1) can’t help keep the voltage constant when the output current is less than the draw of the LDO’s. So I popped off D1 (to isolate the on-board boost circuit) and installed an external boost circuit.
I plugged in the hash board, viola, it came right up and started hashing away. I went back and verified the voltages at the last two LDO’s and they both measure a clean 1.8V output. I put together a quick video to show the steps I went through:
https://youtu.be/5abmjlW9_4I
Lesson learned, even if the boost circuit says ~14V, it doesn’t mean under load this isn’t dropping. You really can’t fully test this unless the board is actively hashing and the LDO’s are drawing current from the on-board boost. If you measure 14V but don’t get 1.8V from the last two LDO’s, consider it might be the boost circuit instead of bad LDO’s, or even both…
One of the challenging things with working on an L3+ hash board is the lack of a full set of schematics or gerber files available for the printed circuit board (PCB) to figure out the full function of the board. I’ve reached out many times to Bitmain but it’s not high on their list to release, guess they don’t want anyone reverse engineering the board…
Where does that put us, well, luckily the L3 has a huge community of folks constantly working on the hash boards and reverse engineering the board. RDC has done a great job of putting together schematics of some of the things the Bitmain manual lacks.
I also decided to dive a little deeper, with that I mean inside the board. I attempted to x-ray the PCB, hoping to gain a little more insight into the trace routing. The issue I ran into was the resolution of the x-ray and the energy put out was too high, washing a lot of it out. If that makes little to no sense, it’s like trying to watch an HD movie on an old tube TV. Regardless, some good came out of it and if you zoom into specific areas you can see some of the traces running to various points.
X-ray of L3+ Hash Board
I built a high res video that shows the location of all components (and silkscreening on the board) for both the top and bottom. I will make the raw files available as well since it’s near impossible to find top side components without removing heatsinks. Luckily I have my practice board available…
Layer 1 of L3+ Hashboard PCB – Courtesy of RDCLayer 2 of L3+ Hashboard PCB – Courtesy of RDC
What is that I see above? That’s ASIC 28 on an L3+ hash board. You probably can’t tell what’s going on here, but I’ll attempt to explain why the image above is bad and I promise I have a better one below.
I was working on a hash board that showed the infamous “0 asics found” and eventually, through blood, sweat, anger, hunger, and tears found that ASIC 28 was actually bad and wound up affecting the entire hash board. Sometimes the ASIC tester points to the exact cause, but most of the time it’s just you, a multimeter, and a worthless $200+ hash board test jig.
Let’s fast forward to me soldering on my first DFN (dual-flat no-leads) package. At first they look intimidating, but once you get the hang of it they’re actually quite simple to put on. The only thing you have to worry about (which bit me in the end, hence why I’m writing this) is that if you don’t have enough solder on the power and ground pads, you won’t get a good enough connection for the ASIC to function.
I followed a few videos online (from Antminer Repair) to learn the basics, but since I didn’t have the fancy solder stencil, I was forced to do my favorite thing, improvise.
I started the painstaking process of removing the old ASIC, which can be a problem itself. Bitmain doesn’t use a low temp solder so you really have to heat things up for a while before the ASIC will give up its death grip. Unfortunately I did a little damage to the PCB (printed circuit board) when removing it, fortunately it was a NC (no connect) pin. You’ll also want to remove the heat sinks from the ASICs around the one you’re replacing so you can get some room to work. About 30 seconds from a heat gun is enough to pop these off easily.
Luckily Pin 15 (torn pad) is a NC (no connect) and on the side you see some copper showing from me prying the old ASIC off.
After following the advice from Antminer Repair on Youtube I tinned both the pads on the PCB and on the ASIC itself. I then placed the part on and heated it until it self aligned and slowly held it in place until the solder hardened. Sounds way too simple, but easy day, right?
After cooling and cleaning I went and plugged the hash board in, nothing, “0 asics found”, I was defeated. I broke out my multimeter and started measuring all the signals (RI, RST, CLK, BO, and CO.) I even measured out the resistance between pins and pads to make sure I hadn’t damaged anything. I was at a loss. Finally it dawned on me, maybe there’s poor connections from the ASIC to the PCB. I don’t have x-ray vision, so I got the next best thing, an actual x-ray.
ASIC 28 (center) with poor connection to power and ground due to me being skimpy with the solder.
As you can see, or maybe not, look at the ASIC above and below, there is a large solid mass where the power and ground planes are. Now look at the one in the middle, not so much. My aha moment was here, without proper power and ground this puppy wasn’t going anywhere.
I reflowed the ASIC and removed it, added a bit more solder, and viola, 72 ASICS found, my dream come true.
Don’t skimp, don’t be cheap, and practice practice practice (or do what I’m going to do in the future and pay a professional.)
Like many of you I’ve had some frustrations with my L3+ units, mostly surrounding the hash boards and various failures. Bitmain previously released an L3+ Maintenance Guide in 2018 that did a great job summarizing some basic failures, how to identify, and then the Internet took over with the various fixes. I’ve reviewed Bitmain’s manual many times and have boiled down most of the pertinent info below, as well I’ve added my two cents. Make that more than two cents, I hopefully dive into some of their topics a little deeper and attempt to explain it in more detail and make it easier to understand. Their manual can be cryptic and it’s missing a lot of the middle pieces tying some things together so I attempted to fill in the blanks as well. Some of the graphics and schematics I pulled from their document so I’d like to acknowledge them here.
This is meant to be a living document so I’ll update it as I find new solutions to problems. Furthermore there are several areas where I couldn’t find enough information or wasn’t able to reverse engineer how something functions so I denoted those with an assumption note.
I’m all about sharing so I’m putting this all out there to help everyone out, but please, if you re-use some of this give me a shout-out. I’ve had things plagiarized in the past and while it’s not anything earth shattering, it’s also a work in progress so I don’t want someone putting info out there that’s not complete nor something they can explain as to how they came to a specific conclusion. It not only makes them look bad, but it can cause distrust in the data being put out there. Long story short, share it and shout outs.
A general note regarding testing voltages at the various test points. You must have the hash board power and I/O cable plugged in and a working L3 control board or test fixture. The PIC (which provides the signal to turn on the 10V) won’t initialize the 10V buck controller until it receives the programming information from the control board, therefore not only the 10V but the 14.2V and the individual power regulators won’t have power to function.
Another quick note, if you don’t have one, getting a hot air rework station will make some of these fixes easier. You can pick up a station complete with a hot air gun and soldering iron for about $50, I use the FEITA 8586 SMD Hot Air Rework Station from Amazon.
PIC (programmable interface controller)
U3 (U74 on V1.6) is a 16LF1704/5 8-bit microcontroller PIC (programmable interface controller.) This is what makes the hash board, well, hash. If this is not programmed properly the hash board may exhibit all sorts of strange errors, anything from missing chips to bad temperature readings, to no or partial hashing, or the board not being recognized at all.
The PIC takes the frequency and voltage information from the L3 Control Board (what you set them at through the software interface) and adjusts the voltage output and frequency for the BM1485 chain. The PIC communicates with the other ASICs and ICs through I2C (SDA, SCL, and GND) which basically is an IC to IC communication protocol.
It is possible to reprogram this from code from your other hash boards which I will cover in another post at a later date (it’s quite an in-depth process and involves using a Pickit3 programmer and a little luck.)
Common Failures related to U3/U74 (PIC)
No 10V found (measured at both sides of C948 or between either pin of L2 and ground.)
The PIC may have dropped its code or you may have cold solder joints in the PIC. There are two methods to replace the code, blissz did a great job walking through folks on flashing a custom firmware that will update this. I haven’t tried this method myself although it seems to be successful and a little more straightforward. The other method is to attach a Pickit3 and manually pull the code from a working hash board in your system then dump it into your bad hash board.
Improper voltage (less than 10V) measured at C948.
Your buck converter may have failed or may have cold solder joints. First try reflowing the joints and if that doesn’t work, replace the part. Depending on the voltage it puts out you may still be able to run at a lower frequency and have it work. I have one board with a failing buck converter (it only puts out around 8V but it varies) and I’ve tested out various frequencies and actually have it working up to 300MHz. That’s more of an FYI and something my curiosity had me research. You should always conduct the repair before operating them in abnormal conditions, I’m just waiting for a replacement part at the moment.
Temperature Sensor
U87 (U88 on V1.6) is a T451/TMP451 temperature sensor. This is powered by the 1.8V voltage regulator in the first chain. You can get failures if the voltage regulator (U86 on V1.6 hash board) is damaged by a voltage spike/surge.
In reviewing the schematic of the temperature sensor, we can see that the T451 uses I2C to communicate on the hash board as well as having its own built in temperature sensor that is attached externally (pin 2(D+) and pin 3(D-)) to ASIC 1 I believe.
TEMP_P and TEMP_N are the PNP or NPN transistor in the ASIC (BM1485) itself that read the temperature. This is most likely a thermal substrate transistor that outputs to the TMP451 for processing then reports this information out via the I2C bus. I also see there are pads across the board for numerous T451’s, probably an enhanced feature they never implemented?
Common Failures related to U88 (Temperature Sensor)
“Failure to get temp data” error in L3+ Kernel log.
Check voltage levels between pin 1 (1.8V) and pin 5 (GND), improper voltage from the first LDO chain can lead to a failure of the temperature sensor working properly.
Your temperature sensor may have failed or may have cold solder joints. First try reflowing the joints and if that doesn’t work, replace the part.
BM1485 ASIC
There are a total of 72 BM1485 ASIC’s on-board the L3+ hash board. There is no data sheet readily available for this but what I have gleaned from info they’ve put out there aren’t too many failures that are easily tied to these (at least nothing that’s easy for a DIY fix.) These are powered by 12 individual 1.8V power domains.
(*Note: assumption) The clock signal comes from Y1, a 25 MHz crystal, and each board communicates with each other through the CI/CO signals for synchronization.
If one BM1485 in the chain goes bad, it may break the communication to BM1485’s down the line. I’ve found that it can either be fairly simple, or really hard, to diagnose these. One key here is that if you get an error that shows only a lower number of ASICs, like 27 ASICs found, then ASIC 28 may be your problem. You can verify this by checking the test points as I have listed under below under the 1.8V Power Domain.
Without a data sheet there is quite a bit of mystery behind how this actually works, but a little reverse engineering and research has led me to assume some of the following (key word is assume):
(*Note: big time assumptions here)
The frequency on each BM1485 (as set in your “Miner Configurations” pages) is set through programming received on CI/CO (command input/output) and RI/RO (respond input/output.) A command string is sent to the BM1485 to set the PLL divide for the various frequency levels on the BM1485.
U1 (ASIC 1) is actually the BM1485 ASIC that the temperature is read from. The TMP451 reads the temperature data from pins 6 and 7 of the BM1485, processes this and returns the information on the I2C bus to pins 15 and 16 of the BM1485.
One question I’m still waiting on Bitmain to answer (not sure if they will ever get to it) is whether or not RF and TF are wired up to each ASIC for the I2C bus.
Common Failures related to the BM1485 ASIC
ASICs showing as “x” or no ASICs found.
Check voltage levels at the following test points on each voltage domain (set your ground lead on the negative size of the tantalum capacitor or ground plane closest to each set of the test points.) Make sure you have the boards plugged into power and I/O cables to the control board. Without this the hash board won’t initialize the voltage domains (if using a text fixture go ahead and press start button or IP_Sig on control board.)
RST – 1.8V
BO – 0V
RI – 1.6V – 1.8V
CO – 1.6V – 1.8V
CLKO – 0.9V
Note: The signals for RST, BO, CO, and CLKO run from from U1 to U72. RI actually runs in reverse from this, going from U72 to U1.
1.8V Power Domains for BM1485
The L3+ has a total of 72 ASIC chips spread across 12 voltage domains (with 6 BM1485’s each.) Each voltage domain is controlled by a 1.8V voltage regulator that has changed over various version of the hash board.
* Version 1.5 and earlier hash boards – One SPX5205M5-L-1-8 1.8V voltage regulator (pkg SOT23-5) for each of the 12 voltage domains (6 – BM1485 ASICs per domain.) These are generally labeled as R18 on the top along with the date code.
* Some versions of the hash board have also used the SGM2202-1.8 (pkg SOT23-5.) These are generally labeled as G49xx on the top (xx is date code.)
* Version 1.6 hash board – U75, U76, U77, U78, U79, U80, U81, U82, U83, U84, U85, and U86 are LN1134A182MR 1.8V voltage regulators (pkg SOT23-5L) for each of the 12 voltage domains (6 – BM1485 ASICs per domain.) The first 10 domains are generally labeled 4VK4. Make sure to use higher input voltage LN1134’s for the last 2 domains as their input voltage from the boost circuit is greater than the other domains. The last two may be labeled as 4AK4.
These are extremely susceptible to damage from a voltage spike out of the boost circuit. To measure the output of each one you need to measure between pin 2 and pin 5. You can measure the input voltage between pins 1 or 3 and pin 5.
Common Failures related to 1.8V Voltage Domains
ASICs showing as “x” or no ASICs found.
Check voltage levels at the following test points on each voltage domain (set your ground lead on the negative size of the capacitor closest to each set of the test points.) Make sure you have the boards plugged into power and I/O cables to the control board. Without this the hash board won’t initialize the voltage domains.
RST – 1.8V
BO – 0V
RI – 1.6V – 1.8V
CO – 1.6V – 1.8V
CLKO – 0.9V
Note: The signals for RST, BO, CO, and CLKO run from from U1 to U72. RI actually runs in reverse from this, going from U72 to U1.
If you encounter a voltage domain that does not output 1.8V on pin 5 of the voltage regulator then it most likely should be replaced. It’s common for these to go out, especially during voltage spikes. The boost circuit has been known to knock the later domains (ASICs 49-54, 55-60, 61-66, and 67-72) when they fail.
This output value is set by the PIC and is derived from the value that the user selects in the firmware (i.e. if you change the chain voltage from the “Miner Configuration -> Advanced Settings” page this is where that voltage changes.)
Improper voltage (less than 10V) measured at C948.
Your buck converter may have failed or may have cold solder joints. First try reflowing the joints and if that doesn’t work, replace the part. Depending on the voltage it puts out you may still be able to run at a lower frequency and have it work. I have one board with a failing buck converter (it only puts out around 8V but it varies) and I’ve tested out various frequencies and actually have it working up to 300MHz. That’s more of an FYI and something my curiosity had me research. You should always conduct the repair before operating them in abnormal conditions, I’m just waiting for a replacement part at the moment.
14.2V Boost Circuit
U111 is an RT8537 switching power supply. This is used to boost the 10V supply (coming through the inductor L2) to 14.2V. It’s generally considered the most finicky part of the hash board itself and seems to fail quite often. Early on Bitmain discovered that components were oxidizing and therefore leading to failure so they started covering the circuit in epoxy. Why you may ask? I have no idea why this oxidizes any more or less than other parts of the circuit, but they seemed convinced this would fix the problem. In the end of the day, nope, they still fail and we still add in external booster circuits.
Basically this circuit boosts 10V from the buck controller to 14.2V. L1 is used to store the energy for the boost and D1 isolates the 14.2V BOOST_OUT through reverse protection. When you install an external boost circuit you have to remove D1 to eliminate the possibility of interference from the old boost circuit.
Failure of the boost circuit can also cause damage to the 1.8V voltage domains.
Common Failures related to the 14.2V boost circuit
Improper voltage at D1 (less than 14V, generally much less.)
Your on-board boost circuit may have failed. Install an external boost circuit and remove D1 to isolate the on-board boost from the external boost.
Your 10V may also be off from the buck controller, this should be researched and solved before assuming the boost circuit is bad.
If your boost circuit is failing it may still show 14V, however the last two power domains may appear to have bad LDO’s (good input voltage however the output is less than 1.8V.) The LDO’s are most likely fine, try installing a boost circuit as this should fix the problem.
Other Components (data later on)
Q10 is a 1AMP SOT-23 package which I believe is a MMBT3904 NPN general purpose amplifier. More on this later once I figure out what’s going on here.
Y1 is a 25.000 MHz crystal that’s used as the clock signal for all ASICs. If this goes out, nothing works, you’ll need an oscilloscope to verify this.
Has anyone else ever been frustrated with an error that simply says, “bitmain_scanreg,crc5 error, should be 00, but check as 01…..” I have, and lacking available source code to trace this down, I’m only guessing to the culprit. The fact that it seems to be looking for something blank makes me think it’s an ASIC check, or possibly something with the PIC.
Disclaimer – This may be hard to follow, it’s sometimes hard to put thoughts as you work a problem down, but it’s a great reference for me in the future and hopefully y’all as well.
Problem – I have a board that doesn’t find any ASICs and obviously won’t hash. The only real clue I had was a long series of:
bitmain_scanreg,crc5 error,should be 00, but check as 01…..
Looking at all the usual places, Reddit, Discord, and google searches only pointed me to people saying the obvious, your board’s bad, send it to Bitmain or a repair shop. Well, regardless of this, I’m determined to figure this out for all of us that want DIY fixes.
TESTING
I’ll start by saying the test fixtures you can buy on Ebay and other sites are basically garbage, at least if you spend $200 on one. I did a quick article on turning your control board into a DIY test fixtureand that’s all I ever use anymore. I always test with the data cable and power cable hooked up, once I press the test button (IP_sig on the control board itself) the PIC initializes the board and I should get ~10V out of the buck converter, 14.2V from the boost, and 1.8V from the LDO’s. A quick check of these voltages show that all are reading perfect, so on to the ASIC by ASIC search. So knowing the proper voltages are present, I moved onto the LDO’s.
All LDO’s tested out perfect, we’ve got ~2.5V in and 1.8V out from each. So much for an easy fix, so now it’s time to get down and dirty, 72 individual checks of the basics, RI and CLK (all the ASICs have test points available on the bottom of the board.) As I moved through each ASIC, starting at 72, I found an anomaly at ASIC 28. I verified proper voltages to RI (1.8V) and CLK (0.8V) at all ASICS until I got to ASIC 28, which has CLK at 0.95V but 0V at RI. ASIC’s from 1-27 also have 0V at RI. Aha! I’m thinking short at this point, so where is the short? I moved to pulling off the heatsinks in that entire row, inspected, reflowed the joints, but still having the “found 0 asics” error in the kernel log.
We now go back to the schematics to see how the chains are organized and if there’s any reason this particular ASIC could take out the chain before it, or is there another one in the chain doing the damage.
I’ve also verified that CO (pin 4) and BO (pin 2) read 1.8V on ASIC28, so if there’s a short to ground causing the 0V at RI (pin 3), it’s not there. If we go back one ASIC in the chain, ASIC 27, we also measure RI as 0V. ASIC 27 outputs RO (pin 26) into RI (pin 3) of ASIC 28. Of note, the two pins on either side of RO are CI (pin 25) and BI (pin 27.) When we go to the schematics we see that CI (pin 25) is the transmit signal that goes to CO (pin 4) of ASIC 28. We also see that BI (pin 27) is shown as tied to ground and is tied to BO (pin 2) of ASIC 28.
OK, that’s a lot of data, but one thing that stands out, if on ASIC 27 BI (pin 27) is tied to ground, can there be a short from BI (pin 27) to RO (pin 26) and would that cause us to see 0V on RI at ASIC 28?
I measured resistance between each of the pins on each ASIC, and verified that each ASIC had continuity from input to output of the next, no issues. Now I’m getting frustrated, I think it’s something with ASIC 28, but that’s a PITA to remove. I’ve tried reflowing to no avail, guess my next step is to attempt to remove it with a heat gun and then replace, this could get interesting.
One hot air reflow knife, 1 pair of tweezers, a lot of flux, and even more patience pays off. It took me a good 2-3 minutes before I was able to pop ASIC 28 off, but I ultimately won. I’ve heard folks say it takes 30 seconds, well, they must have better gear than me. Anyway, after popping ASIC 28 off, cleaning the pads, and cleaning the board I was ready to throw it back into a system and see what we have.
Chain 0, 27 asics found
This was music to my ears, I flipped the board over and measured RI at ASIC 27 and several between that and ASIC 1, 1.8V! As my son would say, “We did it!!!” We found the issue, I looked at the kernel log and all CRC5 errors were gone, and on the miner status page we could clearly see 27 asics found. I felt the biggest sense of relief and victory.
But now the hard part, I’ve never soldered on a new ASIC on one of these. That’s a different story for a different time, but for now we’ve slayed the dragon (the dragon being CRC5 errors.)
SUMMARY
CRC5 errors seem to point to a bad ASIC. If you’re receiving these I would move immediately to testing the voltages (RST, BO, RI, CO, CLK) with power and a data cable installed. You may need to press the test (or IP_Sig) button to start the buck controller so the LDOs are operating so keep that in mind. The board won’t output ~10V from the buck controller without this, and you’ll need to do this every minute or two when the board is in test mode.
If you get an anomaly on one of these output voltages it could be that particular ASIC, or the ASIC up the chain. Shooters choice as to which it can be, but you are much closer to solving this problem.
I recently came across an L3+ hash board that wasn’t producing 10V from the buck controller and therefore no 14.2V from the boost circuit. I know the boost circuit often goes bad, but it’s rare to not see 10V from the buck controller. I was getting 12V power to the hash board, and it was connected with a good I/O cable. I went into the kernel log and it wasn’t being recognized by the control board. I started scratching my head, what would cause a hash board not to be recognized?
Well, it wasn’t long before logic kicked in. If the kernel log doesn’t see it, that means the hash board isn’t (most likely) initializing the PIC, and subsequently not firing up the buck controller to product the 10V needed. I’d never programmed a PIC onboard before, so I got my Pickit3 Programmer, downloaded the Microchip MPLab IDE, and got to work!
In order not to bore you with a lot of writing, I made the following video. I know others have made one too, I just cut to the chase in a lot of it and instead of 30 minutes as I’ve seen from others, you get 5 out of me. I got a lot of great info from customminerservice.com so I wanted to give them a shout out too!
I’m always looking for a project, and a good deal, so I’ll often browse Ebay and see if there’s any cheap “for parts only” complete miners to buy. Most the times these get bid up near complete working unit prices, but from time to time they are either overlooked or scary enough from the pictures that people pass on them. My most recent find I picked up for around $300, complete with all hash boards and power supply.
When I received the unit it was packed well, but dirty, I’m talking that black dust that you can only seem to find in the darkest corner of some mining warehouse. Also a couple boards seemed to have been unplugged for sometime (like they stopped working before the rest) and the control board was broken out of the rails that hold it in place. OK, we’ve got our work cut out for us.
First thing I always check when I buy a miner is the control board. I disconnected this, blew it off with compressed air, cleaned it in an alcohol bath, blew it off again, then hooked it up to my network with a custom power connector I made from an old 12V/600mA DC adapter. It powered up fine and I got lights on the ethernet connector but it never connected to my network and even running a network scan I couldn’t see anything new attaching. Holding down the reset for 10 seconds didn’t clear it, so either we have a corrupt control board or a bad one. I decided let’s go worst case and assume it’s corrupt with a virus so we’ll use the guide that CustomMinerService.com developed to recover the control board. Without stealing his thunder (click on the link to his website for full details), you basically have to complete the following steps:
Solder a jumper wire and 1-10k resistor between two pins on the backside of the control board
Flash Bitmain recovery firmware to an SD card and install this on your control board
Wait a few minutes and pray you get two flashing lights (red/green) on the front
Turn off
Remove the SD card, jumper wire, and resistor
Turn on
It should now boot to Bitmain stock firmware
I’ve obviously skipped and gleaned over some details above, but u/BornShook has it all explained extremely well on his website. I will say I’ve followed this set of instructions several times and it’s worked perfect, and for this latest project that 100% did the trick, so now I’ve got a working control board again! Next step, flashing HiveOS to the control board, which can be tricky with the Bitmain signature checks, however I’ve got an article that takes you through the process step-by-step and I’ve done it dozens of times with success.
Once the HiveOS is loaded it was time to turn to the hash boards and case. One thing I noticed right away was that the fans were installed backwards and were fairly new. I’m not sure if they had some custom venting that had them do that, or just a simple mistake, but we’ll fix that up. Just like the control board, my first steps for testing a hash board include:
Brush off thick dust deposits first (always wear glasses when blowing compressed air)
Blow through board, connectors, and under heatsinks (be careful around heat sinks as you don’t want to get too close) with compressed air
Brush off any remaining dust deposits
Blow one more time
Put in alcohol bath and slosh around, brushing off any remaining thick deposits
Pull out and let drip dry for 30 mins
Blow one more time, making sure to blow out connectors well
Wipe off any residue and inspect board for any obvious defects.
Once I do this for all hash boards, I will install them into the system one at a time and bring the system up. Once I see the board is hashing I will move onto the next. In this case all four came right up so I installed them back into the system, fixed the reversed fans, and brought the whole system up. Slowly I cranked up the speed and ran an autotune
All in all I was into this project for $325, a couple hours of labor, and to my amazement the system is cranking away 14 hours later at 615MH/s with a nonce around 0.01%. That’s a true diamond in the rough.
More and more I’ve seen questions from folks about how many, or how big, of a miner can I fit on a circuit. This is a question that has many unknown variables that I plan on diving into below. Disclaimer – Please keep in mind this is my opinion and not an actual consultation for your mining business, so before you do any wiring or make any changes to your electrical system please consult a licensed electrician.
Now on to the good stuff. In order to answer the question how how many/how much we need to know a little more information; the basics of your panel, the basics of your circuits, the basics of the miner, and anything else you plan to operate.
RESIDENTIAL/INDUSTRIAL CIRCUITS
For starters, each location in question needs to have a load calculation done to insure that you aren’t going to draw more than 80% of the maximum panel rating. As an example, if you have a dedicated 100A panel you can’t draw more than 80A continuous at any time. Since the mining world is in watts, let’s use Ohms law to break that down.
100A Panel @ 80% rule = 80A
Ohms Law tells us that Power = Current x Voltage
80A x 240V = 19,200W
So the key thing to keep in mind before planning out circuits is that you can’t have a continuous load more than 19,200W on this panel.
CIRCUIT/LOAD PLANNING
Now that we know the maximum wattage of our panel, we can use a little math to plan out what our circuits need to be. I’ll make the assumption that whatever miner we are using operates on 240V since that’s the industry standard at the power level they operate on.
There are 2 main amperage ratings for 240V breakers that are used in the mining world (yes I know there are more but these are the most common), 20A and 30A. There are several reasons for choosing one over the other, ranging from length of wire run, number of units per circuit, cost of wiring, and available breaker space. The most important thing to remember about each circuit is that they also have a maximum wattage rating:
20A Circuit @ 80% rule = 16A
16A x 240V = 3840W
30A Circuit @ 80% rule = 24A
24A x 240V = 5760W
MINING EQUIPMENT
So we know the maximum wattage our panel will hold, and we know the maximum wattage different types of circuits hold, only thing we are missing is what type of miner we’re running and what its power requirements are.
Since this is February 2022 I’ll choose two scenarios, the first is using a newly released miner, the second is using a couple generations old miner. I’m partial to Bitmain products so I’ll look at the S19 or L7 for one scenario (~3250W) and the L3++ for the other (~900W, don’t hate on my S9 people!)
We know that our maximum panel wattage is 19,200W, so how many of each miner could we theoretically operate (I say theoretical since we have to split this up in to several circuits and the math may not work out perfect.)
S19/L7 = 3250W
19,200W / 3250W = 5.9
5 S19/L7 = 16,250W
So that leaves us with 2950W available in the panel for other electronics, fans, network equipment, etc.
L3++ = 900W
19,200W / 900W = 21.33
21 L3++ = 18,900W
So this leaves us with only 300W available in the panel for other electronics, fans, network equipment, etc. Of course this is only relevant if you plan on running everything for your operation off this one panel.
FINAL CIRCUIT PLAN
Now that we know what our options are it’s more simple at this point, with a little math for the L3++ scenario.
For the S19/L7 scenario it’s clear that we need 5 – 20A circuits, one circuit for each unit. This allows us to dedicate one circuit per unit and keeps us under the 80% rule for the full panel load (we are only using 16,250W which is only 68%.) Unfortunately you can’t fit two of them per 30A circuit like some other units (you’d need to be closer to 2800W to do this) but we have plenty of space in the 100A panel for 5 20A breakers. Keep in mind that 20A breakers will use 12ga wiring whereas stepping up to a 30A breaker requires 10ga wiring.
For the L3++ scenario we use a little math. We can put 4 units per 20A circuit (3600W) or 6 units per 30A circuit (5400W). To maximize the number of units, and keep it simple, we can also go with 5 – 20A breakers, 4 units for each circuit. This keeps us under the 80% rule for the full panel load (we are only using 18,000W which is only 75%.) Once again we will use 12ga wiring for this.
This was a quick down and dirty calculation and please know that there is more in depth planning that would go into an actual circuit design (like run lengths, ventilation, other equipment draw, main panel draw, etc.)
If you’re like me you’re always wondering how you can tweak or fix every little problem with your miner. I’m very rarely in front of my miners, or even on the same network, but I’m always checking to insure they are running smooth. Most of the time they are, but as the months wear on, seasons change, chips and boards strain, you may find yourself with equipment that doesn’t run like it used to.
If you’re able to get on the same network as your miners, then it’s an easy day. But what if you can’t, what if they’re remote or you’re out of town. We all know the world ends if we have even 1 hour of downtime on one of our miners. Well if that’s how you feel too, then keep reading because I may have an idea that works for you.
A quick caveat, this is specifically geared towards HiveOS firmware. If you’re not running this then you won’t be able to access your miner using this method.
Also a word of warning, be careful what you do here. If you’re not comfortable manually editing the files or using a unix/linux editor then this may not be for you. I use vi, which is a built in editor to the unix/linux OS. It takes a little getting used to if you’ve never played with it, as it’s command line only, but it does the trick once you know your way around.
How to remotely connect to your miner
The first thing you need to do is get in front of a computer and log into your HiveOS farm. I’ve done it from my phone but, at least for me, life is so much easier on a computer. Once logged in you need to click on the specific miner you wish to connect to and pull up their status screen. From there you’ll see a toolbar on the top right, the 4th item in (appears like >_) is the run command shell, click on that and you’ll see a pop-up similar to below.
You will see a pop-up that says, “Worker Commands” and it will list all your recent commands, which may differ from what you see above. From here simply type in, “hssh start” and you’ll start the process of creating an ssh session to your miner. It can take anywhere from a few seconds to a few minutes to create the ssh session so be patient. Once the ssh session is ready you’ll see the status update, “> Hive Shell” with a pop-up link next to it, as seen below.
Once the, “> Hive Shell” pop-up link appears click on the box with the arrow pointing out of it to start the shell. Depending on your network, it will either launch the shell directly or give you another screen with a link as seen below. If you get the screen below, click on the blue highlighted text under, “Web-Link:” to launch the shell.
Your shell should start up and show you the miner you’ve attached to, as well as the time/date and IP and MAC addresses. It should look similar to the screen below.
So now we are connected to our miner, what next? This is where I caution folks, be very careful what you do when you’re in here. I would limit where you go and what you view/edit. I would limit my adventures to just the config directory right off root. To get there simply type, “cd config” hit enter and you’ll be in the directory that contains your base configuration files for the miner settings.
Since we are editing configuration files you’ll actually need to restart your miner after you’re done to have the changes take effect. This is good if you accidentally change something and save it, never fear, you haven’t messed things up yet. Simply go back and undo the changes before restarting, or don’t restart at all.
CONFIG.CONF
In the config.conf file we have the ability to change several items that may help keep the mining running smooth. If you’re going to modify this I strongly suggest making a backup first, copying it to a new name that you can restore if things go south (i.e. config.conf.old.)
If we want to manually change the frequency of a chain (useful if you’re getting a significant number/thousands of HW errors per day) you can edit the bitmain-freq for the specific chain (i.e. bitmain-freq1 for chain 1.) These are whole numbers that correspond with the normal drop down menus. For reference, 384 would be for the stock L3+ and 450 is for the stock L3++.
The next modification you can do is manually set the chain voltage. I’ve found this is extremely useful for chains that get a fair share of HW errors (hundreds per day.) This is set by a number between 1-256 (makes it simple for the DAC), but it’s not as complicated as you think. If I have chain 1 that I’ve initially set to 9.50V, I can bump that up to 9.62V by changing bitmain-voltage1=225 to bitmain-voltage1=205.
Here are some common voltage values found in the file.
145 = 9.98V
175 = 9.80V
205 = 9.62V
225 = 9.50V
If you have several asics that are going bad (i.e. the dreaded “x”) you can also bump up the bitmain-x-restart-threshold to something greater than 3.
MANUAL_FREQS.TXT
In the manual_freqs.txt file we are able to set the frequency for each asic on each chain. This file is created by the HiveOS firmware when running autotune or setting individual chip frequencies from the advanced menu options on the GUI. I haven’t modified this file in the past as it’s difficult to narrow specific problems down without logging directly in on the same network and running the GUI to see how each asic is performing from the advanced menu.
OTHER .CONF FILES
As you can see there are other files that contain configuration information as well. I’m not touching those specific files in this article as those are easily maintained through the HiveOS farm menus and it’s more likely to cause issues if you attempt to modify them from the ssh shell.
CGMINER.CONF – Stores your mining pool information.
AUTOTUNE.CONF – Stores the most recent autotune configuration.
NETWORK.CONF – Stores network info, such as DCHP or static IP, and hostname.
WATCHDOG.CONF – Stores information on when a restart would be initiated.
EXIT
When you’re done making edits, or just browsing around, make sure to type in, “exit” in the shell and then close the window. This insures the ssh session closes properly.
MINER RESTART
Once you’ve made your changes and exited the ssh session make sure to go back to the “run command” button from your menu (the “>_” button.) This time simply type, “miner restart” into the Worker Commands box and your miner will restart and apply your changes. You can also simply select the power icon and restart from there as well.
I hope you found this useful, please feel free to hit me up with questions or comments.
How many times have you seen a post by someone that has a hash board that doesn’t recognize all the ASICs on board, has no temperature reading, and isn’t hashing or the hash rate starts then rolls down to zero?
Well, I’ve seen it a lot, and now just experienced the fun. I had an L3+ hash board I bought on Ebay that was a “FOR PARTS ONLY” so I took that as a challenge. When I first plugged it in it actually worked properly, so much so that I installed it into one of my L3+ units thinking I just hit the gold mine. Well, as soon as I put it in it started having issues; all ASICs found but no hashing, 56 ASICs found the next time, then 0, and so on.
I followed a few simple steps to work through the problem and viola, within 15 minutes I narrowed the problem down to one of the LDO’s that was failing. Using a heat gun I removed the part, replaced it with a similar part (LN1134), a little solder paste and a little heat and that bad boy mounted right down. I find with these smaller parts it’s easier to use solder paste, smear a little on to the pads, the heat gun will reflow it and pull the part on to the pads fairly well.
I put a quick video together to show the process of finding the failure, as always love the feedback and any experiences similar or not.
I’ve seen a lot of posts about crypto miners (all types and brands) where people are having issues such as no hashing, low hash rates, or errors connecting to pools. One thing you should be prepared for, if you ask the Internet what’s wrong, is that you’ll get no shortage of answers. We are in a community that’s more than willing to help, give an idea or two, but sometimes the good ideas and willingness to help aren’t always in line with what the data is telling us.
I’ll stick with some more specific problems I’ve seen over the last few months, and that’s dealing with the power supply for the miners. First and foremost, a 240V supply is superior to a 120V supply (for mining) as it allows us to fit more miners/larger miners per circuit. A standard 120V residential outlet can safely output 1440W, which is enough to carry just a few types of miners. About the largest 120V residential outlet you’ll find is 20A (30A is far less common but exists.) This is by design as wire size increases with amperage. So once you get above 20A you’re having to get much larger wire sizes (10 gauge or larger) which makes for a much harder (and more expensive) run of cable.
Now let’s take your standard electric over, water heater, or a couple S17 Pro miners. Each of these situations require around ~4500W. If we tried to run this on a 120V system (not saying they would, but stick with me here) we would need at least a 50A breaker and 6 gauge wiring. That’s because 50A gives us 4800W after taking the 80% rule into account.
Two problems here, first is if we have a 100A service panel in our home, that’s half your power (roughly speaking.) Second, I can run the same (actually more) power over just a 30A 240V circuit (5760W) and only need 10 gauge wire (commonly available in Romex style.) Now let’s see the cost savings as well, 10 gauge Romex runs about $1.50 a foot whereas the equivalent 6 gauge wiring runs about $3.50 a foot and is a bear to work with.
Conclusion on this, you can run twice the power over the same size wire by going 240V, essentially doubling the amount of miners you can use on that line.
BACK TO THE PROBLEM
OK, back to the problem at hand. Why is my miner not hashing and why are people telling me it’s my 120V power supply. That’s a great question, and probably not the right answer you’re getting from everyone.
A power supply is generally rated in the power (watts) it outputs. This varies slightly with input voltage in the case of power supplies that allow 100-240V, however the output rating is generally close whether it’s 120V or 240V unless otherwise specified. In the case of the APW3++, it’s significantly different. Its rated for 1600W @ 240V but only 1200W @ 120V, more than enough to run an L3+ @120V but not enough for most S9’s (most common problem I hear about.)
Now the question, will that affect my hash rate. Short answer is yes, but it’s more likely to overload your power supply and cause a thermal shut down, which will shut down the entire supply and your unit won’t work at all.
The long answer is that you probably won’t have a situation where your unit isn’t hashing at all. Start looking at the other posts and suggestions folks are giving. But make sure that whatever you are running is supported properly by the power supply you are using.
In the case of the APW3++, it can pump out 133A @ 12V (1596W) when using a 240V input, however with a 120V input it only pumps out 100A @ 12V (1200W.) If you’re using an S9 (14TH) you need ~1370W to run, therefore if you run the supply at 120V it will “most likely” shutdown due to the lack of available power.
Power supplies are meant to pump out amperage at their voltage rating, up until they can’t pump out anymore. At that tipping point there is (or should be in any UL rated power supply) an overcurrent protection that shuts it down completely, it’s not a taper, it’s a hard shut down.
Like many of you I’ve had some frustrations with my L3+ units, mostly surrounding the hash boards and various failures. Bitmain previously released an L3+ Maintenance Guide in 2018 that did a great job summarizing some basic failures, how to identify, and then the Internet took over with the various fixes. I’ve reviewed Bitmain’s manual many times and have boiled down most of the pertinent info below, as well I’ve added my two cents. Make that more than two cents, I hopefully dive into some of their topics a little deeper and attempt to explain it in more detail and make it easier to understand. Their manual can be cryptic and it’s missing a lot of the middle pieces tying some things together so I attempted to fill in the blanks as well. Some of the graphics and schematics I pulled from their document so I’d like to acknowledge them here.
This is meant to be a living document so I’ll update it as I find new solutions to problems. Furthermore there are several areas where I couldn’t find enough information or wasn’t able to reverse engineer how something functions so I denoted those with an assumption note.
I’m all about sharing so I’m putting this all out there to help everyone out, but please, if you re-use some of this give me a shout-out. I’ve had things plagiarized in the past and while it’s not anything earth shattering, it’s also a work in progress so I don’t want someone putting info out there that’s not complete nor something they can explain as to how they came to a specific conclusion. It not only makes them look bad, but it can cause distrust in the data being put out there. Long story short, share it and shout outs.
A general note regarding testing voltages at the various test points. You must have the hash board power and I/O cable plugged in and a working L3 control board or test fixture. The PIC (which provides the signal to turn on the 10V) won’t initialize the 10V buck controller until it receives the programming information from the control board, therefore not only the 10V but the 14.2V and the individual power regulators won’t have power to function.
Another quick note, if you don’t have one, getting a hot air rework station will make some of these fixes easier. You can pick up a station complete with a hot air gun and soldering iron for about $50, I use the FEITA 8586 SMD Hot Air Rework Station from Amazon.
PIC (programmable interface controller)
U3 (U74 on V1.6) is a 16LF1704/5 8-bit microcontroller PIC (programmable interface controller.) This is what makes the hash board, well, hash. If this is not programmed properly the hash board may exhibit all sorts of strange errors, anything from missing chips to bad temperature readings, to no or partial hashing, or the board not being recognized at all.
The PIC takes the frequency and voltage information from the L3 Control Board (what you set them at through the software interface) and adjusts the voltage output and frequency for the BM1485 chain. The PIC communicates with the other ASICs and ICs through I2C (SDA, SCL, and GND) which basically is an IC to IC communication protocol.
It is possible to reprogram this from code from your other hash boards which I will cover in another post at a later date (it’s quite an in-depth process and involves using a Pickit3 programmer and a little luck.)
Common Failures related to U3 (PIC)
No 10V found (measured at both sides of C948 or between either pin of L2 and ground.)
The PIC may have dropped its code or you may have cold solder joints in the PIC. There are two methods to replace the code, blissz did a great job walking through folks on flashing a custom firmware that will update this. I haven’t tried this method myself although it seems to be successful and a little more straightforward. The other method is to attach a Pickit3 and manually pull the code from a working hash board in your system then dump it into your bad hash board.
Improper voltage (less than 10V) measured at C948.
Your buck converter may have failed or may have cold solder joints. First try reflowing the joints and if that doesn’t work, replace the part. Depending on the voltage it puts out you may still be able to run at a lower frequency and have it work. I have one board with a failing buck converter (it only puts out around 8V but it varies) and I’ve tested out various frequencies and actually have it working up to 300MHz. That’s more of an FYI and something my curiosity had me research. You should always conduct the repair before operating them in abnormal conditions, I’m just waiting for a replacement part at the moment.
Temperature Sensor
U87 (U88 on V1.6) is a T451/TMP451 temperature sensor. This is powered by the 1.8V voltage regulator in the first chain. You can get failures if the voltage regulator (U86 on V1.6 hash board) is damaged by a voltage spike/surge.
Common Failures related to U88 (Temperature Sensor)
“Failure to get temp data” error in L3+ Kernel log.
Check voltage levels between pin 1 (1.8V) and pin 5 (GND), improper voltage from the first LDO chain can lead to a failure of the temperature sensor working properly.
Your temperature sensor may have failed or may have cold solder joints. First try reflowing the joints and if that doesn’t work, replace the part.
BM1485 ASIC
There are a total of 72 BM1485 ASIC’s on-board the L3+ hash board. There is no data sheet readily available for this but what I have gleaned from info they’ve put out there aren’t too many failures that are easily tied to these (at least nothing that’s easy for a DIY fix.) These are powered by 12 individual 1.8V power domains.
(*Note: assumption) The clock signal comes from Y1, a 25 MHz crystal, and each board communicates with each other through the CI/CO signals for synchronization.
If one BM1485 in the chain goes bad, it may break the communication to BM1485’s down the line.
Without a data sheet there is quite a bit of mystery behind how this actually works, but a little reverse engineering and research has led me to assume some of the following (key word is assume):
(*Note: big time assumptions here)
The frequency on each BM1485 (as set in your “Miner Configurations” pages) is set through programming received on CI/CO (command input/output) and RI/RO (respond input/output.) A command string is sent to the BM1485 to set the PLL divide for the various frequency levels on the BM1485.
1.8V Power Domains for BM1485
The L3+ has a total of 72 ASIC chips spread across 12 voltage domains (with 6 BM1485’s each.) Each voltage domain is controlled by a 1.8V voltage regulator that has changed over various version of the hash board.
* Version 1.6 hash board – U75, U76, U77, U78, U79, U80, U81, U82, U83, U84, U85, and U86 are LN1134A182MR 1.8V voltage regulators (pkg SOT23-5L) for each of the 12 voltage domains (6 – BM1485 ASICs per domain.)
These are extremely susceptible to damage from a voltage spike out of the boost circuit. To measure the output of each one you need to measure between pin 2 and pin 5. You can measure the input voltage between pins 1 or 3 and pin 5.
Common Failures related to 1.8V Voltage Domains
ASICs showing as “x” or no ASICs found.
Check voltage levels at the following test points on each voltage domain (set your ground lead on the negative size of the capacitor closest to each set of the test points.) Make sure you have the boards plugged into power and I/O cables to the control board. Without this the hash board won’t initialize the voltage domains.
RST – 1.8V
BO – 0V
RI – 1.6V – 1.8V
CO – 1.6V – 1.8V
CLKO – 0.9V
If you encounter a voltage domain that does not output 1.8V on pin 5 of the voltage regulator then it most likely should be replaced. It’s common for these to go out, especially during voltage spikes. The boost circuit has been known to knock the later domains (ASICs 49-54, 55-60, 61-66, and 67-72) when they fail.
This output value is set by the PIC and is derived from the value that the user selects in the firmware (i.e. if you change the chain voltage from the “Miner Configuration -> Advanced Settings” page this is where that voltage changes.)
Improper voltage (less than 10V) measured at C948.
Your buck converter may have failed or may have cold solder joints. First try reflowing the joints and if that doesn’t work, replace the part. Depending on the voltage it puts out you may still be able to run at a lower frequency and have it work. I have one board with a failing buck converter (it only puts out around 8V but it varies) and I’ve tested out various frequencies and actually have it working up to 300MHz. That’s more of an FYI and something my curiosity had me research. You should always conduct the repair before operating them in abnormal conditions, I’m just waiting for a replacement part at the moment.
14.2V Boost Circuit
U111 is an RT8537 switching power supply. This is used to boost the 10V supply (coming through the inductor L2) to 14.2V. It’s generally considered the most finicky part of the hash board itself and seems to fail quite often. Early on Bitmain discovered that components were oxidizing and therefore leading to failure so they started covering the circuit in epoxy. Why you may ask? I have no idea why this oxidizes any more or less than other parts of the circuit, but they seemed convinced this would fix the problem. In the end of the day, nope, they still fail and we still add in external booster circuits.
Basically this circuit boosts 10V from the buck controller to 14.2V. L1 is used to store the energy for the boost and D1 isolates the 14.2V BOOST_OUT through reverse protection. When you install an external boost circuit you have to remove D1 to eliminate the possibility of interference from the old boost circuit.
Failure of the boost circuit can also cause damage to the 1.8V voltage domains.
Common Failures related to the 14.2V boost circuit
Improper voltage at D1 (less than 14V, generally much less.)
Your on-board boost circuit may have failed. Install an external boost circuit and remove D1 to isolate the on-board boost from the external boost.
Your 10V may also be off from the buck controller, this should be researched and solved before assuming the boost circuit is bad.
Q10 is a 1AMP SOT-23 package which I believe is a MMBT3904 NPN general purpose amplifier. More on this later once I figure out what’s going on here.
Y1 is a 25.000 MHz crystal that’s used as the clock signal for all ASICs. If this goes out, nothing works, you’ll need an oscilloscope to verify this.
Like many of you I’ve had the displeasure of my L3+ lose a hash board, frustrating, but sometimes easily fixable.
Often times, and I mean often, you’ll find that the 14.2V boost circuit has died which leads to your hash rate dropping to nothing for that board. I’ve seen many fixes (and attempts at fixes) and talked with a lot of folks about this. Some of the crazier things I’ve seen is removing C943 (still don’t know why they removed it) and running a jumper wire all the way across the board to D1. That gets you ~12V at the boost circuit (they did remove D1 to isolate from the old circuit) however that’s not enough to get the board to work to its full potential and it’s not a clean 12V so you can cause damage to your hash board down the line if you have power spikes.
NOTE – Per the manufacturer – Before applying power for the first time, turn the output voltage adjustment screw 20 full turns counter clockwise (CCW). Several people have reported the buck converter frying on the external module and this should insure it won’t burn out on power-on. Additionally insure you adjust the voltage using a multimeter before attaching the output to your hash board.
So I’ve had a hash board down for a few weeks and just now finally got a chance to take the unit down and see what we’ve got. Upon boot up the L3+ sometimes shows a missing hash board in HIVEOS (L3+_HashBoard_72_V1.6.1), as well as the stock firmware. Other times it comes up as anywhere from 2-72 chips (under ASIC status) found. I swapped cables and PS units and it still appears that way. Of note it also shows 0 for the PCB and chip temperatures, 0.00 for the MH/s (RT and avg), and strangely enough “Infinity” under W/MH.
December 1, 2021
I decided to start down the path of measuring various voltages and found the following after I inspected the 14V boost circuit (most common fault it seems:)
C1074 missing (no idea if it was ever there but the hash board did work fully at one point)
Voltage across C1072 – 11.98V
Voltage ground to L1 – 7V
Voltage across C1221 – 7.02V
So I started wondering if I’ve got a bad switching power supply in U111. I reached out to the Reddit world to see if anyone else experienced this on a hash board. I had a few folks reach out, but most the times it was to point me to the Bitmain L3+ repair guide (translated to EN.) This is a great general resource for L3+ troubleshooting, however it’s somewhat dated and really only deals with the V1.0 hash board which has different part numbering on the PCB and some completely different IC packages.
So time to get dirty again, I traced back to the DC/DC converter and found that there’s only a 7V output from there (which runs through L1), Q3/Q4 and across L2 only shows 6.94V so where do I go from here. I can’t find the V1.6.1 schematic anywhere online and can’t read the DC/DC part number to figure out what part it is. The PIC inputs are correct and it’s sending the keep alive every minute, so I think it’s somewhere between there and the boost circuit.
Update December 2, 2021
I think I have it narrowed down to before the boost circuit in the 10VDC rectifier. The big difference, other than the PCB numbering changing, is that the V1.6.1 uses a up9305W sync-rectified buck controller for the 10VDC and the older versions (1.5 and below) use the LM27402. Completely changes the circuit so the older schematics don’t help. Somehow either the up9305W is bad, or possibly a res or cap. I also verified the Schottky barrier rectifiers (MBR540MFS) on the output of the 10VDC circuit. Q1 and Q5 are tied between 12VDC and 10VDC (output) and Q3 and Q4 are tied between GND and 10VDC (output.) These then run through L2 before hitting the 14VDC boost circuit as well as other components.
Here’s a sample of what the circuit design should look like on the V1.6.1 version hash board, and the 10VDC should be generated by the internal Vref x (R1+R2)/R2.
(*From the data sheet) The output voltage can be programmed to any level between the reference voltage VREF up to the 90% of VIN supply. The lower limitation of output voltage is caused by the internal reference. The upper limitation of the output voltage is caused by the maximum available duty cycle (90% typical). This is to leave enough time for over current detection. Output voltage out of this range is not allowed. A voltage divider sets the output voltage (refer to theTypical Application Circuit on page 3 for detail). In real applications, choose R1 in 1kΩ ~ 10kΩ range and choose appropriate R2 according to the desired output voltage.
Update December 3, 2021
I decided to go ahead and install a new 14V boost circuit. This is an easy addition to the hash board, actually much simpler and cheaper than trying to replace the components on the board itself. It’s a relatively simple part to add, I used step-up boost converters I found on Amazon for just about $1 a piece.
When adding these in the key is to have a quality soldering iron as the temps needed to reflow the solder on the L2 lead is significant. Also insure you remove D1 so you isolate your new 14V boost circuit from the existing components and set Vout to 14V before soldering the wire to the D1 pad (you should solder to the D1 pad furthest away from L1.)
Now the big test, make sure you have both the power cables and comm cable to your control board attached as the processor is required to send the signal to the hash board to operate properly. Forget this step and you’ll just be chasing “ghosts in the computer.”
Did it work, well, yes now I measure 14V however that didn’t fix my hash board issue. Next step, well, I’m going back a step and will attempt to replace the sync-rectified buck controller to see if that’s causing the low voltage (7V) instead of the holy grail of 10V. It’s strange given all the faults on the board, but it’s a good next step given the fact that if the base core voltage is off, that can have a ripple effect across other components.
Update December 23, 2021
I put together a DIY test kit for the L3 and ran some diagnostics. It appears that my temperature sensor is most likely bad (flaky, which is bad.) So my next step is to replace that and see where that takes us!
In reviewing the schematic of the temperature sensor, we can see that the TMP451 uses I2C to communicate on the hash board as well as having its own built in temperature sensor that is attached externally (pin 2 (D+) and pin 3 (D-)).
Credit – Bitmain L3+ Maintenance Guide
TEMP_P and TEMP_N are the PNP or NPN transistor in the ASIC (BM1485) itself that read the temperature. What I’m not sure of is which specific ASIC is it getting the temperature from as they operate in. I believe they use a thermal substrate transistor in one of the nearby BM1485 ASIC’s and have it wired to pin 6 (TEMP_N) and pin 7 (TEMP_P.) According the limited schematics I’ve seen they only show one representative sample of the BM1485 wiring and those pins are unconnected. I also see there are pads across the board for numerous TMP451’s, probably an enhanced feature they never implemented?
Luckily the TMP451 is in a WSON package, which makes it a complete pain to replace unless you have the right equipment and a steady hand. I suggest low temp solder paste in a syringe and a heat gun. I’ll try and video the process and post it when I’m done.
Update January 1, 2022
Finally success! I racked my brain for hours and finally decided that the buck controller wasn’t putting out the proper voltage. I inspected the PIC and since I could get some hashing I knew it was programmed properly, so I went back to PCB (printed circuit board) manufacturing 101, when it doesn’t work, rule #1 – reflow those solder joints.
I reflowed the PIC and the buck controller and viola! She’s hashing up a storm once again.
Reflow those joints, these circuit boards weren’t manufactured in the best possible environment
To continue our series on Bitmain’s Antminer L3+, we enter the troubleshooting phase. Over the last few months I’ve posted about various ways to extend the life of your miner and still maximize the efficiency. Well, sometimes we push a little too hard and see early failures and other times these units just get old and tired and start breaking. The question is what to do when it’s not doing what you think it should. Confusing? Don’t worry, I’ll walk you through creating a quick and easy (and cheap) way to test your L3+ hash boards.
I’ll start with saying Bitmain’s Support has been very helpful in getting me the firmware needed for this. There’s always going to be a slight language and time barrier, but they really did hook me up for this so I wanted to give them a big shout out. I haven’t been through their Antminer Maintenance Training Academy, which is generally a prerequisite for some of the support they gave me, but nonetheless they’ve really gone out of there way to help.
Bitmain created a maintenance firmware load that allows for running diagnostics on a unit without re-flashing the firmware that’s already flashed onboard the L3+ control module. This is exceptional for two reasons, one is obvious, we don’t have to re-flash and redo all our settings again. But two, we can test our units in place, and no special test fixture is required. It does help however to have a separate control board and I/O board for this, but it’s not required.
What you’ll need…
L3+ Control Board (I believe any version after 2.0)
Obviously you’ll also need some hash boards and a power supply, but that’s the whole point of building this so I’m guessing you’ve got those. I ultimately went the route of setting up a separate test fixture as I had the luxury of having a spare L3+ control board and I/O board.
Setting up the Diagnostic Firmware
You’ll need to get an SDCard that’s at least 8GB, clean and formatted FAT32 with only one partition for the entire device, and download the firmware here. The firmware image must be burned properly to the SDCard as there are several partitions that get created within the image. If you simply copy the image to a FAT32 formatted card you’ll be out of luck. I used the open source balenaEtcher which is a great free tool. Once installed, all you need to do is unzip the image to your hard drive theb open balenaEtcher. Once opened, it’s very simple:
And that’s it folks, a few minutes later you’ll have an SDCard that’s ready to roll and you’re already halfway there.
Setting up the L3+ control board for USB to TTL
Your next step is to add the USB to TTL adapter to your L3+ control board. This requires attaching three wires (TXD, RXD, and GND) to your control board. I highly recommend soldering the wires directly instead of the “cram and hold” method as that can lead to shorts and intermittent connectivity which will affect your results.
I used three male to female jumper wires for this as the female socket plugged directly onto the USB to TTL adapter and the male end fits into the L3+ control board sockets making for simple soldering.
For reference it will be easier to install the wires on the primary side (side that has the headers and most the larger components) of the L3+ control board and solder on the secondary side. There’s much more room to solder and less opportunity to damage anything. You just have to remember that you’ll need to bend the wires when you sandwich the I/O board on so plan for that when soldering (don’t mind the extra wires in the bottom right, that’s for another project involving an LCD.)
Attach power, USB cable, and computer
Once you have the control board and I/O board connected, attach the power header, your hash board to be tested on Chain 0 (I do one board at a time), and your USB cable to your computer. Open the terminal software on your computer and setup the USB serial port for 115200 baud rate, 8 bits, no parity, 1 stop bit, and RTS/CTS. Now select to connect (and if available select auto-reconnect as well.)
Insert the SDCard before powering up your unit and insure it’s inserted all the way by listening for the click that it snapped in fully.
Also if you’re running this outside of the chassis then insure you have fans cooling the hash boards or else you’ll create a whole new set of problems.
Now power on the unit and you should immediately be greeted by the following:
Once the system goes through its self-diagnostics it will enter a login screen, wait until it automatically logs in then you will reach the end of the boot up sequence when you see the following:
— get_works: wc_add_step = 1
At this point the system is ready to start testing any hash boards attached. In order to start the test you simply press the IP_Sig button on the L3+ I/O board. This will initiate a check for attached hash boards first, which is determined by the following:
As you can see from above, a value of “1” means the hash board is installed/recognized. The next steps simply read the hash board’s PIC, set the voltage, and set the frequency (stock 384MHz.)
After checking the hash board temperature the next step is running tests on individual chips (all 72.) I believe it accomplishes this by simply reading and writing to the memory. I believe the key thing here is that GAP_ERR, WORK_CRC_ERR, and CMD_CRC_ERR should all be “0” for each line.
After each ASIC has been tested the results will be published at the end:
Congratulations, you’ve run your first set of diagnostics on your hash boards and it should have only cost about $20 in parts and hopefully not too much labor. You will have to scroll through the terminal output to fully understand and appreciate what the test firmware is doing and understand a little bit about memory and registers, hexadecimal stuff, blah blah blah.
While this doesn’t point to anything directly at times, it can start to narrow down what you’re running into. I had one hash board that I believed the 14V boost circuit was bad on, turns out I had a bad temp sensor IC that intermittently read incorrectly so it was tough to track down.
Give it a shot and as always please give me feedback or experiences so I can improve this for others.
I’ve had a lot of folks hit me up with various overclock settings, and while most of them are within an acceptable range, there’s a few that really had me wondering when they would start seeing catastrophic failures. I consider this to be at a level where you far exceed the rated specifications of not only the unit as a whole, but even the individual components on the unit.
I’ll start with a disclaimer, actually more of a request, please don’t go crazy with overclocking until you understand the underlying negative effects and outright dangers. Any device you use to mine, whether it’s an ASIC or a GPU, has basic operating ranges and specifications for a reason, they’re safe, reliable, and repeatable. Once you step outside those limits you are literally taking a risk.
Update 12/27/21: I’ve had many folks ask how to measure their power draw. One solution that works very well is to install a Sense Energy Monitoron either your main electrical panel or a sub-panel that you have dedicated to your miners. This will give you real time feedback on the power (watts) used by your devices and make it easier come tax time to properly divide up your electrical bill and have the proof of the percentage you dedicate to your mining operation.
We’ll stick with the L3+ for this assessment, however many of the same lessons apply across all miners. Overclocking does more than just increasing your hash rate, to understand everything that’s going on we have to dive into some basic electrical principles.
Frequency (MHz not MH/s)
Frequency is basically the speed at which you’re operating the ASIC (MHz). Speed it up and you’re able to complete more “tasks/hashes” in a given period (time/seconds.) That comes with a cost, and that cost is paid for mainly in power and heat. To operate at a higher frequency than stock will require more power (watts), and with more power comes more heat. Yes, there are some variables like lowering voltage but you get the gist. So when you speed up your miner you draw more current, which increases your power, and generate more heat. Pretty simple, I’ll just speed up the fans and get a bigger power supply…
I’m pretty sure that’s what they thought too…
OHM’S LAW
We can go to an electrical staple to explain all of this, Ohm’s law: current (I) = voltage (V) / resistance (R). As you can see, there’s a direct relationship between these and power (W) as well. The Ohm’s law formula wheel is the best representation of the relationship between them all.
How this applies to mining is mainly covered the green (P) section. As you can see, power (P/watts) = voltage (V) x current (I). So when you crank up the frequency on your hash boards you’ll see the wattage go up. Ohm’s law tells us if you’re not manually adjusting the voltage, then you must be increasing the current.
Now let’s run some numbers based off some data I’ve collected in a past experiment. These are all straight frequency settings, at 9.92V, with no individual chip tuning.
Starting with the base L3+, running around 384MHz, you get around 203W per hash board. So 203W = 12V x (I)A, or 16.9A.
@448MHz we have 230W = 12V x (I)A, or 19.2A.
@472MHz we have 240W = 12V x (I)A, or 20A.
@490MHz we have 260W = 12V x (I)A, or 21.7A! (I’ll explain the exclamation below)
Note: There will be slight variations depending on the exact voltage settings you can use in certain firmware. For example, if you run the stock 384MHz but under-volt the boards (stock is 10.11V), you can safely run around 9.4V and you’ll consume less power.
So What, I have an 1800W power supply…
This is true, most power supplies have far more power than the unit requires. But just because you have it, doesn’t mean you should use it. Those little 6 pin connectors on the hash boards (2 per L3+ hash boards) carry a current rating anywhere from 8A to 10A, depending on the manufacturer and wire size. As you can see from the examples above, anything 240W and over you are most likely out of the operating range and that equals some excess heat, which leads to browning of the connector, and the higher/longer you go leads to deformation and failure (likely as a fire.)
Now let’s bring it all together. If we backwards plan our power consumption we see that we can’t support anything over 240W per hash board safely. There’s a key number to keep in mind when you’re tuning your miners. Why, for all the reasons above, we stay within specification of the main artery that runs the hash board. Burst that and you have catastrophic failure. So have at it, pump up that frequency, but when you do, keep the wattage under close inspection. You’ll have to drop the voltages down to keep it safe.
To continue the journey into setting up your crypto miners, specifically the L3+, you should start considering a long term electrical plan. What I mean by this is how can you optimize your existing electrical circuits in your home, office, shed, or wherever to gain the most MH/s(AVG) per watt (W/MH) and overall the most MH/s(AVG) per circuit.
Update 12/27/21: I’ve had many folks ask how to measure their power draw. One solution that works very well is to install a Sense Energy Monitor on either your main electrical panel or a sub-panel that you have dedicated to your miners. This will give you real time feedback on the power (watts) used by your devices and make it easier come tax time to properly divide up your electrical bill and have the proof of the percentage you dedicate to your mining operation.
I’ll assume that all units will operate off 240V for this, as it’s generally considered the most efficient as you pass less current through the wiring than you would if you went the 120V route, which minimizes cost and power transmission loss.
After some testing of various operating power/efficiency levels (Overclocking on the L3+, is the juice worth the squeeze?) of the L3+, I’ve got a good data set that gives me an efficient range to operate the L3+ within. So what now, data, great, how do I put it to use?
I ran the tests at frequencies from 384MHz (stock) to 500MHz and each frequency I ran at 9.5VDC, 9.8VDC, and 9.92VDC. The most ideal setting (with the best W/MH) for overall hashing rate was 469MHz, giving us ~608MH/s(AVG) @ 1.54W/MH (935W total.) The most ideal setting for overall efficiency (W/MH) was 384MHz, giving us ~504MH/s(AVG) @ 1.4W/MH (695W total.) A midrange that balances the two was 450MHz, giving us ~576MH/s(AVG) @ 1.52W/MH (873W total.) We also have to add in the wattage for the control board and fans. I took some measurements with an ammeter and found that the control board was only drawing about 10W and the fans, albeit variable, will generally draw no more than their max rating which would be ~30W each.
Note: These are all numbers that have not had any type of auto-tuning done at the individual chip level so your actual numbers can vary depending on that process if you chose to do it. These are just baseline numbers to go off of.
For those that aren’t familiar with residential or commercial wiring, a quick note on how much to load the circuits. The National Electric Code (NEC) essentially requires that each circuit have the ability to carry 125% of the continuous load. So if we have a 20A circuit, that is our theoretical 125%, which puts the continuous load at 16A (16A x 125% = 20A.) Head math shows that 16A is 80% of 20A, hence the 80% rule. After we determine the size of the circuit (i.e. 20A) we then reference the NEC code to find the appropriate wiring gauge for the circuit. This is code for one very good reason, you don’t want to overload and heat a smaller gauge wire too much or you’ll burn it up, and burn down your structure. I’m sure many folks have seen this on the DC side with wiring from power supplies to either ASIC miners or GPUs. I’ve chosen to use 20A for most my setups, mainly due to cost of the wire (12 gauge wire is significantly cheaper than 10 gauge), but also the efficiency calculations you’ll see later on in this post.
So let’s get into the meat and potatoes of what this is all about. I’ve listed out the most common circuits you’ll find and created scenarios based off those.
20A/240V – Given the 80% rule we have 3840W available to support our L3+ units.
3840W / 1005W = 3.82, so basically we can only run 3 L3+ units with plenty of room to spare and we are getting 1,824MH/s(AVG) out of the 20A circuit.
As a side note, we can toss one more L3+ in there at the most efficient setting (see below for wattage calculation) and that puts us at a total of 3780W and 2,328MH/s(AVG).
3840W / 765W = 5.02, so now we’re up to 5 units and we’re getting 2,520MH/s(AVG) out of the 20A circuit.
BALANCED
L3+ @ 450MHz = 873W + 10W (control board) + 60W (fans) = 943W total.3840W / 943W = 4.07, so now we’re at 4 units and we’re getting 2,304MH/s(AVG) out of the 20A circuit.
30A/240V – Given the 80% rule we have 5760W available to support our L3+ units.
5760W / 1005W = 5.73, so basically we can only run 5 L3+ units with plenty of room to spare and we are getting 3,040MH/s(AVG) out of the 30A circuit.
As a side note, we can toss one more L3+ in there at the most efficient setting and that puts us just over the 30A circuit at 5790W. Promise me you’ll unplug one intake fan (-30W) and that would give us 3,544MH/s(AVG).
5760W / 943W = 6.11, so now we’re at 6 units and we’re getting 3,456MH/s(AVG) out of the 30A circuit.
50A/240V – Did you disconnect your AC or hot tub for these miners or something?
You probably are spending more money in wiring (code says you’ll need 6 gauge wiring) then you can make on this circuit in a week. With the wiring and conduit, you’ll spend close to $5 per foot. In other words, stick with 20A (12 gauge wire) or 30A (10 gauge wire), the wiring is available at your local Lowes or Home Depot and comes in Romex so it’s an easier install without needing conduit. That’s all I have on this.
In summary, efficiency is king. Running out units at 384MHz and 9.5V yields us more than an 8% gain in MH/s(AVG) in a 20A circuit and a 14% gain in MH/s(AVG) in a 30A circuit.
Individual results may vary, take it for what it’s worth, but if you have the units, keep them running efficiently and you’ll get the most bang for your buck!
Like many of you I’ve often wondered just how much can I get out of my L3+/++, at just the level before I completely smoke it. For my own education I did some digging to find out just how much damage can be done at various overclocked frequencies.
Disclaimer – Every L3+/++ is different, every hash board is different, every environment we run these in is different, so these are just some baseline levels to consider.
Let’s just stick with the base L3+ set to the standard factory firmware and base settings. That puts us at an operating frequency of 384MHz and 10.11VDC per hash board IC (I’m using V1.5 hash boards.) That gives us our baseline of 504 MH/s (126 MH/s per hash board.)
Bitmain specs out the baseline L3+ at 800W nominal operating power, so if we take out ~60W for fans and control board, that leaves us with 740W total (185W per hash board.) Although after some research I believe they don’t account for the fans or control board on their specifications so let’s just set that number back to 200W per hash board.
Quick head math gives us roughly 1.59 W/MH, which is pretty much dead on from Bitmain’s specification of 1.6 W/MH. I ran one of my L3+ units at this level over the course of a few hours and saw that Temp(Chip) of each hash board remained at a very reasonable level, as shown in the chart below, but what’s interesting to see if the varying level of W/MH and Temp(Chip) levels throughout the experiment.
A few notes regarding the testing, I ran each speed/voltage for between 10-15 minutes as it takes about that long for the temperature and MH/s rate to stabilize. Additionally the ambient room temperature was 72 degrees and the fans ran at variable speeds, mostly between 5000-5400 RPM. Also a quick learning point for me, using ohms law we can see that each hash board requires approximately 17-20A of 12VDC.
The next three graphs show the three test voltage settings (9.5V, 9.8V, 9.92V) and the correlation of frequency (x axis) to MH/s(AVG) and W/MH (y axis.)
As you can see from the graphs, the actual (MH/s(AVG) is actual normalized rate which basically means this is the closest number to the actual work you’re doing. MH/s(RT) is an instantaneous value that doesn’t take Nonce% into account and can give you a false sense of the actual work you are doing. What does that mean, well as you’ll see in the next series of graphs, the AVG rate and RT rates track until we start getting massive amounts of HW errors which directly impact our Nonce%. For those that don’t know what the Nonce% is, it’s the ratio of HW errors to Nonce (numbers used only once.) The goal is to minimize the Nonce%, as well as the DiffA%. Generally if you’re not overclocking and un-volting too bad you’ll see a low number of HW errors which will keep your Nonce% under 0.03% (I’ve heard this arbitrary number many times and I still can’t explain it.) There is some direct correlation of the Nonce to the DiffA (difficultly of last accepted share), so keeping the Nonce under 0.03% keeps your DiffA near 0.0002% or less.
Long story short, if you hear about some amazing overclock settings where someone is getting 650-700 MH/s from an L3+, that’s most likely their RT and not the AVG, or actual work. I ran up to a 500MHz setting and got nearly 650 MH/s(RT), but as you can see in the data (and data doesn’t lie), the actual AVG rate was closer to 575 MH/s.
OK, great, thanks for all the data. So what does that mean for me, what are the best settings? I, nor anyone on the Internet, cannot answer that for you for the straight up reason that every board and situation is different. There are general guidelines to follow which I hoped to echo through this data. There is a point of diminishing returns, and that happens way before reaching temperatures that may damage your boards (generally under 75-80C.) A few graphs below show the varying temperatures and the slight rise with respect to operating frequency.
There are many different versions of firmware that can autotune, I actually use HiveOS on most of mine, and they do a pretty good job of fine tuning however you have to pick the basic frequency to run at before they can tune your L3+ in. If you are in a similar environment that I described then the data points to a sweet spot around 450MHz at 9.92V. That’s my basic starting point.
I hope this helps, and as always, hit me up with any questions or comments.
Here’s what $16 at Walmart or Amazon will get you…
When I bought my first L3+ miner in early 2021 I was excited to plug it in, set it up, and watch the money pour into my account. I had planned for everything I needed to hit the ground running once it showed up. I had installed a 240V/20A breaker so I could run 3 off the circuit, got some insulated ducting to run the hot air outside, a network router and cables, and of course the 250V/15A rated NEMA cords.
Why a 240V/20A for 3 you may ask? Well, I’m an EE by trade (haven’t done that in a while) so I put my old knowledge to work and did the quick calculation. The rough numbers, 240V x 20A = 4800W. Taking into account electrical 80 rule and not completely trusting what the continuous loads on the L3+ are, I don’t want to load the circuit anymore than 70% (80% is the 80 rule, I’m just being more conservative since it’s running in my house.) So 4800W x 0.70 = 3360W. If we did follow the 80 rule, we are at 3840W, just about what we need to run 4 units.
When I installed my first unit my first thought was, “Holy crap this is loud, my wife is going to be pissed.” Well, let’s say not pissed, but she kindly encouraged me to find a solution to the noise or find a new hobby. That led to quite a bit of research (hence why I started this website in the first place) of ways to quiet the units down without compromising them on heat. I saw various oil based systems (too messy for home), specialized server cabinets (too big and expensive for me), walling in and insulating part of your house to damped noise (I’m not that dedicated yet), then I saw someone post about a small cooler. I wish I could find the original post I read, it really got me thinking. Most of the comments annihilated the person for doing this, “you’ll overheat”, “you’ll melt the cooler”, “you’ll burn out the power supply”, and “where will you put your beer after you cut that all apart?”
Being new to this I took it all in, pretty much all valid points. I decided to move forward with utilizing a cooler, in fact, I tried with two different sizes (a 28 quart and a 48 quart), thank you Amazon! The purpose behind the two sizes was to try one with the power supply outside and one with it inside.
48 Quart (with power supply inside)
Realizing these 4″ fans push a massive CFM for their size, and regularly run 4-6K RPM, I opted to make a much larger intake opening (6″) in the cooler and not restricting it to a 4″ duct. On the exhaust I did install a 4″ duct adapter (4″ ducting end cap from Coolerguys.com) and ran this directly outside. I put a 6″ insulated duct hose on the input that just ran to the cooler and a 4″ insulated ducted hose to the exhaust end cap directly attached to the L3+. This would insure all the hot air would blow directly outside and not into the cooler and plenty of cool air would make it in to cool the L3+ and the power supply. I did some quick measurements of the db levels before and after for reference:
10 ft from L3+ in open air – 75db
2 ft from L3+ in open air – 90db (Bitmain says 60db but I don’t believe that includes the power supply)
10 ft from L3+ in cooler – 61db
2 ft from L3+ in cooler – 71db
But the most important number is that I was under 40db with the door closed, whereas before it was over 60db. My wife can now walk by the room to a light white noise instead of a loud APU from a 747.
I also checked the temperature on each of the hash boards. I was running the system on Hive OS, auto-tuned at the 610 MH/s at ~890W, and before running in the cooler could maintain my hottest chip temp around 67. After putting it in the cooler I only jumped up to 69, essentially no change at all. I credit this to two things, first is the amount of cool air I allow in by using a 6″ opening which more than doubles the area of the exhaust. The second is the variability in the fan speeds. I did worry my fans might be running too hard after this however they only run a few hundred RPM faster and hover around 5000-5200 RPM.
Success!
28 Quart (with power supply inside)
I followed the same setup with respect to ducting however the power supply was placed externally due to room. Since I didn’t have to worry about overheating a power supply I opted to do another slight modification, I added in 1″ acoustic sound-proof foam to see if I could knock the noise down even more. What I quickly realized was that this foam only helped raise the chip temp to the mid-70’s. I took my db readings and realized that our noise problem wasn’t just the L3+, it’s also the power supply. While not nearly as loud as the fans on the L3+, my db readings were almost exactly the same.
Not quite the same success…
Summary
I would say if you have the room and $20 extra that going with the 48 quart cooler option is nice. However if you don’t need it, no reason to do it. Here’s a quick video of my first iteration that includes the sound to get a real feel of how the cooler knocks down a significant amount of sound.
Update 7/1/21
If you’ve wondered why I did insulated ducting over traditional dryer duct, I did it for noise and not for any ambient heating purpose. However I have found that switching to 4″ flexible metal dryer ducting works just as well and is half the price. The temperature never gets high enough from the exhaust to warrant one over the other, nor does the noise, but in the end cost rules the day. 25′ of 4″ insulated ducting is about $40, whereas 12′ of 4″ dryer ducting is about $9.
Update 12/4/21
I’ve found, after buying a few different power supplies and experimenting with fan speeds, that I can run the units just as quiet by using a better power supply (APW7 or Gold/Platinum ATK) and setting the fan speed manually. I’ve also found that usually off-brand cheap power supplies from China are inherently loud and unreliable. Also stay away from server power supplies (like the HP or Liteon rack mount) as these are also louder than the L3+ themselves.
So after learning a thing or two about the stock Bitmain firmware I quickly realized that to continue running these in my home, and remain somewhat profitable given electricity costs, I had to get a better return on each kWH I consumed. I spent a fair amount of time searching the Internet, from one end to the other, and settled on Hive OS. Hive OS has a simple fee structure for ASICS (they essentially blindly use some of your L3+ hashing power to take a ~2% dev fee) and a great management interface.
Installation of Hive OS, a lesson in patience
OK, easy enough, let’s get this Hive OS loaded and see the money rolling in. I went to the website and created my account, downloaded the Hive OS firmware, even read their Hive OS install page. But wait, it’s never that easy, what’s this error, “Cannot Find Signature!!!” Well, roughly back in late 2018 Bitmain added a security feature to its firmware so that you could not load unauthorized (i.e. anything not Bitmain) onto the L3+. As with anything, a work around was quickly found.
I quickly referenced my handy dandy Hive OS install guide and shortly thereafter learned that just because you can figure out a problem, and come up with a solution, doesn’t mean you can properly write that solution down in a format that someone can understand and replicate. While I kept getting the, “Cannot Find Signature!!!” error, for the life of me I couldn’t understand how their remsig firmware was to be loaded and that would fix my entire life? I racked my brain, and the Internet, for a while before finally figuring it out. Here’s the process for anyone going down that route:
Go to the System – Upgrade tab of your L3+ web interface.
Under FLASH NEW FIRMWARE IMAGE choose your remsig file, click “Flash Image”
***IMPORTANT*** Once you flash this, generally within a few seconds this will finish however YOU MUST only click the “back” arrow on your web browser to put you right back at the same page when you first flashed the remsig firmware.
Now under FLASH NEW FIRMWARE IMAGE choose the latest Hive OS firmware, click “Flash Image”
This process should take a few minutes and you should be up and running on Hive OS. You may have to flush your browser cache if the original Bitmain web interface shows up.
Go to the System – Hive OS tab of your L3+ web interface.
Enter your FARM_HASH to properly attach your L3+ to your Hive OS account.
Enjoy!
They also offer the option of copying the Hive OS firmware to an SD Card and flashing the L3+ that way, however I’ve never had much luck with this so I go with what I know.
Update December 4, 2021
Make sure to keep your L3+ up to date with the firmware revisions as well. As of 12/4/2021 they were up to 1.04 (released 12/2/2021) Recently I had a problem with some L3+ units dropping off the Hive OS app, thereby removing one of my biggest reasons for choosing Hive OS, the remote management. It could be anywhere from 4 hours to 7 days, but they would all drop off at one point. I’ve been running the latest firmware for 48 hours now and so far all units have stayed on my farm list (Hive OS calls your list a farm) so hopefully they’ve worked out their API issues permanently.
If you’ve heard it once, you’ve heard it a million times, you’ve got to keep the air cool and clean running into these L3+ units. In scouring the Internet I’ve seen anything from a class 1000 clean room to running big dryer sheets on the intake. I don’t really have the real estate or funding to put an ISO 9000 clean room in, conversely I’m not willing to risk the safety of my family and home to a dryer sheet being sucked in over the intake. The later of the two solutions being the cheapest of course, however the most risky since any restriction on the air intake can rocket the ASIC temperatures and cause an electrical fire that can have devastating effects beyond just the miner itself.
I’m all for minimizing the maintenance and increasing the longevity of my L3+ units, however this is really more of a hobby and not a full time job so finding the proper balance can be tough. I’ve talked to several people that religiously take their miners down on a monthly PM schedule, take the hash boards out, blow compressed air through them to root out any dust or contaminants (which having been in the circuit board manufacturing world in the past I can honestly say is a terrible idea, more on that later), and do the same for each power supply unit. What does that gain them, for starters they will collect less dust than someone that does that semi-annually or more, but how much dust really collects on the hash boards depends on your location.
Not my idea of inherently safe
I’ll be honest and upfront, I have run most my units for close to a year and other than an initial cleaning, I haven’t touched most of them since. That being said I’m not advocating that for everyone, in fact I was concerned with dust from where they are running as it’s an older warehouse, but they continue to crank along. I pulled one recently for a power supply cable issue (that’s another bloguverse entry) and went to inspect the hash boards and found very little dust, and no other contaminants to speak of. The only thing I can imagine is that the CFM of air moving across the hash boards, coupled with the fact that I cleaned them when I first got them, created a smoother surface where dust doesn’t settle. I have another unit I bought used from someone that obviously did oil immersion, that unit I could never quite get the hash boards completely clean and it picks up a tremendous amount of dust, so much that I do filter air for that one unit.
I learned early on, mainly through research but also through some practical testing on one unit, never restrict the intake airflow. How does one filter the air then, two thoughts come to mind, forced air or simply a larger intake that funnels down to the 120mm fan. I didn’t feel the need to invest in a forced air fan so I went with a system that uses a standard sized furnace filter and ducted that through two 4″ inputs that merge into one 4″ duct right at the 120mm fan. I used a 4″ duct shroud for the 120mm fan from Coolerguys.com and tied directly to my filter ducting.
After watching the L3+ status screen off and on for a few hours I saw no noticeable change in chip temps or fan speed. Is it a winner, sure, but when I put that side by side to all the other L3+ units I have I would say, depending on your situation, don’t get wrapped up in the idea you have to filter your incoming air. Definitely check your temps and fan speed daily, but absent any abnormalities, keep it cranking.
