Hacking Supermico 12V Power Supply

I am working on a deep learning server that will have 8 Nvidia 4080 GPUs. The server will draw quite a bit of power and I want redundancy because our power will be 415/240 VAC three-phase system it made sense to use three 2000 W paralleling power supplies. This will give me about 4000 watts / 330 Amps of 12V DC and with 3 the server will still be up and happy with 1 dead supply. In the past, I have had good success with Supermicro so I started looking at PWS-2K04A-1R supplies. This is a nice auto-ranging supply that delivers 2000W of 12V DC power on a 240 VAC input.

The supply is low cost and has a 96% efficiency and supports PMBus 1.2 for remote management. I struggled to find the 54-pin card edge connector. After looking at an old server I found that Supermicro was using an ALL TOP connector, but I had trouble sourcing them. After digging around a bit on google I found that Compuware makes most if not all Supermicro power supplies and that this supply was Supermicro branded Compuware CPR-2021-3M6. With that information, I was able to get the datasheet and discovered the recommended connector, Amphenol GPCP442628110HR. The cool part was that Mouser had them in stock!

1+12V28DC Return
2+12V29DC Return
3+12V30DC Return
4+12V31DC Return
5I Share32RS GND
6A133Spare
7A23412V RS GND
8I2C SCL35Wakeup Bus
9I2C SDA36DC Return
10PS On/Off375Vsb CO
11SMB Alert385Vsb CO
12DC Good39DC Return
13+12V40DC Return
14+12V41DC Return
15+12V42DC Return
16+12V43DC Return
17+12V44DC Return
18+12V45DC Return
19+12V46DC Return
20+12V47DC Return
21+12V48DC Return
22+12V49DC Return
23+12V50DC Return
24+12V51DC Return
25+12V52DC Return
26+12V53DC Return
27+12V54DC Return

I was much further along on my quest, the operation of these supplies is fairly straightforward. To turn on the supply you bring pin 12 PS On/Off to ground. Making sure three supplies can talk to each other and share the load is also straightforward. All you need to do is connect together the three supply pins.

pin 1 – 4 —- +12V
pin 5 —- I share
pin 8 —- I2C SCL
pin 9 —- I2C SDA
pin 11 —- SMB Alert
pin 13 – 27 —- +12V
pin 28 – 31 —- DC Return
pin 35 —- Wake up Bus
pin 39 – 54 —- DC Return

I wanted to use the I2C PMBus, but you could leave that off if you don’t care about PMBus.

With my newfound information, I started designing a PCB for this. I used an 8-pin ribbon cable to connect the 3 boards together, an SMBus connector to connect back to my Supermicro motherboard and a red LED tied to ping 12 DC Good so I could see if the supply was happy.

Since I wanted to use PMBus I had to give each supply a unique address, I could not find any information on how to set this so I connected the SMBus to a Raspberry Pi I2C port and ran i2cdetect as I changed the pins.

AddressA1A2
0x38LowLow
0x39LowHigh
0x3AHighLow
0x3BHighHigh

The pins were pulled high by default, so I just added 2 jumpers to pull them to ground when I applied a jumper.

I designed the PCB in KiCad and sent them off to JLCPCB to get fabricated. I had 10 boards made for $22 including the $17 upcharge for 2oz copper. Now before I get all sorts of flame mail, 2000 Watts of power is more than 160 Amps! I know that my little 2oz copper board is NOT going to handle that much current, but it was a low-cost way to test. Once the design is finalized I can have them made with 10oz copper.

I decided to use ½ copper threaded rod and nuts to carry current between the 3 boards and dual 2/0 welding cable to carry the load to my bus bars.

I connected the SMBus to a Supermicro H12SSL motherboard and went into IPMI and I was able to see stats on all 3 power supplies. I then connected my load bank to the unit and was able to pull 500 amps at 12V (the max of my load bank) with no problem.

Overall I think it worked out ok, one task down, and many more to go.

Leave a Reply

Your email address will not be published.