UberDDR4: The Open-Source DDR4 Controller - Post #19
Back in 2023 I released UberDDR3, my open-source DDR3 controller. Three years later, the successor is now here: UberDDR4, the open-source DDR4 controller.
This blog post walks through the controller, the PHY, the prober, and the verification flow behind it.
And a spoiler for the end: the project I always wanted to do with UberDDR3 but never could, which is UberDDR4 + LiteX + VexRiscv booting Linux on an UltraScale+ FPGA board :)
Table of Contents
I. Introduction
Shown below is the UberDDR4 architecture:
Wishbone master = main user interface to read/write from/to the DDR4 memory. Data width is 8x of the memory. So, with a 1-byte lane of DDR4 memory you will have a 64-bit Wishbone data
Wishbone debug master = separate 32-bit Wishbone interface to access the CSR of UberDDR4: calibration status, read-eye health, BIST counters, which training step failed, and more
ddr4_top = the UberDDR4 wrapper you instantiate
arbiter = decides whether the Wishbone master or the prober owns the controller interface
ddr4_controller = handles all JEDEC DDR4 spec requirements for you: power-up initialization, refresh, request pipeline, per-bank and per-bank-group timers and trackers, packs four DDR4 commands per controller clock, and more
ddr4_prober = handles BIST (Built-In Self-Test) and CSR (Control and Status Registers)
ddr4_phy = PHY module connected to controller via DFI v3.1 interface. Currently, there are 2 PHYs available for UltraScale+ FPGAs:
Component mode PHY = low speed (<= DDR4-1250) but simple IODELAY/IOSERDES architecture
Native mode PHY = high speed (>DDR4-1250) but (very) complex bitslice architecture
The controller runs at 1:4 ratio, so if you want to run at DDR4-2400 (2400 MT/s), then the memory clock needed is 1.2 GHz and the controller clock is 300 MHz. Currently, UberDDR4 is tested up to DDR4-2400 at AXKU3 UltraScale+ FPGA board.
II. Controller
The UberDDR4 controller is big and I honestly don't know where to start, so let's start where the hardware starts: power-up initialization
Reset and wake-up (steps 0–2). Hold RESET_n and CKE low while the supplies ramp, release them in order, then wait tXPR.
Write to Mode registers (steps 3–16). MR3 → MR6 → MR5 → MR4 → MR2 → MR1 → MR0
ZQ calibration and DLL lock (steps 17–18). Trim the drivers against the external 240 Ω, then wait for the DLL to lock.
Precharge all banks (step 19). This is the last step before training and calibration can start.
The power-up initialization ROM freezes and the calibration FSM takes over (steps 20 to 29):
Read gate training: READ the MPR's known 0000_1111 pattern until the PHY finds where DQS actually arrives.
Read eye training: Same pattern but now centering the sampling clock in the data window.
Write leveling: Controller issues write strobes which the PHY uses to align DQS to CK at the DRAM.
One final refresh before handing over the bus for normal traffic, and the controller loops on REFRESH every tREFI (7.8 us).
Once power-up init sequence is done, runtime traffic can now start. Request enters via the Wishbone interface and goes through the following:
Wishbone address is decoded and split into row / bank group / bank / column, with the bank group in the low bits by default. This means sequential access interleaves across bank groups thus having only a wait time of tCCD_S instead of tCCD_L (longer wait time for same bank group access)
Stage 1 registers the request and looks ahead by issuing speculative PRECHARGE or ACTIVATE for the next bank if request is nearing end of row
Stage 2 registers the request and does the actual READ/WRITE request
The scheduler issues command based on stage 1 and stage 2 into one of four DFI slots: PRECHARGE on a row miss, ACTIVATE on a closed bank, READ/WRITE on a hit
Behind the 2-stage pipeline is the scheduler, which decides when to move or pause the pipeline. This contains all timers and bank tracker to meet all of the JEDEC required timing parameters for DDR4:
bank_status_q tracks which banks are open and bank_active_row_q tracks the active row. A row hit skips ACTIVATE
There are four saturating down-counters per bank: before_precharge, before_activate, before_read, before_write. These are reloaded based on the latest issued command
Per bank group timing parameters (ccd / wtr / rrd) take the _L limit within a group and _S across groups. This means there is longer timing penalty for same bank group access.
tFAW caps ACTIVATEs at four per rolling window. This means a fifth waits for the oldest to expire first.
Once the request gets past the 2-stage pipeline and scheduler, it enters another data path pipeline to write data or receive read data from the PHY via the DFI interface while satisfying the CL (read latency) and CWL (write latency) requirements. This also handles the latency of every read and write request until it get ACK-ed on the Wishbone master:
Write: data enter with the WR command and is shifted by CWL − TPHY_WRLAT ddr4 clock cycles to be piped to the PHY via the DFI interface. The PHY adds its own TPHY_WRLAT latency back, so DQ lands exactly CWL after the command.
Note: CWL is the latency from write command to first write data
Read: the read enable from the RD command is shifted by CL + 4 ddr4 clock cycles (4 for the PHY round trip). i_dfi_rddata_valid from the PHY asserts when read DQ is ready.
Note: CL is the latency from read command to first read data
The ack_pipe_q handles the Wishbone ACK:
Read ack is inserted at far end of the ack pipeline, because it needs to wait for the read data round trip.
Write ack is inserted behind the last read, so acks leave in Wishbone request order
III. PHY
As mentioned above, currently there are 2 PHYs available for UberDDR4 which are both for Xilinx UltraScale+ FPGA: component and native mode PHY.
Component mode PHY is built out of primitives every Xilinx user already knows: OSERDESE3 and ISERDESE3 for the 8:1 serialization, IDELAYE3 and ODELAYE3 for the per-bit delay taps, and an IDELAYCTRL referenced to a 300 MHz clock to keep those taps calibrated over voltage and temperature.
DQ: the only lane that goes both ways: two SERDES, two DELAY lines, and one shared buffer. The write delay tap comes from write leveling while the read delay tap from the eye sweep.
Note that the deserializer is clocked by ddr4_clk, not by the strobe coming back from the DRAM.
DQS: only used for write but never for read since DQ deserialization is clocked by ddr4 clock and not by the read DQS. Its delay line is swept during write leveling.
DM_n: write-only data mask with same write delay tap as DQ
CK/CK#: clock generated by the SERDES fed with constant 01010101. Has constant delay tap of tCK/4 thus its 90-degrees shifted from command line
Command bus: serializes the command line via SDR (single data rate) SERDES
For UltraScale+ FPGA, component mode is limited up to DDR4-1250 which requires 625MHz on global clock routing. That is the trade: the component PHY operation is very intuitive, easy to debug, BUT it cannot reach the speeds DDR4 is supposed to run at.
The native mode PHY is the one designed for the high-speed operation of DDR4. BUT, it is non-intuitive to design:
The PLLE4 per I/O region turns the 300 MHz controller clock into a 2.4 GHz CLKOUTPHY. The DDR clock runs on dedicated XPHY routing and never reaches a global clock buffer, which is the only reason this very high rate is possible at all.
Ten states for bring-up: PLL lock, DLY_RDY, EN_VTC, VTC_RDY, then TBYTE_EN and PHY_RDEN.
DQ, DQS and DM_n are assigned into two nibbles of six slots. DQS sits at upper slot 0 and clocks the lower nibble's receivers. Each slot is a full hardware datapath (delays, 8:1 and 1:8 SERDES, and receive FIFO) which BISC (Built-In Self Calibration block) keeps recalibrating during runtime.
What you get is a datapath that self-calibrates and runs faster than fabric can. What it costs is... two months of hopeless hardware debugging.
UG571 documents the XiPHY as a SelectIO resource, it documents the bring-up sequence very well. Then it gives you 64 registers, bit by bit, and says:
"Many of the settings are used for memory applications (MIG) and are provided for completeness. For information on how the Memory IP uses the RIU registers, see PG150."
Every register a DDR4 PHY actually needs is in the set it just declined to explain. And PG150 tells you what MIG's calibration stages do, not what to write. There is no third document.
Oh, and it gets worse: the behavior UG571 does not document is the same behavior the UNISIM model doesn't implement. Simulation passes, the board doesn't. So, every one of those unknowns has to be discovered on hardware.
2 months of nightmare ended.... with me finally giving up. As a last resort, I let Codex do its own iterative debugging on hardware (or perhaps let me call it "reverse-engineering how native PHY behaves") and... IT FIXED IT.
I will elaborate on that soon since that deserves its own post.
IV. Prober
The prober module is a Wishbone master that issues BIST (Built-In Self-Test) to the controller. When enabled, the BIST starts on its own as soon as training and calibration completes and before the normal user traffic can begin. It has 3 phases:
Phase 1 (Sequential): Write and read back the address range in order, stressing sustained sequential throughput. Each address is also written one byte lane at a time so the data-mask path is checked along the way.
Phase 2 (Stress-addressed): Every access opens a different row, stressing frequent PRECHARGE and ACTIVATE.
Phase 3 (Alternating): Write one word then read that exact same word straight back thus stressing the write-to-read turnaround.
The prober exposes a second Wishbone port for CSR (Control and Status Register). A small register file that is the window into everything the controller just did.
Sixteen registers, completely separate from the DRAM data path, so you can read them in whatever state the controller is in.
Almost all are read-only except CONTROL (0xC). This is for manually starting a BIST run, soft-resetting the controller, and to set auto-recovery at BIST failure.
What you can monitor and debug: live controller and PHY state, what calibration settled on per byte lane (delay taps, bitslip, eye width), the current phase of BIST, static identification, and more.
V. Verification Flow
Since the time of UberDDR3, it's been clear to me how miserable it is to debug memory controllers on hardware. So, the strategy is always to make hardware the last ever place a bug can survive.
The verification flows from Verilator lint that takes just seconds, up through elaboration under Icarus and Yosys, formal verification regression under SymbiYosys, and finally a full behavioral simulation regression against Micron's own DDR4 model via Vivado simulation.
This full verification flow can be done by simply running:
$ ./run_compile.sh --all
Shown below is the expected terminal output:
All lint + compile + formal verif regression (28 tasks) + behavioral sim regression (26 tests) passed.
VI. Bring-up on AXKU3 FPGA board
UberDDR4 currently has an example demo for the AXKU3 FPGA board which is an UltraScale+ FPGA board. The example demo simply runs UberDDR4 with BIST enabled running at DDR4-2400. Once calibration and BIST pass, the 3 LEDs lit up.
The hardware platform is ALINX AXKU3 with two Micron x16 DDR4 devices giving a 32-bit bus and 2 GB total. The demo uses BIST_MODE 2 which means all three BIST traffic phases run through the whole address map. The whole address map is 2^26 addresses, so a clean pass is exactly 201,326,592 matching reads and zero errors.
To generate bitstream for this board, simply run make inside the axku3/ example demo: $ cd ./example_demo/axku3
$ make
That's it :) Come back later, perhaps wash some dishes while you are at it.
Once the bitstream is generated, program the board with:
$ make program
As shown on the short demo below, the 3 LEDs of the board lit-up signifying successful calibration and BIST:
VII. Project demo: Run Linux with UberDDR4 + LiteX + VexRISCV
Then I remembered my one regret from UberDDR3, the thing I wanted to do way back but could not because I was far too inexperienced and that is... running Linux. How awesome would it be to have a Linux system running on top of UberDDR3?
AND NOW, I have UberDDR4 and... I'm still just as clueless about Linux 🙂.
BUT of course, the difference between this year 2026 and 2023, when UberDDR3 was first released, is... AI! If I can't do it, Codex will do it for me (hopefully).
And so I gambled my $100 pocket money on a Codex subscription. The plan: a LiteX SoC with a VexRiscv core, with UberDDR4 as the DDR4 controller underneath it, booting Linux to a shell over serial.
Oh boy, my August lunch budget paid off:
I'm hungry, but I'm now running Linux 😏
Generating the SoC design for this linux project is simple:
$ cd ./projects/axku3_linux
$ ./uberddr4.sh all
A snippet of the run is shown below:
CPU runs at 300 MHz, and 300 MHz x 8 = 2400 MT/s, so that's DDR4-2400. The main RAM bus is adapted to 32-bit from the 256-bit Wishbone bus of UberDDR4. 1 GB of the board's 2 GB is mapped as CPU-visible main RAM. The UberDDR4 prober CSR is also connected for debugging thus showing 64 bytes for uberddr4_debug.
VIII. Conclusion and Acknowledgements
Three years ago, the first UberDDR3 release could light up an LED. Now UberDDR4 boots Linux at DDR4-2400.
So yeah, that's the blog post. A DDR4 controller and PHY, checked from lint all the way to Micron's own model, then thrown at real hardware and a real workload so I can say it works without crossing my fingers. Took me two months of native PHY debugging and $100 of lunch money to get here, but hey it works :)
Again, repo is here: AngeloJacobo/UberDDR4: Opensource DDR4 Controller
Clone it, run it on your board. And open an issue when it breaks, because it will. DDR4 do be like that sometimes.
UberDDR4 is GPLv3, so it is free to use, modify and share, as long as your changes stay open too. Full details are in COPYING and NOTICE.
Lastly,
Thank you, NLnet 🙏
This project was funded through the NGI0 Commons Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG Communications Networks, Content and Technology under grant agreement No 101135429. Additional funding is made available by the Swiss State Secretariat for Education, Research and Innovation (SERI).
That's it for this blog post. More features coming for UberDDR4, so see you in the next one.
Comments