What is Enzian?

Enzian is a powerful computer for doing systems software research. You can use a single Enzian on its own, or connect a rack of them together.

Enzian is a cache-coherent 2-node asymmetric NUMA system where one node is a 48-core CPU and one node is an FPGA. It has a maximum of 640GiB of DDR4 RAM, and has up 480Gb/s of network bandwidth, both split between the two nodes. The board looks like this:

Current Enzian specifications

Node 0:

• Marvell Cavium ThunderX-1 CN8890-NT CPU @ 2 GHz (48 x ARMv8.1 cores)
• 128 GiB DDR4, 4x 32 GiB DIMMS @ 2133 MT/s
• PCIe Gen3 x8 slot
• 3 x NVMe connectors
• 4 x SATA connectors
• 2 x 40Gb/s Ethernet QSFP28 connectors
• USB3, serial UARTs
• JTAG

Node 1:

• Xilinx CVU9P FPGA
• 512 GiB DDR4, 4x 128 GiB DIMMS @ 2133 MT/s or 64 GiB DDR4, 4x 16 GiB DIMMS @ 2400 MT/s
• PCIe Gen3 x16 slot
• 1 x NVMe connector
• FMC connector
• 16 x 25Gb/s serial lines in 4 x QSFP28 cages, configurable as 16 25Gb/s or 4 x 100Gb/s Ethernet
• JTAG

Coherent interconnect:

• 24 x 10Gb/s lanes
• MOESI-like directory-based protocol
• Inter-processor interrupts
• Cached and uncached (I/O) accesses
• 128-byte cache line size
• Shared physical address space

Baseboard Management Controller (BMC):

• Enclustra Mercury SoM
• Xilinx Zynq MPSoc CPU running Linux
• Dedicated 1Gb/s RJ45 Ethernet
• JTAG
• All system serial ports brought out as USB UARTs

Software:

• Ubuntu Linux 20.04 LTS, other OSes should boot
• ARM Trusted Firmware
• UEFI
• Board development kit for low-level access

Form factor:

• eATX-format motherboard (305 × 330 mm / 12 × 13 in)
• 2U Rackmount case
• Can accomodate double-width PCIe cards with riser.
• 1000W Power supply

Block diagram