|
|
[<< Home](/home#5-digital-back-end-design-panel-charge-4)
|
|
|
|
|
|
[<< Section 5.3](./5.3)
|
|
|
|
|
|
## 5.4 Distributed File Storage
|
|
|
For storing output data products ALPACA plans to purchase and provide storage
|
|
|
hardware to augment the GBO site distributed storage cluster. We understand that
|
|
|
about a year ago GBO transitioned from using Lustre to BeeGFS as the site
|
|
|
parallel file system, and requiring networking to be done over InfiniBand. We
|
|
|
anticipate and are expecting GBO to integrate, configure, and manage ALPACA
|
|
|
provided storage hardware within their support framework, but the ALPACA grant
|
|
|
will purchase all hardware components.
|
|
|
|
|
|
The ALPACA team proposes to purchase three 4U Tyan Thunder SX FT70-B7100 Intel
|
|
|
storage servers. These servers feature a dual-socket topology using Intel 2nd
|
|
|
Gen Xeon Processors and will have 64 GB of RAM. Each server has 24 drive bays
|
|
|
that will be populated each with 2TB SSD adding a total of 144TB of storage
|
|
|
capacity to GBO. The servers will support using Intel's RSTe software RAID
|
|
|
driver for improved storage performance.
|
|
|
|
|
|
For InfiniBand networking, a Mellanox (NVIDIA Networking) SB7890 30-port 100 Gbps
|
|
|
inband managed IB switch has been purchased. Each server will additionally be
|
|
|
populated with a single dual-port InfiniBand EDR dual-port QSFP28 NIC with the
|
|
|
required cabling.
|
|
|
|
|
|
Note that ALPACA will not provision the rack space or a dedicated UPS for this
|
|
|
provided storage hardware.
|
|
|
|
|
|
The above storage server configuration specifications with more detailed
|
|
|
information is presented in the following table[^tyan-footnote]:
|
|
|
|
|
|
<div align="center">
|
|
|
|
|
|
| Feature| Description|
|
|
|
| :------ | :------ |
|
|
|
| Processors | 2 Intel Xeon Silver 4208 Processor, 8 Cores 16 Threads, 2.1 GHz, 11 MB L3 Cache, 85 W
|
|
|
| Memory | 64 GB DDR4-3200MHz EEC |
|
|
|
| Drive Bays | 24 Hot-swap HDD/SSD SATA 6.0 Gbps|
|
|
|
| Hard Drives | 2 TB Samsung QVO Series 560 SATA SATA SSD (48T total)|
|
|
|
| RAID | Intel RSTe software driver |
|
|
|
| PCIe | Gen 3.0, three x16 slots, three x8 slots |
|
|
|
| NIC | 1 Mellanox MCX555A-ECAT ConnectX-5 dual-port InfiniBand EDR |
|
|
|
| LAN Network | dual port 1 GbE LAN ports (1 shared with IPMI) |
|
|
|
| Power Supply | 800 W, 80 plus platinum, 1+1 redundant, 120VAC/10A, 200-240VAC/5A |
|
|
|
| Height| 4U |
|
|
|
| Rack Mountable | yes |
|
|
|
|
|
|
</div>
|
|
|
|
|
|
The SB7890 InfiniBand switch is pictured below
|
|
|
|
|
|
<div align="center">
|
|
|
<img src="../img/dbe/sb7890-switch.png" width=700"\>
|
|
|
</div>
|
|
|
|
|
|
[Section 6.1 >>](/6-fw-sw-design/6.1)
|
|
|
|
|
|
### Footnotes
|
|
|
[^tyan-footnote]: see [Tyan vendor website][tyan-thunder-sx] for full server
|
|
|
specifications
|
|
|
|
|
|
[tyan-thunder-sx]: https://www.tyan.com/Barebones_FT70B7100_B7100F70V26HR
|
|
|
[mlnx-sb7890-pb]:
|
|
|
https://www.mellanox.com/files/related-docs/prod_ib_switch_systems/pb_sb7890.pdf
|
|
|
[mlnx-sb-78-series]: https://www.nvidia.com/en-us/networking/infiniband/sb7800/
|
|
|
[beegfs]: https://www.beegfs.io/c/
|
|
|
[beegfs-docs]: https://doc.beegfs.io/latest/index.html |
|
|
\ No newline at end of file |