HPE Cray Supercomputing XD665 System: технические характеристики и документация
В архивеOverview
|
HPE Cray XD665 |
|
HPE is bringing the power of supercomputing to datacenters of any size with the HPE Cray XD665 System . HPE Cray XD665 System is a top-performing GPU-accelerated server, delivering mixed-HPC/AI workload solutions to rack-scale, in a rack and roll fashion. HPE Cray XD665 System is a 4U chassis system that contains a single 2x CPU node with 4x Nvidia H100 Tensor Core SXM5 GPUs. It offers a complete, scalable solution for AI & HPC customers everywhere, with flexibility of fabric, memory, storage and operating system. HPE Cray XD665 System provides maximum performance for advanced HPC Simulations, AI Training and Deep Learning. Built with Exascale-ready networking technologies, integrated storage, extensive software portfolio and management tools, HPE Cray XD665 Systems can enable customers to innovate and prepare for tomorrow's challenges. |
|
|
|
HPE Cray XD665 Server System Key Features
NVIDIA and NVLink are trademarks and/or registered trademarks of NVIDIA Corporation in the U.S. and other countries. All third-party marks are the property of their respective owners. AMD and EPYC are trademarks and/or registered trademarks of Advanced Micro Devices, Inc. in the U.S. and other countries. All third-party marks are property of their respective owners |
|
|
|
| |||
|
HPE Cray XD665 System Server Top View | |||
|
Item |
Description |
Item |
Description |
|
1. |
DIMMs for CPU 1 & 2 |
5. |
HHHL PCIe Board(s) |
|
2. |
PCIe Switch |
6. |
GPU Module |
|
3. |
CPU 1 & 2 |
7. |
12x System Fans |
|
4. |
Management I/O board |
|
|
|
| |||
|
| |||||
|
HPE Cray XD665 System Server Rear View | |||||
|
Item |
Description |
Item |
Description | ||
|
1. |
Primary(1-2) & Secondary(3-4) PCIe Slot |
7. |
USB 3.0 port | ||
|
2. |
Vented blank (air solution) |
8. |
1G NIC Port | ||
|
3. |
DLC port with Coolant Hose (DLC solution) |
9. |
BMC port | ||
|
4. |
VGA port |
10. |
12V PSU units (x2) | ||
|
5. |
OCP 3.0 Slot |
11. |
54V PSU units (x4) | ||
|
6. |
PCie Slot |
|
| ||
|
| |||||
|
| |||||
|
HPE Cray XD665 System Server Front View | |||||
|
Item |
|
|
| ||
|
1. |
System Fan (x12) |
5. |
UID button |
Off |
Off |
|
Blue |
ID | ||||
|
Blinking blue |
Remote management session in progress | ||||
|
2. |
Backup battery tray |
6. |
Health LED |
Blinking red |
System is on or off, critical (remains latched on power off) |
|
3. |
Hard drives (x8) |
7. |
Power button |
Off |
No AC |
|
Solid amber |
Standby mode | ||||
|
Blinking green |
User has requested system | ||||
|
Solid green |
Powered on | ||||
|
Blinking amber |
Chassis manager is booting | ||||
|
4. |
I/O Module and Pull tab |
|
| ||
|
| |||||
|
Rack Airflow Requirements HPE Cray XD665 System The increasing power of new high-performance processor technology requires increased cooling efficiency for rack-mounted servers. For maximum cooling, HPE racks are recommended to allow these racks to be fully loaded with servers using the latest processors. For detail information please see Cray XD665 System User Guide: https://support.hpe.com/connect/s/search?language=en_US#q=XD6500 Thermal Boundaries The HPE Cray XD XD665 system offers choices in Direct Liquid Cooling or Liquid-Assisted Air-Cooling. The ability to operate up to 35C will come with limitations on choice of CPU and Memory SKUs. The tables below outline the thermal operating ranges for the CPU and Memory SKUs | |||||
|
| |||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Notes: - If a third-party rack is used, observe the following additional requirements to ensure adequate airflow and to prevent damage to the equipment. - When selecting Direct Liquid Cooling (DLC), HPE racks are required for factory integration. - Always use blanking panels to fill all remaining empty front panel U-spaces in the rack. This arrangement ensures proper airflow. Using a rack without blanking panels will result in improper cooling that can lead to thermal damage. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Standard Features
|
Processors (Select two of the following) | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Notes: - Mixing of Processors is not allowed - All AMD® EPYC® processors can support up to 1.5TB of memory each on the HPE Cray XD665 System - Certain limitations may apply to select processors, please contact your HPE sales representative | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Memory (Select up to 24 of the following) | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Description |
SKU | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
HPE Cray XD 32GB (1x32GB) Dual Rank x8 DDR5-4800 Registered Standard Memory Kit |
P58356-H21 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
HPE Cray XD 64GB (1x64GB) Dual Rank x4 DDR5-4800 Registered Standard Memory Kit |
P58358-H21 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
HPE Cray XD 128GB (1x128GB) Quad Rank x4 DDR5-4800 Registered 3DS Standard Memory Kit |
P58360-H21 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
HPE Cray Supercomputing XD 64GB (1x64GB) Dual Rank x4 DDR5-4800 Registered Standard Memory Kit |
P68663-H21 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
HPE Cray Supercomputing XD 128GB (1x128GB) Quad Rank x4 DDR5-4800 Registered 3DS Standard Memory Kit |
P68664-H21 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Notes: - Minimum memory density per platform is 640 GB to insure adequate performance - Mixing of x4, x8, 3DS memory is not allowed - Mixing of Memory DIMMs is not allowed - Optimal performance requires 6 (1 DIMM / channel) or 12 (2 DIMMs / channel) DIMMs per processor | |
|
Cooling (select one of the following) | |
|
Description |
SKU |
|
HPE Cray Supercomputing XD665 FIO Radiator Cooling Kit |
P54096-B21 |
|
HPE CRAY XD6500 DLC FIO kit |
P59457-B21 |
|
Graphics Options (select one of the following) |
|
|
Description |
SKU |
|
NVIDIA H100 80GB GPU FIO Accelerator |
P63395-B21 |
|
Storage M.2 Controllers - All CTO Models (System: Std 0 // Max 1) (User Selection: Min 0 // Max 1)
| |
|
Description |
SKU |
|
HPE NS204i-t Gen10 Plus x2 Lanes NVMe PCIe 3 x8 Boot Controller |
P20292-B21 |
|
M.2 NVME Drives - All CTO Models (System: Std 0 // Max 2) (User Selection: Min 0 // Max 2) |
|
|
Samsung PM9A3 960GB PCIe Gen4 Mixed Use M.2 22110 Self-encrypting SSD for HPE Cray XD 2000 |
P48695-H21 |
|
Notes: M.2 Controller supports up to 2x M.2 drives |
|
|
| |
|
SFF RAID Controllers |
|
|
HPE Cray Supercomputing XD665 MR9560-16i 8-port 8GB Cache PCIe4 x8 Tri-Mode Controller |
P68050-B21 |
|
Notes: - If RAID is not required for SFF SSDs, no controllers are required - For RAID with over 4 SFF Drives, select 1x P68050-B21 + 1x CVPM05 Super-Cap P69144-B21 - For RAID with 4 or less SFF Drives, select 1x P68050-B21 - All drive cable kits are included in the top-level SKU and do not need to be ordered separately - When configuring XD665 with XD665 Broadcom MegaRAID 9560-16i (P68050-B21), the Firmware of the backplane cannot be updated.
| |
|
SFF Drives - All CTO Models (System: Std 0 // Max 8) (User Selection: Min 0 // Max 8) |
|
|
HPE Cray Supercomputing 960GB NVMe Read Intensive SFF U.3 7450 SSD |
P65375-H21 |
|
HPE Cray Supercomputing 1.92TB NVMe Read Intensive SFF U.3 7450 SSD |
P65376-H21 |
|
HPE Cray Supercomputing 3.84TB NVMe Read Intensive SFF U.3 7450 SSD |
P65377-H21 |
|
HPE Cray Supercomputing 7.68TB NVMe Read Intensive SFF U.3 7450 SSD |
P65378-H21 |
|
HPE Cray Supercomputing 15.36TB NVMe Read Intensive SFF U.3 7450 SSD |
P68661-H21 |
|
HPE Cray Supercomputing 800GB NVMe Mixed Use SFF U.3 7450 SSD |
P69368-H21 |
|
HPE Cray Supercomputing 1.6TB NVMe Mixed Use SFF U.3 7450 SSD |
P69369-H21 |
|
HPE Cray Supercomputing 3.2TB NVMe Mixed Use SFF U.3 7450 SSD |
P69370-H21 |
|
HPE Cray Supercomputing 6.4TB NVMe Mixed Use SFF U.3 7450 SSD |
P69371-H21 |
|
HPE Cray Supercomputing 12.8TB NVMe Mixed Use SFF U.3 7450 SSD |
P69372-H21 |
|
| |
|
RAID Settings |
|
|
HPE RAID 0 Drive 1 FIO Setting |
339777-B21 |
|
HPE RAID 1 Drive 1 FIO Setting |
339778-B21 |
|
HPE RAID 5 Drive 1 FIO Setting |
339779-B21 |
|
HPE Raid 5 w/SP Drive 1 FIO Setting |
339780-B21 |
|
HPE RAID FIO Advanced Data Guarding Option |
339781-B21 |
|
HPE Customer Defined RAID Setting Service |
389692-B21 |
|
Notes: - If SKU 389692-B21 is selected then the following RAID rules do not apply: o All integrated hard drives must match. o Minimum drive quantity requirement for each RAID level. o RAID level must be selected if Factory Installed OS is present and MegaRAID controller (embedded or otherwise) is the primary controller. - If SKU 389692-B21 is selected, then at least 1 hard drive must be on the order. - If SKU 389692-B21 is selected, then preinstalled OS must be on the order." o A Customer Intent Document must be supplied if SKU 389692-B21 is ordered. - This Customer Intent Document should include all details about the desired RAID custom configuration. (This includes drive part numbers and quantities, RAID levels desired, which drives should be applied to each RAID level, and if a preinstalled OS has been ordered - which RAID set it should be installed on). - Warning Message - HPE Recommends using below standard RAID Settings. o 339777-B21 HPE RAID 0 Drive 1 FIO Setting o 339778-B21 HPE RAID 1 Drive 1 FIO Setting o 339779-B21 HPE RAID 5 Drive 1 FIO Setting o 339780-B21 HPE RAID 5 w/SP Drive 1 FIO Setting o 339781-B21 HPE Raid Adv Data Guarding FIO" - General RAID rules: o Only a single set of RAID will be offered and will only be applied to all applicable drives installed in a server. o RAID requires selection of a controller and a specific number of matching drives. (i.e., same part number) o RAID must be selected if both Factory Installed OS and controller (embedded or otherwise) are present. o RAID 0 requires at least 1 drive. o RAID 1 requires at least 2 or even number of drives. o RAID 1 with Spare requires at least 3 drives o RAID 5 requires at least 3 drives. o RAID 5 with Spare requires at least 4 drives. o RAID 6 requires at least 4 drives. - RAID levels are not allowed if is not included in the order | |||||||||||||||
|
| |||||||||||||||
|
InfiniBand & Ethernet - All CTO Models (System: Std 0 // Max 5) (User Selection: Min 0 // Max 5) | |||||||||||||||
|
Description |
SKU | ||||||||||||||
|
HPE InfiniBand HDR/Ethernet 200Gb 1-port QSFP56 PCIe4 x16 MCX653105A-HDAT Adapter |
P23664-H21 | ||||||||||||||
|
HPE InfiniBand HDR100/Ethernet 100Gb 1-port QSFP56 PCIe4 x16 MCX653105A-ECAT Adapter |
P23665-H21 | ||||||||||||||
|
HPE InfiniBand HDR100/Ethernet 100Gb 2-port QSFP56 PCIe4 x16 MCX653106A-ECAT Adapter |
P23666-H21 | ||||||||||||||
|
HPE InfiniBand HDR/Ethernet 200Gb 1-port QSFP56 PCIe4 x16 OCP3 MCX653435A-HDAI Adapter |
P31323-H21 | ||||||||||||||
|
HPE InfiniBand HDR/Ethernet 200Gb 2-port QSFP56 PCIe4 x16 MCX653106A-HDAT Adapter |
P31324-H21 | ||||||||||||||
|
HPE InfiniBand HDR/Ethernet 200Gb 2-port QSFP56 PCIe4 x16 OCP3 MCX653436A-HDAI Adapter |
P31348-H21 | ||||||||||||||
|
Notes: - The MAX PCIe Card capacity limit is 5. It is "all inclusive" and must account for ALL PCIe cards from all categories and sub-categories (except Graphic Options category and sub-category). - If configured for a Cray or Slingshot Solution, select the Slingshot networking card. | |||||||||||||||
|
InfiniBand & Ethernet - All CTO Models (System: Std 0 // Max 5) (User Selection: Min 0 // Max 5) | |||||||||||||||
|
HPE InfiniBand NDR 1-port OSFP PCIe5 x16 MCX75310AAS-NEAT Adapter |
P45641-H21 | ||||||||||||||
|
HPE InfiniBand NDR200 1-port OSFP PCIe5 x16 MCX75310AAS-HEAT Adapter |
P45642-H21 | ||||||||||||||
|
Mellanox MCX623105AS-VDAT Ethernet 200Gb 1-port QSFP56 Adapter for HPE |
P10180-B21 | ||||||||||||||
|
Mellanox MCX623106AS-CDAT Ethernet 100Gb 2-port QSFP56 Adapter for HPE |
P25960-B21 | ||||||||||||||
|
Intel E810-CQDA2 Ethernet 100Gb 2-port QSFP28 Adapter for HPE |
P21112-B21 | ||||||||||||||
|
Mellanox MCX512F-ACHT Ethernet 10/25Gb 2-port SFP28 Adapter for HPE |
P13188-B21 | ||||||||||||||
|
HPE Slingshot SA210S Ethernet 200Gb 1-port PCIe NIC |
R4K46A | ||||||||||||||
|
Intel E810-XXVDA2 Ethernet 10/25Gb 2-port SFP28 OCP3 Adapter for HPE |
P10106-B21 | ||||||||||||||
|
Intel E810-CQDA2 Ethernet 100Gb 2-port QSFP28 OCP3 Adapter for HPE |
P22767-B21 | ||||||||||||||
|
Intel I210 (Embedded NIC) |
N/A | ||||||||||||||
|
| |||||||||||||||
|
Transceivers | |||||||||||||||
|
HPE 100Gb 1-port OP101 QSFP28 x16 PCIe Gen3 with Intel Omni-Path Architecture Adapter |
829335-B21 | ||||||||||||||
|
Drive Cage (1 cage with up to 8 carrier per chassis) |
P22894-002 | ||||||||||||||
|
ASSY, CARRIER,HDD,BASIC,SNAP IN,SFF,10P |
| ||||||||||||||
|
Rail Kit ( 1 per chassis ) |
| ||||||||||||||
|
HPE 4U Server Rail Kit |
878571-B21 | ||||||||||||||
|
Blanks |
| ||||||||||||||
|
ASSY, HDD BLANK,SFF,Gen8 |
652999-001 | ||||||||||||||
|
P/S ASSY, BLANK, A6500, Gen10P |
P26595-001 | ||||||||||||||
|
XD665 OCP blank, Vented |
P59863-001 | ||||||||||||||
|
XD665 DLC blank, Vented |
P59864-001 | ||||||||||||||
|
| |||||||||||||||
|
Power Supplies - All CTO Models Require 1 power supply kit from the menu below | |||||||||||||||
|
Description |
SKU | ||||||||||||||
|
HPE Apollo 6500 Gen10 Plus Modular Accelerator Titanium Hot Plug Power Supply Kit |
P49948-B21 | ||||||||||||||
|
HPE Apollo 6500 Gen10 Plus Modular Accelerator Platinum Hot Plug N Power Supply Kit |
P31662-B21 | ||||||||||||||
|
| |||||||||||||||
|
Interfaces | |||||||||||||||
| |||||||||||||||
|
| |||||||||||||||
|
Industry Standard Compliance
|
| ||||||||||||||
|
|
| ||||||||||||||
|
HPE Cray XD665 System UEFI Unified Extensible Firmware Interface (UEFI) is an industry standard that provides better manageability and more secured configuration than the legacy ROM while interacting with your server at boot time. The UEFI System utilities is embedded in the system ROM. Its features enable you to perform a wide range of configuration activities, including:
|
| ||||||||||||||
HPE Cray XD665 Systems with UEFI can provide:
|
| ||||||||||||||
|
|
| ||||||||||||||
|
UEFI UEFI provides a higher level of security by protecting against unauthorized operating systems and malware rootkit attacks, validating that only authenticated ROMs, pre-boot applications, and OS boot loaders that have been digitally signed are run. Configure and boot your servers securely with industry standard Unified Extensible Firmware Interface (UEFI). The HPE Cray XD665 System is a UEFI Class 3 solution and does not support the less secure CSM (Combability Support Module) BIOS. |
| ||||||||||||||
|
|
| ||||||||||||||
|
Software Portfolio for HPE Cray XD665 System |
| ||||||||||||||
Notes: For more information on Hewlett Packard Enterprise Certified and Supported Servers for OS and Virtualization Software and latest listing of software drivers available for your server. Open Source software is not factory-installed nor is it supported by HPE. https://www.hpe.com/us/en/servers/server-operating-systems.html |
| ||||||||||||||
|
|
| ||||||||||||||
|
Fabric software |
| ||||||||||||||
|
Description |
SKU |
| |||||||||||||
|
NVIDIA UFM Enterpise with Business Standard per HCA 1-year Subscription E-LTU |
S1D73AAE |
| |||||||||||||
|
NVIDIA UFM Enterpise with Business Standard per HCA 3-year Subscription E-LTU |
S1D74AAE |
| |||||||||||||
|
NVIDIA UFM Enterpise with Business Standard per HCA 5-year Subscription E-LTU |
S1D75AAE |
| |||||||||||||
|
|
| ||||||||||||||
|
Server Management HPE Cray XD665 System Baseboard Management Controller Embedded, in-depth server-level monitoring and management technology offering system management, service alerting, reporting and remote management including remote console and virtual media mount. Industry Standard Redfish The HPE Cray XD665 System supports industry standard DMTF Redfish that provides API conformance and offers simplified server management automation such as configuration and maintenance tasks based on modern industry standards. Learn more at: https://dmtf.org/standards/redfish. For clustered HPE Cray XD665 System deployments (for HPC or other emerging workloads such as AI), customers can use the following cluster management software solutions: |
| ||||||||||||||
|
HPE Performance Cluster Manager Fully integrated system management solution offering all the functionalities you need to manage your HPE Linux®-based high performance computing (HPC) clusters, all day every day. HPE Performance Cluster Manager aggregates system metrics. The software provides:
Alternatively, to manage heterogeneous clusters or for customers with additional requirements, HPE also offers: Bright Cluster Manager Automates process of building cluster in the datacenter and in the cloud. Ideal for heterogeneous clusters or customers with additional requirements. |
|
|
|
Software Development Tools (Programming languages, debuggers, libraries) HPE Cray Programming Environment is a fully integrated software development suite offering programmers comprehensive set of tools for developing, porting, debugging, and tuning of their applications so they can shorten application development time and accelerate their performance. CPE cannot be supported on Ubuntu OS. Notes: For more information on HPE Cray Programming Environment visit this page Additional 3rd party software developmental tools:
HPE Message Passing Interface (MPI) - an MPI development environment designed by HPE to enable optimization of high-performance computing (HPC) applications on HPE Cray XD665 Systems . The HPE Performance Software-MPI leverages a scalable MPI library and boosts performance of existing applications without requiring recompilation. |
|
|
|
Security
|
|
|
Service and Support
|
HPE Services - Service and Support Get the most from your HPE Products. Get the expertise you need at every step of your IT journey with HPE Services Services. We help you lower your risks and overall costs using automation and methodologies that have been tested and refined by HPE experts through thousands of deployments globally. HPE Services Advisory Services, focus on your business outcomes and goals, partnering with you to design your transformation and build a roadmap tuned to your unique challenges. Our Professional and Operational Services can be leveraged to speed up time-to-production, boost performance and accelerate your business. HPE Services specializes in flawless and on-time implementation, on-budget execution, and creative configurations that get the most out of software and hardware alike. |
|
|
|
Recommended Services HPE Tech Care Service. HPE Tech Care Service is the new operational service experience for HPE products. Tech Care goes beyond traditional support by providing access to product specific experts, an AI driven digital experience, and general technical guidance to not only reduce risk but constantly search for ways to do things better. HPE Tech Care Service has been reimagined from the ground up to support a customer-centric, AI driven, and digitally enabled customer experience to move your business forward. ForHPE Cray XD665 Systems, HPE Tech Care Service is available in two response levels. Essential which provides a 15-minute response time 24x7 for most enterprise level customers, and Critical which includes a 6-hour repair commitment where available and outage management response for severity 1 incidents. https://www.hpe.com/services/techcare HPE Complete Care Service HPE Complete Care Service is a modular, edge-to-cloud IT environment service that provides a holistic approach to optimizing your entire IT environment and achieving agreed upon IT outcomes and business goals through a personalized and customer-centric experience. All delivered by an assigned team of HPE Services Services experts. HPE Complete Care Service provides:
https://www.hpe.com/services/completecare |
|
|
|
HPE Server Hardware Installation Provides for the basic hardware installation of HPE branded servers, storage devices and networking options to assist you in bringing your new hardware into operation in a timely and professional manner. |
|
|
|
HPE Installation and Startup Service Provides for the installation of your HPE hardware according to product specifications including options. The HPE service delivery technician will connect the product to a LAN as appropriate and enable remote support to allow for automatic case creation for hardware failures. Installation and start up services also include the installation of one supported operating system type (Windows® or Linux). |
|
|
|
HPE Service Credits Offers flexible services and technical skills to meet your IT demands as your business evolves. With a menu of services, you can access additional resources and specialist skills to help you maintain peak performance in your IT. HPE Service Credits help you proactively respond to your dynamic IT and business needs. |
|
|
|
HPE Education Services Provides comprehensive training designed to expand the skills of your IT staff and keep them up to speed with the latest technologies. Consult your HPE Sales Representative or Authorized Channel Partner of choice for any additional questions and support options |
|
|
|
HPE Factory Express for Servers and storage HPE Factory Express offers configuration, customization, integration, and deployment services for HPE servers and storage products. Customers can choose how their factory solutions are built, tested, integrated, shipped, and deployed. Factory Express offers service packages for simple configuration, racking, installation, complex configuration, and design services as well as individual factory services, such as image loading, asset tagging, and custom packaging. HPE products supported through Factory Express include a wide array of servers and storage: HPE Cray EX, HPE Cray XD, HPE Integrity, HPE ProLiant, HPE Apollo, HPE ProLiant Server Blades, HPE BladeSystem, HPE 9000 servers, as well as the HPE MSAxxxx, HPE 3PAR suite, HPE XP, rackable tape libraries and configurable network switches. HPE support high-security system builds at HPE manufacturing facility in Chippewa Falls, Wisconsin. From our Chippewa Falls facility, HPE has built and deployed systems to the most security-conscious government agencies in the world. Consult your HPE Sales Representative or Authorized Channel Partner of choice for any additional questions and support options |
|
|
|
Warranty This product is covered by a global limited warranty and supported by Hewlett Packard Enterprise Services and a worldwide network of Hewlett Packard Enterprise Authorized Channel Partners (may vary by region). Hardware diagnostic support and repair is available for three years from date of purchase. Support for software and initial setup is available for 3 years from date of purchase. Enhancements to warranty services are available through HPE Services operational services or customized service agreements. Notes: Server Warranty includes 3 years of Parts, Labor and on-site. Warranty repairs may be accomplished using Customer Self Repair (CSR) parts. Mandatory CSR parts are designed for easy replacement. A travel and labor charge will result when customers decline to replace a Mandatory CSR part. |
|
|
|
Parts and Materials Hewlett Packard Enterprise will provide HPE-supported replacement parts and materials necessary to maintain the covered hardware product in operating condition, including parts and materials for available and recommended engineering improvements. Parts and components that have reached their maximum supported lifetime and/or the maximum usage limitations as set forth in the manufacturer's operating manual, product QuickSpecs, or the technical product data sheet will not be provided, repaired, or replaced as part of these services. The defective media retention service feature option applies only to Disk or eligible SSD/Flash Drives replaced by Hewlett Packard Enterprise due to malfunction. |
|
|
Configuration Information
|
This section lists some of the steps required to configure a Factory Integrated Model. To ensure only valid configurations are ordered, Hewlett Packard Enterprise recommends the use of a Hewlett Packard Enterprise approved configurator. Contact your local sales representative for information on configurable product offerings and requirements. Notes: FIO indicates that this option is only available as a factory installable option. A Mechanized Lift will be required to accompany the order due to the excessive size and weight of the HPE Cray XD665 Systems. If the customer already confirms that they already have an appropriate Lift in-house, one will not need to be ordered. | |
|
Step 1: Choose a Chassis | |
|
HPE Cray Supercomputing XD665 Configure-to-order Server Chassis |
P52377-B21 |
|
| |
|
Step 2: Select Cooling Method |
|
|
HPE Cray Supercomputing XD665 FIO Radiator Cooling Kit |
P54096-B21 |
|
HPE CRAY XD6500 DLC FIO kit |
P59457-B21 |
|
Step 3: Select M.2 Storage Options (max 2) |
|
|
HPE Cray Supercomputing 960GB NVMe Read Intensive SFF U.3 7450 SSD |
P65375-H21 |
|
HPE Cray Supercomputing 1.92TB NVMe Read Intensive SFF U.3 7450 SSD |
P65376-H21 |
|
HPE Cray Supercomputing 3.84TB NVMe Read Intensive SFF U.3 7450 SSD |
P65377-H21 |
|
HPE Cray Supercomputing 7.68TB NVMe Read Intensive SFF U.3 7450 SSD |
P65378-H21 |
|
HPE Cray Supercomputing 800GB NVMe Mixed Use SFF U.3 7450 SSD |
P69368-H21 |
|
HPE Cray Supercomputing 1.6TB NVMe Mixed Use SFF U.3 7450 SSD |
P69369-H21 |
|
HPE Cray Supercomputing 3.2TB NVMe Mixed Use SFF U.3 7450 SSD |
P69370-H21 |
|
HPE Cray Supercomputing 6.4TB NVMe Mixed Use SFF U.3 7450 SSD |
P69371-H21 |
|
HPE Cray Supercomputing 12.8TB NVMe Mixed Use SFF U.3 7450 SSD |
P69372-H21 |
|
Notes: - Maximum quantity of 2 M.2 SSD Drives - All selected M.2 drive quantity are of the same SKU | capacity |
|
|
| |
|
Step 4: Select a Processor (Dual Processor Configuration Only) | |
|
AMD EPYC 9224 2.25GHz 24-core 200W Processor Kit for HPE Cray Supercomputing |
P61457-B21 |
|
AMD EPYC 9654 2.4GHz 96-core 360W Processor Kit for HPE Cray Supercomputing |
P56262-B21 |
|
AMD EPYC 9174F 3.8GHz 16-core 320W Processor Kit for HPE Cray Supercomputing |
P56263-B21 |
|
AMD EPYC 9534 2.4GHz 64-core 280W Processor Kit for HPE Cray Supercomputing |
P56264-B21 |
|
AMD EPYC 9554 2.65GHz 64-core 320W Processor Kit for HPE Cray Supercomputing |
P56265-B21 |
|
AMD EPYC 9354 2.85GHz 32-core 260W Processor Kit for HPE Cray Supercomputing |
P56266-B21 |
|
AMD EPYC 9634 2.00GHz 84-core 290W Processor Kit for HPE Cray Supercomputing |
P56267-B21 |
|
AMD EPYC 9474F 3.4GHz 48-core 360W Processor Kit for HPE Cray Supercomputing |
P56268-B21 |
|
AMD EPYC 9254 2.55GHz 24-core 200W Processor Kit for HPE Cray Supercomputing |
P56269-B21 |
|
AMD EPYC 9454 2.35GHz 48-core 290W Processor Kit for HPE Cray Supercomputing |
P56270-B21 |
|
AMD EPYC 9374F 3.5GHz 32-core 320W Processor Kit for HPE Cray Supercomputing |
P56271-B21 |
|
AMD EPYC 9274F 3.6GHz 24-core 320W Processor Kit for HPE Cray Supercomputing |
P56272-B21 |
|
AMD EPYC 9334 2.5GHz 32-core 210W Processor Kit for HPE Cray Supercomputing |
P56273-B21 |
|
Notes: Certain limitations may apply to select processors.Please refer to the thermal table above or contact your HPE sales representative | |
|
| |
|
Step 5: Select from the following Memory options (minimum 1,028 GB) | |
|
HPE Cray XD 32GB (1x32GB) Dual Rank x8 DDR5-4800 Registered Standard Memory Kit |
P58356-H21 |
|
HPE Cray XD 64GB (1x64GB) Dual Rank x4 DDR5-4800 Registered Standard Memory Kit |
P58358-H21 |
|
HPE Cray XD 128GB (1x128GB) Quad Rank x4 DDR5-4800 Registered 3DS Standard Memory Kit |
P58360-H21 |
|
Notes: - Minimum System Memory of 1,024 GB Required - Certain limitations may apply to select Memory options.Please refer to the thermal table above or contact your HPE sales representative | |
|
| |
|
Step 6: Choose from the following SFF Storage and RAID options | |
|
Hot Plug SFF SSD |
|
|
HPE Cray Supercomputing 960GB NVMe Read Intensive SFF U.3 7450 SSD |
P65375-H21 |
|
HPE Cray Supercomputing 1.92TB NVMe Read Intensive SFF U.3 7450 SSD |
P65376-H21 |
|
HPE Cray Supercomputing 3.84TB NVMe Read Intensive SFF U.3 7450 SSD |
P65377-H21 |
|
HPE Cray Supercomputing 7.68TB NVMe Read Intensive SFF U.3 7450 SSD |
P65378-H21 |
|
HPE Cray Supercomputing 800GB NVMe Mixed Use SFF U.3 7450 SSD |
P69368-H21 |
|
HPE Cray Supercomputing 1.6TB NVMe Mixed Use SFF U.3 7450 SSD |
P69369-H21 |
|
HPE Cray Supercomputing 3.2TB NVMe Mixed Use SFF U.3 7450 SSD |
P69370-H21 |
|
HPE Cray Supercomputing 6.4TB NVMe Mixed Use SFF U.3 7450 SSD |
P69371-H21 |
|
HPE Cray Supercomputing 12.8TB NVMe Mixed Use SFF U.3 7450 SSD |
P69372-H21 |
|
SFF Hardware RAID Controller Notes:Cable Kits are automatically included with the Chassis top-level SKU Super-Cap option required for > 4 SFF SSDs requiring hardware RAID 2x Controllers required for > 4 SFF SSDs requiring hardware RAID 1x Controller required for 4 or fewer SSDs requiring hardware RAID |
|
|
HPE Cray Supercomputing XD665 MR9560-16i 8-port 8GB Cache PCIe4 x8 Tri-Mode Controller |
P68050-B21 |
|
| |
|
Step 7: Choose Factory Configuration Setting | |
|
Cray Compute Node Identifier (no max) |
|
|
HPE Cray Compute Node FIO Configuration |
R9H92A |
|
| |
|
Step 8: Choose Networking Card Configuration (Max 4) | |
|
Ethernet PCIe |
|
|
Broadcom BCM57416 Ethernet 10Gb 2-port BASE-T OCP3 Adapter for HPE |
P10097-B21 |
|
Mellanox MCX623105AS-VDAT Ethernet 200Gb 1-port QSFP56 Adapter for HPE |
P10180-B21 |
|
Mellanox MCX623106AS-CDAT Ethernet 100Gb 2-port QSFP56 Adapter for HPE |
P25960-B21 |
|
Intel E810-CQDA2 Ethernet 100Gb 2-port QSFP28 Adapter for HPE |
P21112-B21 |
|
Mellanox MCX512F-ACHT Ethernet 10/25Gb 2-port SFP28 Adapter for HPE |
P13188-B21 |
|
HPE Slingshot SA210S Ethernet 200Gb 1-port PCIe NIC |
R4K46A |
|
Intel E810-XXVDA2 Ethernet 10/25Gb 2-port SFP28 OCP3 Adapter for HPE |
P10106-B21 |
|
Intel E810-CQDA2 Ethernet 100Gb 2-port QSFP28 OCP3 Adapter for HPE |
P22767-B21 |
|
Intel E810-XXVDA4 Ethernet 10/25Gb 4-port SFP28 OCP3 Adapter for HPE |
P41614-B21 |
|
InfiniBand PCIe |
| |||
|
HPE InfiniBand HDR/Ethernet 200Gb 1-port QSFP56 PCIe4 x16 MCX653105A-HDAT Adapter |
P23664-H21 | |||
|
HPE InfiniBand HDR100/Ethernet 100Gb 1-port QSFP56 PCIe4 x16 MCX653105A-ECAT Adapter |
P23665-H21 | |||
|
HPE InfiniBand HDR100/Ethernet 100Gb 2-port QSFP56 PCIe4 x16 MCX653106A-ECAT Adapter |
P23666-H21 | |||
|
HPE InfiniBand HDR/Ethernet 200Gb 1-port QSFP56 PCIe4 x16 OCP3 MCX653435A-HDAI Adapter |
P31323-H21 | |||
|
HPE InfiniBand HDR/Ethernet 200Gb 2-port QSFP56 PCIe4 x16 MCX653106A-HDAT Adapter |
P31324-H21 | |||
|
HPE InfiniBand HDR/Ethernet 200Gb 2-port QSFP56 PCIe4 x16 OCP3 MCX653436A-HDAI Adapter |
P31348-H21 | |||
|
HPE InfiniBand NDR 1-port OSFP PCIe5 x16 MCX75310AAS-NEAT Adapter |
P45641-H21 | |||
|
HPE InfiniBand NDR200 1-port OSFP PCIe5 x16 MCX75310AAS-HEAT Adapter |
P45642-H21 | |||
|
HPE Data Processing Unit InfiniBand NDR/Ethernet 400Gb 1-port QSFP112 HHHL B3140H Adapter |
P66387-H21 | |||
|
HPE IB NDR/EN 400G 1p OSFP Adptr |
P45641-H23 | |||
|
HPE IB NDR200/EN 200G 1p OSFP Adptr |
P45642-H22 | |||
|
| ||||
|
Step 9: Select Power Cables (Max 6) |
| |||
|
A power supply kit consisting of 6x Titanium 3,000-Watt Supplies is already embedded within the chassis. Select Power Cords (required - max 6) | ||||
|
HPE SDG300 - C20 250V 16Amp Black 1m Jumper Cord |
P24675-B21 | |||
|
HPE SDG300 - C20 250V 16Amp Black 0.5m Jumper Cord |
P24672-B21 | |||
|
HPE SDG300 - C20 250V 16Amp Black 2m Jumper Cord |
P24678-B21 | |||
|
HPE SDG23A-SDG23B 277V 0.76m Jumper Cord |
P9B75A | |||
|
HPE SDG23A-SDG23B 277V 0.76m Jumper Cord |
P9B75A | |||
|
| ||||
|
Step 10: Select Rack Options Rack Options (max 1) | ||||
|
HPE 4U Server Rail Kit |
878571-B21 | |||
|
| ||||
|
Step 11: Select Operating Systems | ||||
|
Red Hat HPC Operating Systems (max 99) |
| |||
|
HPE Performance Cluster Manager FIO Software |
Q9V61A | |||
|
Red Hat Enterprise Linux for HPC Head Node 1yr Subscription 24x7 Support E-LTU |
R1P34AAE | |||
|
Red Hat Enterprise Linux for HPC Head Node 3yr Subscription 24x7 Support E-LTU |
R1P35AAE | |||
|
Red Hat Enterprise Linux for HPC Head Node 5yr Subscription 24x7 Support E-LTU |
R1P36AAE | |||
|
Red Hat Enterprise Linux for HPC Head Node 1yr Subscription 9x5 Support E-LTU |
R1P37AAE | |||
|
Red Hat Enterprise Linux for HPC Head Node 3yr Subscription 9x5 Support E-LTU |
R1P38AAE | |||
|
Red Hat Enterprise Linux for HPC Head Node 5yr Subscription 9x5 Support E-LTU |
R1P39AAE | |||
|
Red Hat Enterprise Linux for HPC Compute Node 1yr Subscription E-LTU |
R1P40AAE | |||
|
Red Hat Enterprise Linux for HPC Compute Node 3yr Subscription E-LTU |
R1P41AAE | |||
|
Red Hat Enterprise Linux for HPC Compute Node 5yr Subscription E-LTU |
R1P42AAE | |||
|
Red Hat Operating Systems (max 99) |
| |||
|
Red Hat Enterprise Linux Server 2 Sockets or 2 Guests 1 Year Subscription 24x7 Support E-LTU |
G3J28AAE | |||
|
Red Hat Enterprise Linux Server 2 Sockets or 2 Guests 1 Year Subscription 9x5 Support E-LTU |
G3J29AAE | |||
|
Red Hat Enterprise Linux Server 2 Sockets or 2 Guests 3 Year Subscription 24x7 Support E-LTU |
G3J30AAE | |||
|
Red Hat Enterprise Linux Server 2 Sockets or 2 Guests 3 Year Subscription 9x5 Support E-LTU |
G3J31AAE | |||
|
Red Hat Enterprise Linux Server 2 Sockets or 2 Guests 5 Year Subscription 24x7 Support E-LTU |
G3J32AAE | |||
|
Red Hat Enterprise Linux Server 2 Sockets or Guests 5 Year Subscription 9x5 Support E-LTU |
G3J33AAE | |||
|
Red Hat Enterprise Linux 8 FIO Software |
R4T04A | |||
|
HPE Performance Cluster Manager | ||||
|
For additional information, please visit HPE Performance Cluster Manager QuickSpecs here | ||||
|
Description |
SKU | |||
|
HPE Performance Cluster Manager 1 Node 3yr 24x7 Support Perpetual E-LTU |
Q9V60AAE | |||
|
Notes: - One license per node - Includes three years of support - This is an electronic license - This is a perpetual license. The Software will continue working even when the support term ends | ||||
|
HPE Performance Cluster Manager 1 Node 3yr 24x7 Support Perpetual LTU |
Q9V60A | |||
|
Notes: - One license per node. - Includes three years of support. - This is a perpetual license. The software will continue working even when the support term ends. | ||||
|
HPE Performance Cluster Manager FIO Software |
Q9V61A | |||
|
Notes: - This SKU does not include the license. Please order Q9V60AAE. - Order one per node | ||||
|
HPE Performance Cluster Manager Media Kit |
Q9V62A | |||
|
Notes: One media kit per solution. | ||||
|
| ||||
|
HPE Power Distribution Units Power Distribution Units (PDUs) are an integral piece to this data center solution and HPE offers several types. Basic PDUs provide reliable power with 0U or 1U installation options. Metered PDUs have added intelligence to precisely track power usage and switched PDUs provide both local and remote power management. There are additional metered PDUs that are recommended for this solution that are not part of the mainstream PDU product offering. They are as follows:
| ||||
|
HPE Switched 3-phase 66.5kVA/60309 5-wire 100A/277V 21-breaker Vertical NA PDU |
R8P19A | |||
|
HPE Metered 3Ph 66.5kVA/60309 100A 5-wire 480/277V Outlets (21) SDG23/Vertical NA PDU |
879034-B21 | |||
|
HPE Metered 3Ph 39.9kVA/60309 60A 5-wire 480/277V Outlets (21) SDG23/Vertical NA PDU |
880459-B21 | |||
|
HPE Metered 3Ph 57.6kVA/60309 100A 5-wire 80A/230V Outlets (3) C13 (18) C19/Vertical NA PDU |
880460-B21 | |||
|
HPE Metered 3Ph 34.5kVA/60309 60A 5-wire 48A/230V Outlets (3) C13 (18) C19/Vertical NA FIO PDU |
880461-B21 | |||
|
HPE Cray Supercomputer 60A 415V 3 Phase 24 CX PDU |
R4N30A | |||
|
HPE Mtrd 3P 69.1kVA 125A 96A230V FIO PDU |
880462-B21 | |||
|
HPE Metered 3Ph 45.1kVA/60309 63A 5-wire 63A/230V Outlets (3) C13 (18) C19/Vertical INTL FIO PDU |
880463-B21 | |||
|
HPE Cray Supercomputer 63A 400V 3 Phase 24 CX PDU |
R4N29A | |||
|
HPE G2 Metered/Switched 3Ph 17.3kVA/60309 4-wire 48A/208V Out (12) C13 (12) C19/Vertical NA/JP PDU |
P9S22A | |||
|
HPE G2 Metered 3Ph 17.3kVA/60309 60A 4-wire 48A/208V Outlets (12) C13 (12) C19/Vertical NA/JP PDU |
P9R86A | |||
|
HPE G2 Metered Modular 3Ph 17.3kVA/60309 60A 4-wire 48A/208V Outlets (6) C19/1U Horizontal NA/JP PDU |
P9R80A | |||
|
HPE G2 Metered/Switched 3Ph 22kVA/60309 5-wire 32A/230V Out (12) C13 (12) C19/Vertical INTL PDU |
P9S24A | |||
|
HPE G2 Metered 3Ph 22kVA/60309 5-wire 32A/230V Outlets (12) C13 (12) C19/Vertical INTL PDU |
P9R87A | |||
|
HPE G2 Metered Modular 3Ph 22kVA/60309 5-wire 32A/230V Outlets (6) C19/1U Horizontal INTL PDU |
P9R81A | |||
|
| ||||
Technical Specifications
|
HPE Cray XD665 System Chassis / Server | ||
|
Dimensions |
Height |
7.00 in (17.8 cm) - 4U |
|
Width |
19.17 in (48.7 cm) - includes ears | |
|
Depth |
37.20 in (94.5 cm) | |
|
Shipping Dimensions |
Height |
11 in (27.94 cm) |
|
Width |
23.63 in (60 cm) | |
|
Depth |
41.37 in (105.08 cm) | |
|
Weight (approximate) |
Minimum |
145 lbs. (66.0 kg) |
|
Weight (approximate) |
Maximum |
155 lbs. (70.5 kg) |
|
System Inlet Temperature Standard Operating Support |
10° to 30°C (50° to 86°F) |
At sea level with an altitude derating of 1.0°C per every 305 m (1.8°F per every 1000 ft) above sea level to a maximum of 3050 m (10,000 ft), no direct sustained sunlight. The maximum rate of change is 20°C/hr (36°F/hr). The upper limit and rate of change may be limited by the type and number of options installed. |
|
Extended Ambient Operating Support
|
35° to 45°C (95° to 104°F) |
System performance during standard operating support may be reduced if operating with a fan fault or above 30°C (86°F).
For Direct Liquid Cooling configurations, the supported system inlet range is extended to be 35° to 40°C (95° to 104°F) at sea level with an altitude derating of 1.0°C per every 175 m (1.8°F per every 574 ft) above 900 m (2953 ft) to a maximum of 3050 m (10,000 ft). The approved hardware configurations for this system are listed at the URL https://www.hpe.com/support/ASHRAEGen11
For approved hardware configurations, the supported system inlet range is extended to be: 40° to 45°C (104° to 113°F) at sea level with an altitude derating of 1.0°C per every 125 m (1.8°F per every 410 ft) above 900 m (2953 ft) to a maximum of 3050 m (10,000 ft).
System performance may be reduced if operating in the extended ambient operating range or with a fan fault. |
|
-30° to 60°C (-22° to 140°F). |
The maximum rate of change is 20°C/hr (36°F/hr). | |
|
Relative Humidity (non-condensing) Operating |
8% to 90% |
Relative humidity (Rh), 28°C maximum wet bulb temperature, non-condensing. |
|
Non-operating |
5 to 95% |
Relative humidity (Rh), 38.7°C (101.7°F) maximum wet bulb temperature, non-condensing. |
|
Altitude Operating |
3050 m (10,000 ft). |
This value may be limited by the type and number of options installed. The maximum allowable altitude change rate is 457 m/min (1500 ft/min). |
|
Non-operating |
9144 m (30,000 ft). |
The maximum allowable altitude change rate is 457 m/min (1500 ft/min). |
|
|
|
Thermal limitations For a full list of thermal limitations please see the HPE Cray XD665 System thermal guidelines. https://www.hpe.com/support/xd665-thermal |
|
|
Summary of Changes
|
Date |
Version History |
Action |
Description of Change |
|
02-Dec-2024 |
Version 11 |
Changed |
Overview section was updated |
|
04-Nov-2024 |
Version 10 |
Changed |
TCO section was removed |
|
21-Oct-2024 |
Version 9 |
Changed |
Standard Features section was updated |
|
03-Jun-2024 |
Version 8 |
Changed |
Configuration Information section was updated |
|
06-May-2024 |
Version 7 |
Changed |
Series name was updated. Configuration Information section was updated |
|
01-Apr-2024 |
Version 6 |
Changed |
Standard Features and Configuration Information sections were updated |
|
08-Jan-2024 |
Version 5 |
Changed |
Overview and Standard Features sections were updated |
|
04-Dec-2023 |
Version 4 |
Changed |
Added Cooling Options (Air & DLC), Thermal Tables for CPU, Memory |
|
20-Nov-2023 |
Version 3 |
Changed |
Configuration Information section was updated |
|
16-Oct-2023 |
Version 2 |
Changed |
Technical Specifications section was updated |
|
05-Sep-2023 |
Version 1 |
New |
New QuickSpecs Draft - HPE Cray Supercomputing XD665 System |
Copyright
|
|
| ||||||
|
|
|
© Copyright 2024 Hewlett Packard Enterprise Development LP. The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein. Microsoft and Windows NT are US registered trademarks of Microsoft Corporation. Intel, the Intel logo, Xeon and Xeon Inside are trademarks of Intel Corporation in the U.S. and other countries. AMD OpteronTM is a US registered trademark of AMD Corporation. a50007004enw - 17114 - Worldwide - V11 - 02-December-2024 | ||||||

































