HPE Inference Solutions on ProLiant: технические характеристики и документация

HPE Inference Solution on ProLiant

The HPE Inference Solution - ProLiant is an AI/ML platform optimized for accelerating ML inference by providing a multi-tenant, self-service environment for data scientists to securely build, deploy, and scale GPU optimized and containerized ML and DL models. This solution consists of HPE ProLiant DL servers with NVIDIA® T4 GPUs running NGC Containers with orchestration through Red Hat® OpenShift® Container Platform.  It aims to reduce development time for AI-powered intelligent applications with utilization of integrated DevOps capabilities that can be extended to the full ML/DL lifecycle.  Organizations can speed up procurement, deployment, and scaling of their AI infrastructure with this pre-engineered solution ideal for AI Inference workloads in a multitude of environments. 

In today's data driven economy, to remain competitive, serve customers better, increase revenue, and save costs, businesses must invest in artificial intelligence (AI) and machine learning (ML) tools and applications. Using AI, organizations are developing and putting into production processes and industry applications that automatically learn, discover, and make recommendations or predictions that can be used to set strategic goals and provide a competitive advantage. To accomplish these strategic goals, organizations require data scientists and analysts who are skilled in developing models and performing analytics on enterprise data. These data analysts require specialized tools, applications, and compute resources to create complex models and analyze massive amounts of data. Deploying and managing these compute intensive applications can place a burden on already taxed IT organizations. Red Hat® OpenShift® Container Platform, running on HPE ProLiant DL servers with NVIDIA® T4 GPUs, can provide an organization with a powerful, highly available DevOps platform that allows these compute intensive data analytics tools and applications to be rapidly deployed to end users through a self-service OpenShift registry and catalog. Additionally, OpenShift compute nodes equipped with NVIDIA® T4 GPUs provide the compute resources to meet the performance requirements of these computationally intensive applications and queries.

What's Included

• 1 - HPE 42U 600mmx1200mm G2 Kitted Advanced Shock Rack
• 1 - HPE Aruba Networking 2930F 48G 4SFP Switch
• 1 - HPE SN2100M 100GbE 16QSFP28 Half Width Switch
• 3 - HPE DL360 Gen10 8SFF CTO Server
• 3 - HPE DL380 Gen10 8SFF CTO Server
• 8 - NVIDIA® T4 GPUs
• 48 - Red Hat® OpenShift® Container Platform 2 Core /4vCPU 1yr 9x5 License
• 48 - Red Hat® OpenShift® Container Storage 2 Cores 1yr 9x5 License
• Factory Express Installation Services
• Point Next Red Hat Container Deployment Service (see Installation, Startup, and Support Services section) 

For full BOM please see Reference Architecture: http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50001578enw . 

XXXXXX-X21 is SKU designation formed by a common six-digit part number and a -X21 suffix that identifies a SKU that is available across multiple server family lines.  Refer to the table below to find the SKU suffix that applies to the specific server product line this option can be ordered with.

-B21

-H21

-K21

COMPUTE Server Line

SPECIALIZED COMPUTE Server Line

STORAGE Line

HPE Cloudline CL2100/CL2200/CL2800/CL3100/
CL4100/CL5200/CL5800 Servers

HPE Composable Cloud for ProLiant DL

HPE ProLiant BL460c/BL660c Servers

HPE ProLiant DL20/DL160/DL180 Servers

HPE ProLiant DL325/DL360/DL380/DL385/
DL560/DL580 Servers

HPE ProLiant DX360/DX380 Servers

HPE ProLiant MicroServer

HPE ProLiant for Microsoft Azure Stack

HPE ProLiant ML30/ML110/ML350 Servers

HPE Synergy 480/660 Systems

HPE ProLiant DX170r/DX190r, DX2000 Servers

HPE ProLiant DX560 Gen10 server

HPE ProLiant DX4200 Gen10 server

HPE Apollo 35/40/70 Systems

HPE Apollo 2000/6000 Servers

HPE XL170r/XL190r/XL270d (Apollo 6500) Gen10 Server for BlueData Software

HPE Converged System 300/500/700/750

HPE Edgeline Systems and Servers

HPE Integrity BL860c i6/BL870c i6/BL890c i6 Server Blades

HPE Integrity MC990 X Server

HPE Integrity rx2800 i6 Server

HPE Integrity Superdome

HPE SGI 8600 System

HPE Solutions for SAP HANA (TDI)

HPE Apollo 4200 Gen9/Gen10 Servers

HPE Apollo 4200 Gen10 LFF Server for BlueData Software

HPE Apollo 4510 Gen10 System

HPE D2220sb/D2500sb Storage Blade

HPE D3000/D6020/D8000 Disk Enclosures

HPE Scalable Object Storage with Scality RING

HPE SimpliVity 2600

HPE SimpliVity 325/380 Gen10

HPE Storage File Controllers

HPE StoreEasy 1460/1560/1650/1660/1860

Disclaimer: This may not be a complete listing of applicable servers

At A Glance

The HPE Inference solution - ProLiant provides customers with the fastest path to start running AI workloads.  The software stack is optimized for inferencing with pre-trained or transfer learning AI/ML models using container orchestration through an infrastructure that provides broad scalability and flexibility for each unique environment. 

At the top of the stack, this solution is optimal for pulling NVIDIA NGC Containers.  NGC manages a catalog of fully integrated and optimized deep learning framework containers that take full advantage of NVIDIA GPUs in both single GPU and multi-GPU configurations.  This includes major deep learning frameworks such as TensorFlow, PyTorch, CNTK, and many others.  NVIDIA updates these deep learning containers monthly to ensure they continue to provide peak performance with their GPUs and will increase the longevity of the solution.

Orchestration of the containers in the stack is managed through the Red Hat OpenShift Container Platform.  This platform seamlessly builds, deploys, and scales containers with Kubernetes Orchestration, allowing developers more time designing and testing, and less time deploying and managing containers.  With Openshift organizations can streamline their developer's workflow, create repeatable process for delivery, and easily deploy at the simple click of a button.   With the wide range of provisioning and monitoring tools, an extensive ecosystem, and leading expertise support, OpenShift provides an ideal foundation for container orchestration and management.

The hardware infrastructure consists of three HPE DL360 Gen10 servers, performing as the OCP main nodes, two HPE ProLiant DL380 Gen10 servers equipped with NVIDIA T4 GPU modules as the worker nodes, and one HPE ProLiant DL380 Gen10 server that will be used as the OCP bootstrap and can be converted afterwards to an additional worker node.  The end-to-end solutions are covered by HPE Tech Care, Red Hat, and NGC support services and can be sourced entirely from Hewlett Packard Enterprise, inclusive of Red Hat and NGC licenses.

Additional key features and benefits of this solution include:

• Agility and Flexibility - Utilizing containers with Kubernetes orchestration, customers can deploy multiple AI models to production that can work together seamlessly through independent, lightweight microservices.  Allowing them the agility, flexibility, and portability they need to update, deploy, and manage their AI easily without further taxation on DevOps.  
• Accelerated - This solution takes advantage of NVIDIA® T4 GPUs to accelerate Inference.  The NVIDIA® T4 GPU accelerates AI workloads, including high-performance computing, deep learning training and inference, machine learning, data analytics, and graphics. Based on the new NVIDIA Turing^TM architecture and packaged in an energy-efficient 70-watt, small PCIe form factor, T4 is optimized for mainstream computing environments and features multi-precision Turing Tensor Cores and new RT Cores. Combined with accelerated containerized software stacks from NGC, T4 delivers revolutionary performance at scale.  
• GPU Optimized Containers - Customers can utilize NVIDIA® NGC Containers with this solution to further optimize the speed of deployment, updating, and the performance of their AI containers.  NGC manages a catalog of fully integrated and optimized deep learning framework containers that take full advantage of NVIDIA GPUs in both single GPU and multi-GPU configurations. They include CUDA Toolkit, DIGITS workflow, and many deep learning frameworks such as: NVCaffe, Caffe2, Microsoft Cognitive Toolkit (CNTK), MXNet, PyTorch, TensorFlow, Theano, and Torch. These framework containers are delivered ready-to-run, including all necessary dependencies such as CUDA runtime, NVIDIA libraries, and an operating system.  The top deep learning containers are updated monthly to keep customer systems running at peak performance. All containers provide easy access to fully tested and optimized software releases.

Scalable - This solution comes default with 3 DL380 worker nodes.  Two with GPU accelerators, and one for additional compute (converted from bootstrap).  However, additional worker nodes can easily be added to the cluster.  Allowing customers to further scale their infrastructure specifically to their requirements.

• NGC Ready - The ProLiant DL380 is certified by NVIDIA as NGC Ready, having passed an extensive suite of tests that validates its ability to deliver high performance running NGC containers.  Being certified NGC Ready allows support by NVIDIA NGC Support Services which provides enterprise-grade support to ensure NGC-Ready systems run optimally, maximizing system utilization and user productivity. The service gives enterprise IT direct access to NVIDIA subject matter experts to quickly address software issues and minimize system downtime.
  
• Factory Express - This solution is Factory Express enabled by default.  With factory integration, the hardware is assembled and configured in the HPE factory specifically to this solution's requirements using extensively tested configuration guidelines.
  
• Point Next Container Adoption Service - The HPE Inference Solution is qualified for utilizing HPE's Container Adoption Service.  With this service, customers can take advantage of HPE's experience and expertise in container technology in helping them adopt their container platform strategy.  Allowing them to effectively learn, develop, architect, migrate, deploy, and operate container platforms in large-scale environments in a way that can achieve significant cost savings.  This service, in conjunction with Factory Express, will provide a turnkey approach for the customer, not only properly configuring their hardware, but also Red Hat OCP to get them up and running immediately.
• Industry leading support - Hewlett Packard Enterprise offers its HPE 4-hour 24x7 Tech Care Service as base support, with options for Tech Care Advanced as well as Complete Care. This support experience is designed to accelerate time to resolution and is delivered by an HPE Center of Excellence staffed with HPE support experts well versed in all aspects of HPE offerings.  Customers also have the ability to select from various service level options such as HPE 6-Hour Call-to-Repair Hardware Support, or options for defective media retention.

HPE Web Sites

• Reference Architecture: http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50001578enw
• Alliance page: https://www.hpe.com/us/en/alliance/red-hat.html

HPE Services

No matter where you are in your digital transformation journey, you can count on HPE Services to deliver the expertise you need when, where and how you need it. From planning to deployment, ongoing operations and beyond, our experts can help you realize your digital ambitions.

https://www.hpe.com/services

Consulting Services

No matter where you are in your journey to hybrid cloud, experts can help you map out your next steps.  From determining what workloads should live where, to handling governance and compliance, to managing costs, our experts can help you optimize your operations. 

https://www.hpe.com/services/consulting

HPE Managed Services

HPE runs your IT operations, providing services that monitor, operate, and optimize your infrastructure and applications, delivered consistently and globally to give you unified control and let you focus on innovation. 

HPE Managed Services | HPE

Operational services

Optimize your entire IT environment and drive innovation.  Manage day-to-day IT operational tasks while freeing up valuable time and resources. Meet service-level targets and business objectives with features designed to drive better business outcomes.

https://www.hpe.com/services/operational

HPE Complete Care Service

HPE Complete Care Service is a modular, edge-to-cloud IT environment service designed to help optimize your entire IT environment and achieve agreed upon IT outcomes and business goals through a personalized experience. All delivered by an assigned team of HPE Services experts.  HPE Complete Care Service provides:

• A complete coverage approach -- edge to cloud
• An assigned HPE team
• Modular and fully personalized engagement
• Enhanced Incident Management experience with priority access
• Digitally enabled and AI driven customer experience  

https://www.hpe.com/services/completecare

HPE Tech Care Service

HPE Tech Care Service is the operational support service experience for HPE products. The service goes beyond traditional support by providing access to product specific experts, an AI driven digital experience, and general technical guidance to not only reduce risk but constantly search for ways to do things better. HPE Tech Care Service delivers a customer-centric, AI driven, and digitally enabled customer experience to move your business forward. HPE Tech Care Service is available in three response levels. Basic, which provides 9x5 business hour availability and a 2-hour response time. Essential which provides a 15-minute response time 24x7 for most enterprise level customers, and Critical which includes a 6-hour repair commitment where available and outage management response for severity 1 incidents.

https://www.hpe.com/services/techcare

HPE Lifecycle Services

HPE Lifecycle Services provide a variety of options to help maintain your HPE systems and solutions at all stages of the product lifecycle.  A few popular examples include:

• Lifecycle Install and Startup Services: Various levels for physical installation and power on, remote access setup, installation and startup, and enhanced installation services with the operating system.
• HPE Firmware Update Analysis Service: Recommendations for firmware revision levels for selected HPE products, taking into account the relevant revision dependencies within your IT environment.
• HPE Firmware Update Implementation Service: Implementation of firmware updates for selected HPE server, storage, and solution products, taking into account the relevant revision dependencies within your IT environment.
• Implementation assistance services: Highly trained technical service specialists to assist you with a variety of activities, ranging from design, implementation, and platform deployment to consolidation, migration, project management, and onsite technical forums.
• HPE Service Credits: Access to prepaid services for flexibility to choose from a variety of specialized service activities, including assessments, performance maintenance reviews, firmware management, professional services, and operational best practices.

Notes: To review the list of Lifecycle Services available for your product go to:
https://www.hpe.com/services/lifecycle 

For a list of the most frequently purchased services using service credits, see the HPE Service Credits Menu

Other Related Services from HPE Services:

HPE Education Services

Training and certification designed for IT and business professionals across all industries. Broad catalogue of course offerings to expand skills and proficiencies in topics ranging from cloud and cybersecurity to AI and DevOps. Create learning paths to expand proficiency in a specific subject.  Schedule training in a way that works best for your business with flexible continuous learning options.

https://www.hpe.com/services/training

Defective Media Retention

An option available with HPE Complete Care Service and HPE Tech Care Service and applies only to Disk or eligible SSD/Flash Drives replaced by HPE due to malfunction. 

Consult your HPE Sales Representative or Authorized Channel Partner of choice for any additional questions and services options.

Parts and Materials

HPE will provide HPE-supported replacement parts and materials necessary to maintain the covered hardware product in operating condition, including parts and materials for available and recommended engineering improvements. 

Parts and components that have reached their maximum supported lifetime and/or the maximum usage limitations as set forth in the manufacturer's operating manual, product quick-specs, or the technical product data sheet will not be provided, repaired, or replaced as part of these services.

How to Purchase Services

Services are sold by Hewlett Packard Enterprise and Hewlett Packard Enterprise Authorized Service Partners:

• Services for customers purchasing from HPE or an enterprise reseller are quoted using HPE order configuration tools.
• Customers purchasing from a commercial reseller can find services at https://ssc.hpe.com/portal/site/ssc/

AI Powered and Digitally Enabled Support Experience

Achieve faster time to resolution with access to product-specific resources and expertise through a digital and data driven customer experience. 

Sign into the HPE Support Center experience, featuring streamlined self-serve case creation and management capabilities with inline knowledge recommendations. You will also find personalized task alerts and powerful troubleshooting support through an intelligent virtual agent with seamless transition when needed to a live support agent. 

https://support.hpe.com/hpesc/public/home/signin

Consume IT On Your Terms

HPE GreenLake edge-to-cloud platform brings the cloud experience directly to your apps and data wherever they are-the edge, colocations, or your data center. It delivers cloud services for on-premises IT infrastructure specifically tailored to your most demanding workloads. With a pay-per-use, scalable, point-and-click self-service experience that is managed for you, HPE GreenLake edge-to-cloud platform accelerates digital transformation in a distributed, edge-to-cloud world.

• Get faster time to market
• Save on TCO, align costs to business
• Scale quickly, meet unpredictable demand
• Simplify IT operations across your data centers and clouds  

To learn more about HPE Services, please contact your Hewlett Packard Enterprise sales representative or Hewlett Packard Enterprise Authorized Channel Partner.  Contact information for a representative in your area can be found at "Contact HPE" https://www.hpe.com/us/en/contact-hpe.html 

For more information

http://www.hpe.com/services

Red Hat Licensing

Notes: This solution uses Red Hat OpenShift Container Storage to provide persistent storage for the OpenShift Image Registry. If OpenShift Container Storage is removed from the configuration then an alternate persistent storage solution must be available to provide persistent storage for the OpenShift Image Registry.

This solution requires licenses for Red Hat OpenShift Container Platform.  OpenShift Container Platform is available on a 2 core per license basis. You must have enough 2 core licenses to equal the number of cores for each worker node that will be a part of the cluster.  Also included as part of this solution is OpenShift Container Storage.  To utilize OCS, licenses are required at the same 2 core license count as the number of cores for each node supporting OCS.  Though not required, if OpenShift Container Storage is not part of the configuration, then an alternate persistent storage solution must be available to provide persistent storage for the OpenShift Image Registry.  

Subscription Activation Instructions for your Red Hat software purchased from HPE

The subscription activation instructions for your Red Hat software purchased from Hewlett Packard Enterprise is a two-step process:

• Step 1: Use your Entitlement Order Number above to register your support on the HPE Licensing Portal http://myenterpriselicense.hpe.com/
• Step 2: Upon completing your support registration, you will receive a Red Hat Activation Key. You must activate your Red Hat subscription by entering this key at: http://www.redhat.com/activate

Red Hat Certification & Supported Matrix

An HPE ProLiant server is certified if it has passed the Red Hat certification test suite. Hewlett Packard Enterprise supports all Red Hat certified servers, which means Hewlett Packard Enterprise will support any HPE delivered driver or management software for Red Hat. 

For more information, visit our support matrix: http://www.hpe.com/servers/rhelcert

How to order

Customers can purchase the HPE Inference Solution - ProLiant directly from Hewlett Packard Enterprise or through Hewlett Packard Enterprise authorized resellers. There is a dedicated solution offering within our Hewlett Packard Enterprise sales and authorized partner ordering tool [OCA]. It is located in the "AI - ML category" under the new "Reference Builds" tab located within the Catalog.  The name of the solution offering within Reference Build is:

HPE Inference FIO Solution - ProLiant

P37712-B21

Installation, Startup, and Support Services

HPE Factory Express

To ensure optimal setup and deployment, Hewlett Packard Enterprise defaults Factory Express installation services as part of the default Bill of Materials.  Used in conjunction with the P37712-B21 SKU will ensure proper installation of hardware per specification of the solution's Reference Architecture: http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=a50001578enw. 

Factory Express SKUs

HPE 42U 600mmx1200mm G2 Kitted Advanced Shock Rack with Side Panels and Baying

P9K10A

HPE Factory Express Complex Unit of SVC

H4F42A1

HPE Integration Center Routing Service

HC790A

HPE Rack Customization Package

ZU708A

HPE Hardware Customization Service

HA838A1

HPE System Settings Customization Service

HA839A1

HPE Container Adoption Service

The HPE Container Adoption service is highly recommended for this solution as it will ensure proper deployment of OCP as well as consultation to help the customer properly design and build into their container platform strategy.  The service can be added via the Statement of Work link located in the warning message of Red Hat Operating Systems section header as well as CLIC.  Look for the following warning message and click the corresponding link to fill out the SOW: 

"If you are configuring HPE Inference Solution - ProLiant, we recommend the SOW for HPE Services A&PS Service for RH OCP consultative implementation services. Please the below link to include in the overall offering for your customer"

NVIDIA NGC Support Services

Support for NVIDIA NGC are available for this solution as well but are not defaulted.  They are highly recommended for customers utilizing NGC. 

HPE Red Hat Software OpenShift Container Platform

Red Hat OpenShift Container Platform 2 Cores or 4 vCPUs 1yr Subscription 9x5 RH Support E-LTU

R6J33AAE

Red Hat OpenShift Container Platform 2 Cores or 4 vCPUs 3yr Subscription 24x7 RH Support E-LTU

R6J34AAE

Red Hat OpenShift Container Platform 2 Cores or 4 vCPUs 3yr Subscription 9x5 RH Support E-LTU

R6J35AAE

Notes: Red Hat OpenShift Container Platform Notes: Support provided by Red Hat

HPE Red Hat OpenShift Storage

Notes:

- Software requirements: Resell offering. Support provided by Red Hat

- This solution uses Red Hat OpenShift Container Storage to provide persistent storage for the OpenShift Image Registry. If OpenShift Container Storage is removed from the configuration then an alternate persistent storage solution must be available to provide persistent storage for the OpenShift Image Registry.

.

© Copyright 2024 Hewlett Packard Enterprise Development LP. The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein.

Microsoft and Windows are registered trademarks or trademarks of Microsoft Corporation in the U.S. and/or other countries.

VMware is a registered trademark of VMware, Inc. in the United States and/or other jurisdictions.

a00094644enw - 16628 - Worldwide - V5 - 19-February-2024