Ironic 裸金属实例的部署流程

目录

逻辑架构

部署架构

前提条件

官方文档：https://docs.openstack.org/ironic/latest/user/index.html

• Dependent packages to be configured on the Bare Metal service node(s) where ironic-conductor is running like tftp-server, ipmi, syslinux etc for bare metal provisioning.
• Nova must be configured to make use of the bare metal service endpoint and compute driver should be configured to use ironic driver on the Nova compute node(s).
• Flavors to be created for the available hardware. Nova must know the flavor to boot from.
• Images to be made available in Glance. Listed below are some image types required for successful bare metal deployment:
  • bm-deploy-kernel
  • bm-deploy-ramdisk
  • user-image
  • user-image-vmlinuz
  • user-image-initrd
• Hardware to be enrolled（注册） via Ironic RESTful API service.

部署流程

1. A boot instance request comes in via the Nova API, through the message queue to the Nova scheduler.
2. Nova scheduler applies filters and finds the eligible hypervisor. The nova scheduler also uses the flavor’s extra_specs, such as cpu_arch, to match the target physical node.
3. Nova compute manager claims the resources of the selected hypervisor.
4. Nova compute manager creates (unbound) tenant virtual interfaces (VIFs) in the Networking service according to the network interfaces requested in the nova boot request. A caveat here is, the MACs of the ports are going to be randomly generated, and will be updated when the VIF is attached to some node to correspond to the node network interface card’s (or bond’s) MAC.
5. A spawn task is created by the nova compute which contains all the information such as which image to boot from etc. It invokes the driver.spawn from the virt layer of Nova compute. During the spawn process, the virt driver does the following:
   • Updates the target ironic node with the information about deploy image, instance UUID, requested capabilities and various flavor properties.
   • Validates node’s power and deploy interfaces, by calling the ironic API.
   • Attaches the previously created VIFs to the node. Each neutron port can be attached to any ironic port or port group, with port groups having higher priority than ports. On ironic side, this work is done by the network interface. Attachment here means saving the VIF identifier into ironic port or port group and updating VIF MAC to match the port’s or port group’s MAC, as described in bullet point 4.
   • Generates config drive, if requested.
6. Nova’s ironic virt driver issues a deploy request via the Ironic API to the Ironic conductor servicing the bare metal node.
7. Virtual interfaces are plugged in and Neutron API updates DHCP port to set PXE/TFTP options. In case of using neutron network interface, ironic creates separate provisioning ports in the Networking service, while in case of flat network interface, the ports created by nova are used both for provisioning and for deployed instance networking.
8. The ironic node’s boot interface prepares (i)PXE configuration and caches deploy kernel and ramdisk.
9. The ironic node’s management interface issues commands to enable network boot of a node.
10. The ironic node’s deploy interface caches the instance image (in case of iscsi deploy interface), and kernel and ramdisk if needed (it is needed in case of netboot for example).
11. The ironic node’s power interface instructs the node to power on.
12. The node boots the deploy ramdisk.
13. Depending on the exact driver used, either the conductor copies the image over iSCSI to the physical node (iSCSI deploy) or the deploy ramdisk downloads the image from a temporary URL (Direct deploy). The temporary URL can be generated by Swift API-compatible object stores, for example Swift itself or RadosGW. The image deployment is done.
14. The node’s boot interface switches pxe config to refer to instance images (or, in case of local boot, sets boot device to disk), and asks the ramdisk agent to soft power off the node. If the soft power off by the ramdisk agent fails, the bare metal node is powered off via IPMI/BMC call.
15. The deploy interface triggers the network interface to remove provisioning ports if they were created, and binds the tenant ports to the node if not already bound. Then the node is powered on.
16. The bare metal node’s provisioning state is updated to active.

iSCSI Deploy UML

With iscsi deploy interface, the deploy ramdisk publishes the node’s hard drive as an iSCSI share. The ironic-conductor then copies the image to this share. This interface is used by default.

openstack baremetal node create --driver ipmi --deploy-interface iscsi
openstack baremetal node set  --deploy-interface iscsi

1. 通过 Nova API 发起裸金属实例部署请求。
2. Nova Scheduler 根据请求中的参数信息（e.g. 硬件模板、镜像等）筛选合适的 ironic node。例如：调度因子通常包含在 Flavor 的 extra_spec 属性中（e.g. cpu_arch、baremetal:deploy_kerner_id、baremmetal:deploy_ramdisk_id）。
3. Nova Compute 将部署裸机所需要的信息都整理好，然后 Spawn 一个 Build 任务，主要是调用并传递参数到 Ironic API 实际执行，Ironic 将此任务中所需要的硬件资源信息持久化到数据库中。
4. Ironic 与 OpenStack 的其他服务交互，从 Glance 服务获取 Images（Deploy Images & User Images），调用 Neutron 服务为裸机创建端口，调用 Cinder 服务获取Volumes 等。
5. Ironic 开始执行真正的裸机部署，PXE Driver 准备好 TFTP 和 Bootloader，IPMI Driver 设置裸机启动模式为 PXE 并起电。
6. 裸机启电后，通过 DHCP 获得 Ironic Conductor 的地址并尝试通过 TFTP 从 Conductor 获取 Deploy Images，Conductor 部署好 RAMDisk 之后，IPA 就可以通过 iSCSI 协议将裸机的硬盘暴露出来，Conductor 随后注入 User Images 到裸机磁盘作为根磁盘。
7. 根磁盘部署完成后，IPMI Driver 调整裸机引导顺序，完成部署。

PXE Deploy Driver

Direct Deploy UML

*With direct deploy interface, the deploy ramdisk fetches the image from an HTTP location. It can be an object storage (swift or RadosGW) temporary URL or a user-provided HTTP URL. The deploy ramdisk then copies the image to the target disk. *

openstack baremetal node create --driver ipmi --deploy-interface direct
openstack baremetal node set  --deploy-interface direct

1. 加入 Provision Network。
2. 重启裸机。
3. IPA 通过 lookup 从 Ironic Conductor 取得它对应的 ironic_node_uuid。
4. IPA 通过 Heartbeat 与 Ironic 通信，驱动部署流程触发 ironic-conductor 发送 prepare_image 命令到 IPA，让 IPA 直接**（Direct）**下载并注入用户镜像到本地磁盘。写镜像是一个耗时较长的动作，通常 300GB 大小的镜像需要十分钟左右。
5. ironic-conductor 收到 Heartbeat 消息后（从 ironic-api 过来的 RPC 消息），根据当前状态的不同有相应的处理。第一个 Heartbeat 消息会触发 ironic-conductor 下发 prepare_image 到 IPA，以后的 Heartbeat 消息会触发 ironic-conductor 查询 IPA 上的 command status，它的目的是要监控 prepare_image 等 command 是否还在运行中。一旦 command status 显示命令结束。就会继续下一步部署操作，包括设置物理服务器从磁盘启动、关机、切换网络、开机。知道部署流程结束。

IPA Deploy Driver