openstack-裸金属ironic创建及调度-1

注册裸机节点

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
#openstack baremetal node create --driver ipmi --name BM01
+------------------------+--------------------------------------+
| Field                  | Value                                |
+------------------------+--------------------------------------+
| allocation_uuid        | None                                 |
| automated_clean        | None                                 |
| bios_interface         | no-bios                              |
| boot_interface         | pxe                                  |
| chassis_uuid           | None                                 |
| clean_step             | {}                                   |
| conductor              | openstack07.control    |
| conductor_group        |                                      |
| console_enabled        | False                                |
| console_interface      | ipmitool-socat                       |
| created_at             | 2021-04-25T06:27:44+00:00            |
| deploy_interface       | iscsi                                |
| deploy_step            | {}                                   |
| description            | None                                 |
| driver                 | ipmi                                 |
| driver_info            | {}                                   |
| driver_internal_info   | {}                                   |
| extra                  | {}                                   |
| fault                  | None                                 |
| inspect_interface      | no-inspect                           |
| inspection_finished_at | None                                 |
| inspection_started_at  | None                                 |
| instance_info          | {}                                   |
| instance_uuid          | None                                 |
| last_error             | None                                 |
| maintenance            | False                                |
| maintenance_reason     | None                                 |
| management_interface   | ipmitool                             |
| name                   | BM01                                 |
| network_interface      | neutron                              |
| owner                  | None                                 |
| power_interface        | ipmitool                             |
| power_state            | None                                 |
| properties             | {}                                   |
| protected              | False                                |
| protected_reason       | None                                 |
| provision_state        | enroll                               |
| provision_updated_at   | None                                 |
| raid_config            | {}                                   |
| raid_interface         | no-raid                              |
| rescue_interface       | no-rescue                            |
| reservation            | None                                 |
| resource_class         | None                                 |
| storage_interface      | noop                                 |
| target_power_state     | None                                 |
| target_provision_state | None                                 |
| target_raid_config     | {}                                   |
| traits                 | []                                   |
| updated_at             | None                                 |
| uuid                   | 5e10c067-c0c6-48be-ac6e-749a70045b97 |
| vendor_interface       | ipmitool                             |
+------------------------+--------------------------------------+

当前许多 ironic node info 都是 None，需要后续继续更新。

  • 设置部署接口类型，现在可支持 iSCSI、Direct、Ansible 等类型，每种类型都有不同的行为模型，可根据实际情况选择，这里我们选择direct 、不使用（占用 Provisioning Network 的带宽）的 iSCS 类型。
1
2
openstack baremetal  node set 5e10c067-c0c6-48be-ac6e-749a70045b97 \
    --deploy-interface direct
  • 设置 driver_info，这里即 IPMI info，主要是提供 IPMI 的登录账户信息。
1
2
3
4
5
openstack baremetal node set 5e10c067-c0c6-48be-ac6e-749a70045b97 \
    --driver-info ipmi_username=admin \
    --driver-info ipmi_password=admin \
    --driver-info ipmi_address=172.18.22.106 \
    --driver-info ipmi_port=623

NOTE：IPMI Driver 官方文档

  • 设置 Deploy Images，通过 RAMDisk 的方式启动。
1
2
3
4
5
openstack image list

openstack baremetal node set 5e10c067-c0c6-48be-ac6e-749a70045b97 \
    --driver-info deploy_kernel=07852c3c-f4b4-43ab-b6b3-aeafe12c07d4 \
    --driver-info deploy_ramdisk=e3429cf3-8fed-4ff8-afb0-50056aacc8f0
  • 设置 Provisioning/Cleaning Network。
1
2
3
4
5
6
7
neutron net-create ironic_deploy --shared  --provider:network_type flat --provider:physical_network physnet1

neutron subnet-create --name flat-subnet --gateway 192.168.99.1 --dns-nameserver 8.8.8.8  --allocation-pool start=192.168.99.120,end=192.168.99.250  ironic_deploy 192.168.99.0/24

openstack baremetal node set 5e10c067-c0c6-48be-ac6e-749a70045b97 \
    --driver-info cleaning_network=7fa970fd-60c7-4f7e-83a2-cd611470dfc6 \
    --driver-info provisioning_network=7fa970fd-60c7-4f7e-83a2-cd611470dfc6
  • 设置 ironic node 的 PXE 网卡 MAC 地址，在 Provisioning Network 中通过这个 MAC 地址来为其分配 IP 地址。
1
2
openstack baremetal port create 78:ac:44:04:f1:a9 --node 5e10c067-c0c6-48be-ac6e-749a70045b97

NOTE：部署裸金属实例成功之后 PXE 网卡的 MAC 地址会被绑定到对应的 Tenant Network Port。

  • 为 ironic node 设置 Placement 筛选候选人的 Resource Class 类型，nova-compute for Ironic 会自动为其创建 Placement Resource Provider。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
$ openstack baremetal node set 5e10c067-c0c6-48be-ac6e-749a70045b97 --resource-class BAREMETAL

$ openstack resource provider list
+--------------------------------------+--------------------------------------+------------+--------------------------------------+----------------------+
| uuid                                 | name                                 | generation | root_provider_uuid                   | parent_provider_uuid |
+--------------------------------------+--------------------------------------+------------+--------------------------------------+----------------------+
| f00d6063-1253-4103-8f4d-899ea700936f | openstack08.control    |          5 | f00d6063-1253-4103-8f4d-899ea700936f | None                 |
| 0cbb35fe-44b7-4fe7-a93f-41ffacaf6a0a | openstack07.control    |          5 | 0cbb35fe-44b7-4fe7-a93f-41ffacaf6a0a | None                 |
| 9ee63716-854f-4554-8211-9de6516e3aa3 | 9ee63716-854f-4554-8211-9de6516e3aa3 |          3 | 9ee63716-854f-4554-8211-9de6516e3aa3 | None                 |
| 4a22be4a-fd35-43d8-842e-6ac7db30bd4e | 4a22be4a-fd35-43d8-842e-6ac7db30bd4e |          0 | 4a22be4a-fd35-43d8-842e-6ac7db30bd4e | None                 |
+--------------------------------------+--------------------------------------+------------+--------------------------------------+----------------------+

$ openstack resource provider inventory list 5e10c067-c0c6-48be-ac6e-749a70045b97  
+------------------+------------------+----------+----------+----------+-----------+-------+
| resource_class   | allocation_ratio | min_unit | max_unit | reserved | step_size | total |
+------------------+------------------+----------+----------+----------+-----------+-------+
| CUSTOM_BAREMETAL |              1.0 |        1 |        1 |        1 |         1 |     1 |
+------------------+------------------+----------+----------+----------+-----------+-------+
  • 为 ironic node 设置 Placement 筛选候选人的 Resource Traits 标签。
1
2
3
4
5
6
7
8
9
10
11

openstack baremetal node add trait 5e10c067-c0c6-48be-ac6e-749a70045b97 \
   HW_CPU_X86_VMX

[root@controller ~]# openstack resource provider trait list e322f49a-ad50-468d-a031-29bde068c290
+----------------+
| name           |
+----------------+
| HW_CPU_X86_VMX |
| CUSTOM_TRAIT1  |
+----------------+

NOTE：这个操作需要较高的 Placement API 版本 <= 1.17

  • 设置 ironic node 的基础资源信息，作为 Placement 筛选候选人的参数因子。
1
2
3
4
openstack baremetal node set 5e10c067-c0c6-48be-ac6e-749a70045b97 \
--property cpus=40 \
--property memory_mb=262144 \
--property local_gb=600
  • 如果裸机服务器设定的是 UEFI，那么需要设置 ironic node 的 boot mode。(传统bios请忽略！！！)
1
openstack baremetal node set 5e10c067-c0c6-48be-ac6e-749a70045b97 --property capabilities='boot_mode:uefi'
  • 验证上述录入的 ironic node infos 是否合规。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
$ openstack baremetal node validate e322f49a-ad50-468d-a031-29bde068c290
+------------+--------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Interface  | Result | Reason                                                                                                                                                                                                                                                                    |
+------------+--------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| bios       | False  | Driver ipmi does not support bios (disabled or not implemented).                                                                                                                                                                                                          |
| boot       | False  | Cannot validate image information for node e322f49a-ad50-468d-a031-29bde068c290 because one or more parameters are missing from its instance_info and insufficent information is present to boot from a remote volume. Missing are: ['ramdisk', 'kernel', 'image_source'] |
| console    | False  | Missing 'ipmi_terminal_port' parameter in node's driver_info.                                                                                                                                                                                                             |
| deploy     | False  | Cannot validate image information for node e322f49a-ad50-468d-a031-29bde068c290 because one or more parameters are missing from its instance_info and insufficent information is present to boot from a remote volume. Missing are: ['ramdisk', 'kernel', 'image_source'] |
| inspect    | True   |                                                                                                                                                                                                                                                                           |
| management | True   |                                                                                                                                                                                                                                                                           |
| network    | True   |                                                                                                                                                                                                                                                                           |
| power      | True   |                                                                                                                                                                                                                                                                           |
| raid       | True   |                                                                                                                                                                                                                                                                           |
| rescue     | False  | Driver ipmi does not support rescue (disabled or not implemented).                                                                                                                                                                                                        |
| storage    | True   |                                                                                                                                                                                                                                                                           |
+------------+--------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

NOTE：这里出现了 4 个 False，但没有致命影响，其中 bios、console 是因为我们没有提供相应的驱动，属于可选项。而 boot、deploy 在 Nova Driver for Ironic 的环境中是无法通过验证的。

  • 验证 ironic node 是否能够被纳管。
1
2
3
4
5
6
7
8
9
# To move a node from enroll to manageable provision state
$ openstack baremetal node manage 5e10c067-c0c6-48be-ac6e-749a70045b97
$ openstack baremetal node show 5e10c067-c0c6-48be-ac6e-749a70045b97 | grep provision_state
| provision_state        | manageable  
                                                                                                                                                                                                                                                                                                                                                        
# To move a node from manageable to available provision state
$ openstack baremetal node provide 5e10c067-c0c6-48be-ac6e-749a70045b97
$ [root@controller ~]# openstack baremetal node show 5e10c067-c0c6-48be-ac6e-749a70045b97 | grep provision_state
| provision_state        | available
  • 查看当前的 ironic node 状态
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
[root@controller ~]# openstack baremetal node show 5e10c067-c0c6-48be-ac6e-749a70045b97
+------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field                  | Value                                                                                                                                                                                                                                                                                                                                                    |
+------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| allocation_uuid        | None                                                                                                                                                                                                                                                                                                                                                     |
| automated_clean        | None                                                                                                                                                                                                                                                                                                                                                     |
| bios_interface         | no-bios                                                                                                                                                                                                                                                                                                                                                  |
| boot_interface         | pxe                                                                                                                                                                                                                                                                                                                                                      |
| chassis_uuid           | None                                                                                                                                                                                                                                                                                                                                                     |
| clean_step             | {}                                                                                                                                                                                                                                                                                                                                                       |
| conductor              | openstack08.control                                                                                                                                                                                                                                                                                                                        |
| conductor_group        |                                                                                                                                                                                                                                                                                                                                                          |
| console_enabled        | False                                                                                                                                                                                                                                                                                                                                                    |
| console_interface      | ipmitool-socat                                                                                                                                                                                                                                                                                                                                           |
| created_at             | 2021-04-25T10:13:09+00:00                                                                                                                                                                                                                                                                                                                                |
| deploy_interface       | direct                                                                                                                                                                                                                                                                                                                                                   |
| deploy_step            | {}                                                                                                                                                                                                                                                                                                                                                       |
| description            | None                                                                                                                                                                                                                                                                                                                                                     |
| driver                 | ipmi                                                                                                                                                                                                                                                                                                                                                     |
| driver_info            | {'ipmi_port': 623, 'ipmi_username': 'admin', 'deploy_kernel': '07852c3c-f4b4-43ab-b6b3-aeafe12c07d4', 'ipmi_address': '172.18.22.106', 'deploy_ramdisk': 'e3429cf3-8fed-4ff8-afb0-50056aacc8f0', 'ipmi_password': '******', 'provisioning_network': '7fa970fd-60c7-4f7e-83a2-cd611470dfc6', 'cleaning_network': '7fa970fd-60c7-4f7e-83a2-cd611470dfc6'} |
| driver_internal_info   | {}                                                                                                                                                                                                                                                                                                                                                       |
| extra                  | {}                                                                                                                                                                                                                                                                                                                                                       |
| fault                  | None                                                                                                                                                                                                                                                                                                                                                     |
| inspect_interface      | no-inspect                                                                                                                                                                                                                                                                                                                                               |
| inspection_finished_at | None                                                                                                                                                                                                                                                                                                                                                     |
| inspection_started_at  | None                                                                                                                                                                                                                                                                                                                                                     |
| instance_info          | {}                                                                                                                                                                                                                                                                                                                                                       |
| instance_uuid          | None                                                                                                                                                                                                                                                                                                                                                     |
| last_error             | None                                                                                                                                                                                                                                                                                                                                                     |
| maintenance            | False                                                                                                                                                                                                                                                                                                                                                    |
| maintenance_reason     | None                                                                                                                                                                                                                                                                                                                                                     |
| management_interface   | ipmitool                                                                                                                                                                                                                                                                                                                                                 |
| name                   | BM01                                                                                                                                                                                                                                                                                                                                                     |
| network_interface      | neutron                                                                                                                                                                                                                                                                                                                                                  |
| owner                  | None                                                                                                                                                                                                                                                                                                                                                     |
| power_interface        | ipmitool                                                                                                                                                                                                                                                                                                                                                 |
| power_state            | power on                                                                                                                                                                                                                                                                                                                                                 |
| properties             | {}                                                                                                                                                                                                                                                                                                                                                       |
| protected              | False                                                                                                                                                                                                                                                                                                                                                    |
| protected_reason       | None                                                                                                                                                                                                                                                                                                                                                     |
| provision_state        | available                                                                                                                                                                                                                                                                                                                                                |
| provision_updated_at   | 2021-04-25T10:30:35+00:00                                                                                                                                                                                                                                                                                                                                |
| raid_config            | {}                                                                                                                                                                                                                                                                                                                                                       |
| raid_interface         | no-raid                                                                                                                                                                                                                                                                                                                                                  |
| rescue_interface       | no-rescue                                                                                                                                                                                                                                                                                                                                                |
| reservation            | None                                                                                                                                                                                                                                                                                                                                                     |
| resource_class         | BAREMETAL                                                                                                                                                                                                                                                                                                                                                |
| storage_interface      | noop                                                                                                                                                                                                                                                                                                                                                     |
| target_power_state     | None                                                                                                                                                                                                                                                                                                                                                     |
| target_provision_state | None                                                                                                                                                                                                                                                                                                                                                     |
| target_raid_config     | {}                                                                                                                                                                                                                                                                                                                                                       |
| traits                 | ['HW_CPU_X86_VMX']                                                                                                                                                                                                                                                                                                                                       |
| updated_at             | 2021-04-25T10:30:35+00:00                                                                                                                                                                                                                                                                                                                                |
| uuid                   | 5e10c067-c0c6-48be-ac6e-749a70045b97                                                                                                                                                                                                                                                                                                                     |
| vendor_interface       | ipmitool                                                                                                                                                                                                                                                                                                                                                 |
+------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
[root@17471e0f8de8 sh1]#
  • 验证，自动添加了 Ironic Neutron Agent。
1
2
[root@baremetal ~]# openstack network agent list
2ecbcf17-b11b-4e6a-87b5-349dde0fcee3 | Baremetal Node | 5e10c067-c0c6-48be-ac6e-749a70045b97 | None              | :-)   | UP    | ironic-neutron-agent

参考：https://docs.openstack.org/ironic/stein/admin/boot-from-volume.html#storage-interface