How to use tempest in kolla

Tempest 介绍

在接触tempest之前我们应该有如下疑问。

Tempest是什么

Tempest为OpenStack的功能测试、集成测试项目,它被设计为可在各种不同项目中使用。在OpenStack核心项目中的单元测试代码中经常可以看到它的身影,在一些孵化项目中也会使用Tempest去测试。它可以验证代码的正确性,已经成为OpenStack项目中不可或缺的组成部分.

主要用于什么测试?

(1)API接口测试

(2)复杂场景测试

(3)单元测试

使用kolla部署tempest

  1. 在/etc/kolla/globals.yml开启tempest同时设置tempest相应的配置参数(image_id等)

    注: 我是使用cirros镜像进行测试,按照实际情况替换相应参数

1
2
3
4
5
6
[root@control01 ~]# vim /etc/kolla/globals.yml
enable_tempest: "yes"
tempest_image_id: "570cab5e-4609-43cc-8c1e-3f7d7217ed30"
tempest_flavor_ref_id: "1"
tempest_public_network_id: "4c5928f2-6ade-498f-a692-98a775a7183a"
tempest_floating_network_name: "public1"

参数以如下命令获得

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# image id
[root@control01 ~]# openstack image show cirros -c id -f value
570cab5e-4609-43cc-8c1e-3f7d7217ed30
# flavor_ref_id
[root@control01 ~]# nova flavor-list
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
| ID | Name | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public |
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
| 1 | m1.tiny | 512 | 1 | 0 | | 1 | 1.0 | True |
| 2 | m1.small | 2048 | 20 | 0 | | 1 | 1.0 | True |
| 3 | m1.medium | 4096 | 40 | 0 | | 2 | 1.0 | True |
| 4 | m1.large | 8192 | 80 | 0 | | 4 | 1.0 | True |
| 5 | m1.xlarge | 16384 | 160 | 0 | | 8 | 1.0 | True
# public_network_id
[root@control01 ~]# openstack network show public1 -c id -f value
4c5928f2-6ade-498f-a692-98a775a7183a
# floating_network_name
[root@control01 ~]# openstack network list
+--------------------------------------+----------+--------------------------------------+
| ID | Name | Subnets |
+--------------------------------------+----------+--------------------------------------+
| 0474116d-0fbd-41a3-ab0c-2c2d11545937 | demo-net | 7366e03d-027b-4a44-b270-d0478584f3e8 |
| 4c5928f2-6ade-498f-a692-98a775a7183a | public1 | 2c47f34c-f55a-4739-a677-2cbfa56b6312 |
+--------------------------------------+----------+--------------------------------------+
  1. 使用kolla-ansible deploy部署tempest
1
2
3
4
root@control01 ~]# kolla-ansible deploy -t tempest
## 看到tempest容器运行部署就算成功
root@control01 ~]# docker ps | grep tempest
95b57e77b006 kolla/centos-source-tempest:5.0.0 "kolla_start" 46 hours ago Up 21 hours

Tempest使用

准备工作

了解tempest cmd基本使用

官网文档

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
(tempest)[root@control01 tempest-16.0.1.dev380]# tempest --help
usage: tempest [--version] [-v | -q] [--log-file LOG_FILE] [-h] [--debug]
Tempest cli application
optional arguments:
--version show program's version number and exit
-v, --verbose Increase verbosity of output. Can be repeated.
-q, --quiet Suppress output except warnings and errors.
--log-file LOG_FILE Specify a file to log output. Disabled by default.
-h, --help Show help message and exit.
--debug Show tracebacks on errors.
Commands:
account-generator Create accounts.yaml file for concurrent test runs.
cleanup Cleanup after tempest run
complete print bash completion command
help print detailed help for another command
init Setup a local working environment for running tempest
list-plugins List all tempest plugins
run Run tempest
verify-config Verify your current tempest configuration
workspace list Outputs the name and path of all known tempest workspaces
workspace move Changes the path of a given tempest workspace --name to --path
workspace register Registers a new tempest workspace via a given --name and --path
workspace remove Deletes the entry for a given tempest workspace --name
workspace rename Renames a tempest workspace from --old-name to --new-name

常用的命令有

1
2
3
4
5
6
7
8
# 为tempest初始化working environment
tempest init
# 验证配置是否正确
tempest verify-config
# 执行tempest测试
tempest run
配置tempest.conf

下面是kolla tempest.conf的模版

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
[DEFAULT]
debug = {{ openstack_logging_debug }}
log_file = tempest.log
use_stderr = False
log_dir = /var/log/kolla/tempest/
[auth]
admin_username = {{ openstack_auth.username }}
admin_password = {{ keystone_admin_password }}
admin_project_name = {{ openstack_auth.project_name }}
admin_domain_name = {{ openstack_auth.domain_name }}
[dashboard]
dashboard_url = {{ internal_protocol }}://{{ kolla_internal_fqdn }}
login_url = {{ internal_protocol }}://{{ kolla_internal_fqdn }}/auth/login/
[service_available]
cinder = {{ enable_cinder }}
neutron = {{ enable_neutron }}
glance = {{ enable_glance }}
swift = {{ enable_swift }}
nova = {{ enable_nova }}
heat = {{ enable_heat }}
horizon = {{ enable_horizon }}
ceilometer = {{ enable_ceilometer }}
[compute]
max_microversion = latest
image_ref = {{ tempest_image_id }}
image_ref_alt = {{ tempest_image_alt_id }}
flavor_ref = {{ tempest_flavor_ref_id }}
flavor_ref_alt = {{ tempest_flavor_ref_alt_id }}
region = {{ openstack_region_name }}
[dashboard]
dashboard_url = {{ internal_protocol }}://{{ kolla_internal_fqdn }}/
login_url = {{ internal_protocol }}://{{ kolla_internal_fqdn }}/auth/login
[identity]
region = {{ openstack_region_name }}
auth_version = v3
uri = {{ internal_protocol }}://{{ kolla_internal_fqdn }}:{{ keystone_admin_port }}/v2.0
uri_v3 = {{ internal_protocol }}://{{ kolla_internal_fqdn }}:{{ keystone_admin_port }}/v3
[image]
region = {{ openstack_region_name }}
http_image = {{ image_url }}
[network]
region = {{ openstack_region_name }}
public_network_id = {{ tempest_public_network_id }}
floating_network_name = {{ tempest_floating_network_name }}
project_networks_reachable = false
[network-feature-enabled]
ipv6 = false
[object-storage]
region = {{ openstack_region_name }}
[orchestration]
region = {{ openstack_region_name }}
[volume]
region = {{ openstack_region_name }}
[volume-feature-enabled]
api_v1 = False
[validation]
image_ssh_user = {{ tempest_image_ssh_user }}
image_ssh_password = {{ tempest_image_ssh_password }}

使用tempest测试api接口

明确需求

  • 指定某些api进行测试;例如我只想测试 compute的相关api
  • 执行所有的测试
  • 单项api测试
  • 去掉某些我不想测试的api
执行所有测试
1
2
3
4
5
6
7
8
# 验证你的配置是否正确
tempest verify-config
# 执行所有测试
tempest run --subunit > result.subunit
# 生成测试报告result.html
subunit2html result.subunit
黑名单–blacklist-file的使用
1
2
3
4
5
--blacklist-file <file name>
Sample regex file:
(^tempest\.api) # Comments about this regex
tempest.scenario.test_server_basic_ops # Matches this test explicitly

例如除了tempest.scenario不测,其他的都测

1
2
3
4
(tempest)[root@control01 tempest-16.0.1.dev380]# cat blacklist
tempest.scenario.*
tempest run --blacklist-file blacklist
白名单–whitelist-file的使用
1
2
3
4
5
6
7
8
9
10
11
12
# 只测试tempest.api.compute
(tempest)[root@control01 tempest-16.0.1.dev380]# cat whitefile
tempest.api.compute.*
tempest run --whitelist-file whitefile
# 测试一个api接口
(tempest)[root@control01 tempest-16.0.1.dev380]# cat one_whitefile
tempest.api.compute.images.test_list_image_filters.ListImageFiltersTestJSON.test_list_images_filter_by_changes_since
tempest run --whitelist-file one_whitefile
使用ipdb调试tempest api

安装ipdb

1
pip install ipdb

在程序需要中断的地方插入

1
import ipdb; ipdb.set_trace()

运行程序后, 会在执行到set_trace()的时候中断程序 并出现提示符

(ipdb) …

这时输入help即可看到ipdb下常用的命令啦

比较常用的是看看当前的变量 a 以及下一步 n

还有就是 dir() 方法 可以查看一个对象有那些方法可以调用

ipdb比pdb的强大在于 他包含啦 ipython 特性. 可以支持tab补全

下面ipdb调试tempest api接口的一个例子

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
vim tempest/api/compute/images/test_list_image_filters.py
@decorators.idempotent_id('a3f5b513-aeb3-42a9-b18e-f091ef73254d')
def test_list_images_filter_by_status(self):
# The list of images should contain only images with the
# provided status
import ipdb; ipdb.set_trace()
params = {'status': 'ACTIVE'}
images = self.client.list_images(**params)['images']
self.assertNotEmpty([i for i in images if i['id'] == self.image1_id])
self.assertNotEmpty([i for i in images if i['id'] == self.image2_id])
self.assertNotEmpty([i for i in images if i['id'] == self.image3_id])
## 进行调试
(tempest)[root@control01 tempest-16.0.1.dev380]# python -m testtools.run tempest.api.compute.images.test_list_image_filters.ListImageFiltersTestJSON.test_list_images_filter_by_changes_since
Tests running...
/var/lib/kolla/venv/lib/python2.7/site-packages/IPython/core/debugger.py:243: DeprecationWarning: The `color_scheme` argument is deprecated since version 5.1
DeprecationWarning)
> /tempest-source/tempest-16.0.1.dev380/tempest/api/compute/images/test_list_image_filters.py(100)resource_setup()
99 # Create instances and snapshots via nova
--> 100 cls.server1 = cls.create_test_server()
101
ipdb>
测试脚本

下面是一个简单的测试脚本,测试基本都是通过的,在控制节点执行这个脚本就ok;

报告生成在 /tmp/tempest目录下。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
[root@control01 kolla-test]# cat run_tempest.sh
#!bin/bash
TEMPEST_PATH="/tmp/tempest"
TEMPEST_FILE="/tmp/tempest/results.html"
if [ ! -d "$TEMPEST_PATH" ]; then
mkdir "$TEMPEST_PATH"
fi
if [ -f "$TEMPEST_FILE" ]; then
rm -f "$TEMPEST_FILE"
fi
cat << EOF > "blacklist"
tempest.api.compute.images.test_list_image_filters.ListImageFiltersTestJSON
tempest.api.network.admin.test_metering_extensions.MeteringTestJSON
tempest.api.network.test_extensions.ExtensionsTestJSON
tempest.api.volume.test_volume_delete_cascade.VolumesDeleteCascade
tempest.api.compute.admin.test_auto_allocate_network.AutoAllocateNetworkTest
tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON
tempest.api.compute.admin.test_server_diagnostics.ServerDiagnosticsV248Test
tempest.api.compute.images.test_images.ImagesTestJSON
tempest.api.compute.servers.test_delete_server.DeleteServersTestJSON
tempest.api.compute.volumes.test_attach_volume.AttachVolumeShelveTestJSON
tempest.api.compute.images.test_images_oneserver.ImagesOneServerTestJSON
tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON
tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON
tempest.api.volume.admin.test_volume_types.VolumeTypesTest
tempest.api.volume.admin.test_volumes_backup.VolumesBackupsAdminTest
tempest.api.volume.test_volume_delete_cascade.VolumesDeleteCascade
tempest.api.volume.test_volumes_backup.VolumesBackupsTest
tempest.api.volume.test_volumes_snapshots.VolumesSnapshotTestJSON
tempest.scenario.test_minimum_basic.TestMinimumBasicScenario
tempest.scenario.test_shelve_instance.TestShelveInstance
tempest.scenario.test_snapshot_pattern.TestSnapshotPattern
tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern
tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps
tempest.scenario.test_encrypted_cinder_volumes.TestEncryptedCinderVolumes
EOF
docker cp blacklist tempest:/tempest/blacklist
docker exec tempest bash -c 'tempest run --config-file /etc/tempest/tempest.conf --blacklist-file /tempest/blacklist --subunit > /tmp/results.subunit'
docker exec tempest bash -c 'subunit2html /tmp/results.subunit /tmp/results.html'
docker cp tempest:/tmp/results.html $TEMPEST_FIL
坚持原创技术分享,您的支持将鼓励我继续创作!