| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-12-15 | |||
| 15:25:47 | leakypipes | andreykurilin_: ty for changing the subject :) | |
| 15:25:58 | andreykurilin_ | ha | |
| 15:27:07 | andreykurilin_ | leakypipes: sorry, but such topic cannot be changed so quickly. Do you have any photos from your boarding school ? heh | |
| 15:27:59 | cdent | leakypipes: was that brit, yankee, or something else boarding school? | |
| 15:28:00 | leakypipes | andreykurilin_: yes :) you can see every room I lived in over four years in the movie: http://www.imdb.com/title/tt0097165/ | |
| 15:28:12 | leakypipes | cdent: yank | |
| 15:28:39 | cdent | thank you, that helps to distinguish the style of horrible abuse you experience | |
| 15:28:59 | leakypipes | heh | |
| 15:31:41 | leakypipes | andreykurilin_: btw, I'm not kidding. that really is the school I went to. | |
| 15:31:57 | leakypipes | andreykurilin_: it's not called the same name in the movie, but that's the school. | |
| 15:32:23 | leakypipes | andreykurilin_: in fact, three of my teachers from high school play parts in the movie :) | |
| 15:32:33 | leakypipes | I arrived the year after it was released. | |
| 15:33:20 | cfriesen | when shelving, do we preserve the contents of the ephemeral disk(s)? the code looks like it only snapshots the main image, but ephemeral disks are supposed to last the life of the server. | |
| 15:33:46 | mriedem | andreykurilin_: comments / questions in https://review.openstack.org/#/c/424256/ but looks pretty good | |
| 15:34:45 | mriedem | cfriesen: i doubt it | |
| 15:34:49 | cdent | aw, leakypipes, you are a youth | |
| 15:34:54 | mriedem | we snapshot the image and then delete the guest | |
| 15:34:56 | leakypipes | heh | |
| 15:35:23 | mriedem | andreykurilin_: go ahead on the hooks thing, i didn't dig into that very deep, just noticed it while doing something else | |
| 15:42:44 | andreykurilin_ | leakypipes: wow. cool | |
| 15:46:56 | leakypipes | figleaf: you can thank mriedem for that. | |
| 15:47:29 | mriedem | maciejjozefczyk: giblet: questions inline https://review.openstack.org/#/c/520024/ | |
| 15:47:36 | mriedem | and you probably want leakypipes to look at ^ as well | |
| 15:48:16 | vivsoni__ | mriedem: Hi | |
| 15:48:49 | mriedem | hi | |
| 15:49:08 | maciejjozefczyk | mriedem: ok | |
| 15:50:14 | vivsoni__ | mriedem: the patch proposed by you to fix for tempest issue(connector empty to terminate_connection) | |
| 15:51:01 | yyangs | Hi | |
| 15:51:02 | mriedem | vivsoni__: yeah? | |
| 15:51:18 | vivsoni__ | mridem: is restored right ? | |
| 15:51:21 | mriedem | yes | |
| 15:51:24 | vivsoni__ | ok | |
| 15:51:37 | vivsoni__ | so will wait for smcginnis for review then | |
| 15:51:46 | mriedem | there were some concerns about it potentially breaking force volume detach, but i think i've handled that in the latest revision | |
| 15:52:01 | vivsoni__ | yes i have seen that | |
| 15:52:25 | mriedem | i am, however, no expert on the delicate intricacies of the cinder volume manager | |
| 15:52:38 | mriedem | plus, i just wanted to say "delicate intricacies" | |
| 15:53:30 | vivsoni__ | in case if Sean disagree, then i need to handle that in our code | |
| 15:54:00 | vivsoni__ | by just returning 'None' in case of empty connector ... right ? | |
| 15:54:58 | yyangs | we have some noisy neighbours and we are trying to use cpu_quota and cpu_period on flavor to limit the cpu cycles by those vms. It is fine in single vcpu servers. But for multi vcpus server, we are a little confused about the cpu_quota. | |
| 15:56:02 | yyangs | In our tests, we set cpu_period to 200ms, for 2 vcpus server, if we want to allow 50% cpu, should we set to 200ms? or 100ms? | |
| 15:57:02 | leakypipes | yyangs: are you referring to Nova or Kubernetes? | |
| 15:57:14 | yyangs | nova | |
| 15:57:42 | yyangs | leakypipes: we are using kvm | |
| 15:58:24 | yyangs | leakypipes: I though nova also relies on cgroup, right? | |
| 15:58:31 | leakypipes | yyangs: no. | |
| 15:59:02 | leakypipes | yyangs: well, maybe libvirt is configuring cgroups for the VM behind the scene, yeah | |
| 15:59:07 | smcginnis | vivsoni__: Sorry, distracted with some other things. But you still need to handle it in your code, even after mriedem's delicate bits. | |
| 15:59:45 | vivsoni__ | smcginnis: ok | |
| 16:00:05 | leakypipes | yyangs: I'm afraid that feature wasn't particularly complete... I'm pretty sure stuff in https://wiki.openstack.org/wiki/InstanceResourceQuota doesn't work. | |
| 16:00:21 | vivsoni__ | smcginnis: just wanted to confirm are you OK with mridem's fix ? | |
| 16:00:44 | yyangs | leakypipes: we did a couple of tests on cpu | |
| 16:01:00 | smcginnis | vivsoni__: I need to look again, but last update looked right. Doesn't change the need on the driver to handle valid cases where the connector is not passed in. | |
| 16:01:26 | yyangs | leakypipes: we have fixed cpu_period = 200ms, For single vcpus, we can see cpu_quota is actually applied properly | |
| 16:01:57 | yyangs | leakypipes: when cpu_quota 100ms, it only consumes half of cpu cycles | |
| 16:03:04 | yyangs | leakypipes: the question is on multi core, based on cgroups docs, say 2 vcpus 50% cpu cycles, seems cpu_period = 200ms, cpu_quota = 2 * 100=200ms | |
| 16:03:06 | leakypipes | gotta love 5-line patches that add CPU share functionality that have zero tests in them. :( https://review.openstack.org/#/c/22106/4 | |
| 16:03:36 | vivsoni__ | smcginnis: ok | |
| 16:04:05 | leakypipes | yyangs: I'm trying to tell you that this "feature" is untested and probably does not work. | |
| 16:04:19 | yyangs | leakypipes: I see | |
| 16:05:26 | leakypipes | yyangs: I don't see any evidence from the patch I linked above that this "feature" worked on VMs with >1 VCPU | |
| 16:06:25 | mriedem | http://git.openstack.org/cgit/openstack/nova/tree/doc/source/user/flavors.rst#n140 | |
| 16:06:30 | mriedem | it's in the docs, but nothing in the nova code uses this | |
| 16:06:37 | mriedem | so probably an incomplete blueprint | |
| 16:06:42 | leakypipes | yyangs: here is what I would recommend... | |
| 16:07:23 | leakypipes | yyangs: spin up a VM with the number of full VCPU you wish to manage, then set cgroups within that VM manually to control smaller slices of resource (containers, processes, whatever you want). | |
| 16:08:13 | yyangs | leakypipes: e... ok | |
| 16:08:20 | mriedem | test_get_guest_config_with_cpu_quota claims to test thsi | |
| 16:08:21 | mriedem | *this | |
| 16:08:28 | leakypipes | yyangs: I know it's not the answer you were looking for, sorry :( | |
| 16:08:32 | mriedem | but i'm not sure where it's getting the values from in the libvirt driver | |
| 16:09:11 | yyangs | leakypipes: well, when we were doing 2 vcpus test, it actually did give us something. I mean it did limit the cpu cycle | |
| 16:09:31 | mriedem | ah here we go | |
| 16:09:35 | mriedem | see _update_guest_cputune in the libvirt driver | |
| 16:10:00 | mriedem | looks like it depends on your kernel config | |
| 16:10:18 | leakypipes | yyangs: you are trying to do something like 50% of 2 CPUs? | |
| 16:10:35 | yyangs | leakypipes: yest | |
| 16:10:42 | mriedem | http://git.openstack.org/cgit/openstack/nova/tree/nova/virt/libvirt/driver.py#n3938 | |
| 16:10:48 | mriedem | ^ is the method that handles these extra specs | |
| 16:10:50 | openstackgerrit | Merged openstack/nova stable/newton: Fix 'force' parameter in os-quota-sets PUT schema https://review.openstack.org/526427 | |
| 16:10:54 | yyangs | mriedem: i think our kernel has proper config | |
| 16:10:55 | openstackgerrit | Merged openstack/nova stable/newton: Get original image_id from volume for volume-backed instance rebuild https://review.openstack.org/526544 | |
| 16:10:57 | mriedem | if the kernel doesn't handle it, there should be an error | |
| 16:11:01 | openstackgerrit | Merged openstack/nova stable/pike: propagate OSError to MigrationPreCheckError https://review.openstack.org/515330 | |
| 16:11:23 | mriedem | does the guest domain xml have the cputune values in it? | |
| 16:11:27 | yyangs | mriedem: we do see the cputune blob in xml, | |
| 16:12:09 | mriedem | https://libvirt.org/formatdomain.html#elementsCPUTuning | |
| 16:13:07 | mriedem | looks like this has all been around in qemu for a long time | |
| 16:13:13 | mriedem | why it wouldn't be working, not sure | |
| 16:13:16 | mriedem | maybe ask kashyap | |
| 16:14:27 | yyangs | mriedem: we didnt do the cpu pinning, but we do see shares, period and quota in cputune blob | |
| 16:14:35 | leakypipes | mriedem, yyangs: it's not a QEMU/libvirt problem. it's a Nova problem... | |
| 16:14:55 | leakypipes | mriedem, yyangs: it's erroneously setting cpu_shares to 1024 * number of vCPUs: | |
| 16:14:56 | leakypipes | guest.cputune.shares = 1024 * guest.vcpus | |
| 16:15:39 | giblet | mriedem: tried to answer your questions in https://review.openstack.org/#/c/520024/ but I agree to involve leakypipes | |
| 16:15:49 | leakypipes | mriedem, yyangs: shouldn't that be setting CPU share for a single vCPU/qemu-kvm process? | |
| 16:16:38 | yyangs | leakypipes: we didn't try cpu share actually. just tweak cpu_quota with fixed cpu_period. | |
| 16:17:03 | leakypipes | yyangs: unless you specify cpu_shares, it will default to 1024 * VCPUs | |
| 16:17:11 | leakypipes | yyangs: http://git.openstack.org/cgit/openstack/nova/tree/nova/virt/libvirt/driver.py#n3955 | |
| 16:17:15 | openstackgerrit | Andrey Kurilin proposed openstack/python-novaclient master: [CLI] Fix token auth type https://review.openstack.org/424256 | |