| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-04-06 | |||
| 10:51:41 | bauzas | johnthetubaguy: stephenfin: that would solidly refrain the need for pushing new filters in-tree and would at least require a spec, which I think is always good | |
| 10:52:10 | bauzas | and since people can provide their own out-of-tree filters and do what they want, they wouldn't be hit | |
| 10:52:25 | bauzas | I like that plan actually | |
| 10:54:39 | gibi | bauzas: replied in https://review.opendev.org/#/c/708436/ | |
| 10:55:33 | bauzas | gibi: cool, we're on the same page | |
| 10:56:20 | bauzas | I'd appreciate johnthetubaguy to agree on https://review.opendev.org/#/c/708436/16//COMMIT_MSG@12 | |
| 10:56:41 | bauzas | but if he's ok, I'll change my vote to +2 (at least once I'm fully done with reviewing) | |
| 10:57:58 | openstackgerrit | Stephen Finucane proposed openstack/nova master: api: Add framework for extra spec validation https://review.opendev.org/704643 | |
| 10:58:01 | stephenfin | bauzas: ^ | |
| 11:01:04 | bauzas | stephenfin: +W | |
| 11:01:26 | bauzas | stephenfin: but you need to rebase the whole tree now | |
| 11:01:51 | bauzas | (sorry, if it was other thing but a commit msg, I would have proposed a FUP) | |
| 11:02:14 | stephenfin | I'm not sure if I do or if Gerrit will do it for me, but I'm reworking the doc patch so can do so shortly if needed | |
| 11:02:27 | stephenfin | I'll wait to see if there are review comments first | |
| 11:11:27 | ierdem | Hello guys, i have a question about resizing a running VM. I tried to increase vCPU counts of a running VM and after that i connected succesfully but it is too slow to run anything. When i see process list via "top" command, i saw CPU usage is approximately 1000 percent and it decrease sometimes but increase again. Any suggestions please? | |
| 11:12:30 | stephenfin | bauzas: out out? :O | |
| 11:12:34 | stephenfin | :P | |
| 11:12:46 | bauzas | out from my room :p | |
| 11:13:05 | bauzas | Who Let the Dogs out out ? | |
| 11:13:44 | bauzas | https://www.youtube.com/watch?v=Qkuu0Lwb5EM | |
| 11:15:50 | sean-k-mooney | well regardign fancy new keys. i would rather just define a custom: name spaces that should be used for user defined extra_specs and declare all other namespaced keys as owned by nova | |
| 11:16:16 | sean-k-mooney | and if a new intree filter wants to add keys it gets its own namespace | |
| 11:16:44 | sean-k-mooney | if our of tree filter need to define keys the do it via custom: | |
| 11:17:11 | sean-k-mooney | ideally with a prefix e.g. custom:myfilter_mykey | |
| 11:30:14 | sean-k-mooney | ierdem: i assume you have oversubsrtion enabled on your cloud? | |
| 11:30:25 | sean-k-mooney | ierdem: is the host over subsibed | |
| 11:31:53 | sean-k-mooney | ierdem: it should like you either have someing in the vm that is broken/maliusly consumeing cpu, the vm workload need more vcpus or the host is over subsribed | |
| 11:32:54 | sean-k-mooney | ierdem: simply seeing high cpu usage in the guest is not an indication that something is wrong form a nova point of view | |
| 11:33:36 | ierdem | hmm, how can i check if the host is oversubscribed? | |
| 11:34:32 | ierdem | by the way, before resizing vCPU count, it was working fine | |
| 11:35:00 | kplant | a resize can move the instance to another hypervisor | |
| 11:36:41 | kplant | if you just do a 'nova hypervisor-show <uuid>' | |
| 11:36:49 | kplant | you can check vcpus vs vcpus_used | |
| 11:36:58 | kplant | if vcpus_used > vcpus; you're over subscribed | |
| 11:37:38 | ierdem | ok, thanks i will try and return to you | |
| 11:38:12 | kplant | also vcpus_used == vcpus is a bad idea, unless nova knows about vcpus you're reserving | |
| 11:38:53 | sean-k-mooney | yes by default resize to same host is disabled so it normally does a move and will only stay on the same host if the weigher consider it to be the best host | |
| 11:39:13 | sean-k-mooney | if you enable same host resize in the config | |
| 11:40:28 | sean-k-mooney | kplant: if you use vcpu_pin_set or (cpu_shared_set and cpu_dedicated_set) then yes vcpu_used == vcpu shared is fine | |
| 11:40:59 | kplant | yeah, if not ideal :-) | |
| 11:41:18 | sean-k-mooney | ideally you should sue the *_sets for doing host reservation instead fo the reserved_host_cpus | |
| 11:41:25 | sean-k-mooney | *use | |
| 11:42:18 | sean-k-mooney | if you use the set you can choose which cpus to reserve and then you can use systemd to run the host process only on thos cores | |
| 11:42:34 | kplant | you can go further and use the bootloader | |
| 11:42:56 | sean-k-mooney | using isolcpus? is so you should really avoid that | |
| 11:43:07 | kplant | my only complaint with *_sets is you can't have open ended ranges | |
| 11:43:13 | kplant | like "3-" | |
| 11:43:24 | kplant | makes it easier to deal with hosts with, not vastly, different configs | |
| 11:43:40 | sean-k-mooney | you can do negations | |
| 11:43:51 | sean-k-mooney | "^0-2" | |
| 11:43:59 | kplant | ooo | |
| 11:44:10 | sean-k-mooney | i think negated ranges works | |
| 11:44:18 | sean-k-mooney | you can negate indivcual cores | |
| 11:45:01 | sean-k-mooney | we prably could add open ended ranges too you could always file a bug/blueprint | |
| 11:45:27 | kplant | i think negated ranges provide the same result | |
| 11:45:38 | kplant | nice | |
| 11:46:03 | sean-k-mooney | ill triple check the code to make sure that works | |
| 11:46:16 | sean-k-mooney | we support it for cpu_realtime_mask | |
| 11:46:27 | sean-k-mooney | but i think its supported in teh *_sets too | |
| 11:47:59 | sean-k-mooney | kplant: this is the code and it inclde the negation example for 1 core | |
| 11:48:20 | openstackgerrit | Balazs Gibizer proposed openstack/nova-specs master: template: consider openstack client besides novaclient https://review.opendev.org/717722 | |
| 11:48:46 | sean-k-mooney | kplant: yep negated ranges should work https://github.com/openstack/nova/blob/31aa4a6d7f0b8c301b093ad176ee2b44b5d3cec8/nova/virt/hardware.py#L134-L140 | |
| 11:50:05 | sean-k-mooney | sorry miss read that | |
| 11:50:53 | sean-k-mooney | the negation only works with 1 cpu | |
| 11:51:20 | kplant | looking at 113 -> 116 | |
| 11:51:26 | kplant | i think you might have been right originally | |
| 11:51:36 | kplant | it tests for ^str == '^' | |
| 11:51:39 | kplant | then proceeds as normal | |
| 11:51:47 | kplant | the next try/catch is for a range | |
| 11:52:38 | kplant | oof, that's ignoring what you highlighted though | |
| 11:53:32 | sean-k-mooney | https://github.com/openstack/nova/blob/824bc358c25564ed6603d8f94587abd8902fe5af/nova/tests/unit/virt/test_hardware.py#L132-L133 | |
| 11:53:38 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: doc: require openstack client change for every new API microversion https://review.opendev.org/717727 | |
| 11:53:41 | sean-k-mooney | unit tests are awsome :) | |
| 11:53:41 | kplant | that's still not a show stopper | |
| 11:53:48 | sean-k-mooney | although docs would also help | |
| 11:54:01 | kplant | as long as the _set string is _only_ negations, every other vcpu should implicitly be available for scheduling, no? | |
| 11:54:52 | sean-k-mooney | that is the behavior i would want as an enduser | |
| 11:56:26 | sean-k-mooney | but no unfortunetly that is not what the behvior is today | |
| 11:56:30 | kplant | i probably just over thought it and gave myself a reason to be lazy and use reserved_host_cpus instead of cpu_dedicated_set | |
| 11:56:41 | kplant | oh, really? | |
| 11:57:00 | openstackgerrit | Merged openstack/nova master: Add info about affinity requests to the troubleshooting doc https://review.opendev.org/715092 | |
| 11:57:01 | sean-k-mooney | it starts with an empty set cpuset_ids = set() | |
| 11:57:08 | openstackgerrit | Merged openstack/nova master: Stabilize functional tests https://review.opendev.org/717070 | |
| 11:57:09 | kplant | eek | |
| 11:57:15 | openstackgerrit | Merged openstack/nova master: Introduce scope_types in security groups policy https://review.opendev.org/716786 | |
| 11:57:16 | sean-k-mooney | and after we loop over every thing cpuset_ids -= cpuset_reject_ids | |
| 11:57:39 | sean-k-mooney | it would not be hard to add that behavior. | |
| 11:58:02 | kplant | i would expect assuming all cpus to actually be faster code | |
| 11:58:13 | kplant | that's more likely in line with what the result will be | |
| 11:58:37 | sean-k-mooney | kplant: that code does not know how many cpus you have | |
| 11:59:44 | kplant | fair, it would come from sql | |
| 12:00:45 | sean-k-mooney | not quite where this is used is on the comptue node before we store the info in the db but anyway if you do have a propoasl for improving this feel free to write it up | |
| 12:02:09 | kplant | is this the better behavior? the current behavior forces the user to be explicit | |
| 12:02:39 | kplant | i guess i can always submit it and get input that way | |
| 12:06:40 | sean-k-mooney | kplant: if you dont set the config options all cores are assumed to be usable | |
| 12:06:59 | sean-k-mooney | kplant: so the idea was of you opt in you should say what you want | |
| 12:08:35 | kplant | i think that mindset still applies, just change the behavior from cpu_*_set to a merge behavior instead of replace | |
| 12:08:43 | kplant | i think that's reasonable | |
| 12:08:47 | sean-k-mooney | kplant: that said i have wanted to remove the reserved_host_cpus options since we first added vcpu_pin_set so makeing it nicer to use the reserved_host_cpus will help with that goal | |
| 12:08:49 | kplant | just want to make sure before i waste time with a bp | |