| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-03-12 | |||
| 13:57:39 | sean-k-mooney | # List of dicts | |
| 13:57:41 | sean-k-mooney | - identification: | |
| 13:57:43 | sean-k-mooney | uuid: $COMPUTE_NODE | |
| 13:57:45 | sean-k-mooney | inventories: | |
| 13:57:47 | sean-k-mooney | additional: | |
| 13:57:49 | sean-k-mooney | VCPU: | |
| 13:57:51 | sean-k-mooney | max_unit: 32 | |
| 13:59:04 | belmoreira | If we can do that in Ussuri it looks good enough to me for any special case | |
| 13:59:23 | belmoreira | thanks sean-k-mooney | |
| 13:59:25 | sean-k-mooney | actully it wont work for your usecase | |
| 13:59:41 | sean-k-mooney | i forgot we can only use this for CUSTOM_ resouces | |
| 14:00:09 | sean-k-mooney | we intentionally blocked overwriding sthe standard ones form the virt driver | |
| 14:05:25 | belmoreira | my thinking on all of this is how an operator should move forward when (for whatever reason) the resource inventory changes. | |
| 14:06:12 | sean-k-mooney | well form a downstream and i thnk upstream point of vew we do not supprot changing the number of hyper treads on a host with vms | |
| 14:06:13 | belmoreira | For my particular case, vCPUs were cut in half, but we need to keep the oversubcribed instances running. Now to live migrate then we need "patch" the compute nodes | |
| 14:06:40 | sean-k-mooney | the same aslo gos for numa nodes e.g. by enable cluster on die ot changeing numa per socket in the bios | |
| 14:07:00 | sean-k-mooney | so today the only supported way it to do a resize to a differnt flavor | |
| 14:07:46 | sean-k-mooney | belmoreira: yep i understand unfortuetly the senario your are attempting to do is not currenlty supported by nova | |
| 14:07:52 | belmoreira | sean-k-mooney :) fair enough. I didn't think we would ever disable SMT in production nodes. | |
| 14:08:34 | sean-k-mooney | the "keep vms alive requirement is really the toughest" | |
| 14:09:16 | sean-k-mooney | belmoreira: for live migration we cant really change the toplogy of the guest | |
| 14:10:07 | sean-k-mooney | we can change teh mapping the host but putting that 32 core vm on a 16 core host would not be a valid schudlign decision based on our current oversubscpiton rules | |
| 14:10:33 | sean-k-mooney | belmoreira: actully i hate to say this but did you attempt a force live migration | |
| 14:10:57 | sean-k-mooney | i assume that still fails because placment will block it | |
| 14:11:11 | sean-k-mooney | when we try to update the allcoations | |
| 14:11:21 | belmoreira | sean-k-mooney it's fine. And in most part I agree with you, I just wanted to raise this issue here because other operators may suffer from the same thing. And these are the "use cases" that we didn't think about... Like I said I never thought we would need to disable SMT in production nodes. | |
| 14:12:01 | sean-k-mooney | it is certenly something we could put in the docs somewhwere | |
| 14:12:09 | belmoreira | sean-k-mooney actually I didn't... but in the case is placement, so it should fail | |
| 14:12:55 | sean-k-mooney | i think the allocation update will fail but i have never tried this | |
| 14:13:16 | sean-k-mooney | nova with the old microversion will skip the schduler fileter if you force it | |
| 14:13:27 | sean-k-mooney | but i think we always do the placment update | |
| 14:14:19 | sean-k-mooney | belmoreira: if you want to hack around it without changing the code there is one thing you could try | |
| 14:14:49 | sean-k-mooney | you could increase the interval taht we update placment at in the config on a node temporally so say once an hour | |
| 14:15:09 | sean-k-mooney | and you could manully chagne the value in the placment inventory with osc-placment | |
| 14:15:18 | sean-k-mooney | then migrate | |
| 14:15:48 | sean-k-mooney | that will end up with an invalidly placed vm but you could force it that way | |
| 14:17:10 | belmoreira | sean-k-mooney yeah, that would work. What i'm doing is just putting a fake value for max_int in the update placement. These nodes will be removed anyway. | |
| 14:17:52 | sean-k-mooney | ya its basiclaly the same just via the api for those that cant hack the code direcly in prodcution | |
| 14:18:33 | belmoreira | sean-k-mooney thanks a lot for your comments. | |
| 14:18:49 | belmoreira | Having something in the docs may help others. I think I can move this bug forward and update the docs. | |
| 14:19:39 | sean-k-mooney | i hope you dont mind that i marked it as invalid but you could bring it up at the ptg | |
| 14:19:59 | sean-k-mooney | or in a nova team meeting/mailing list to get more input form others | |
| 14:20:53 | belmoreira | sean-k-mooney I think having something in the docs is reasonable | |
| 14:21:21 | sean-k-mooney | yep i agree if you wanted to convert that to a docs bug i think it would be good | |
| 14:24:08 | belmoreira | I need some guidance... should I mentioned in the bug that we discussed this and we agreed that a docs change could be enough? or this bug needs to be submitted somewhere else? | |
| 14:24:33 | sean-k-mooney | you can link to the irc convo | |
| 14:25:04 | sean-k-mooney | http://eavesdrop.openstack.org/irclogs/%23openstack-nova/latest.log.html#t2021-03-12T13:47:17 | |
| 14:25:34 | sean-k-mooney | then we can triage it as valid | |
| 14:25:49 | belmoreira | ok thanks a lot | |
| 14:26:16 | sean-k-mooney | so you can reuse the same bug if you like althotuhg you might want to also update the titile to refelct its a docs change now | |
| 14:28:41 | belmoreira | will do | |
| 14:32:54 | sean-k-mooney | stephenfin: im going to put my func test for the api block in a different class. they way you are currently mocking the vdpa devices is more complex then i would like and since i dont actully need them since im blocking these ops at the api level im going to skip mocking them for now | |
| 14:33:24 | sean-k-mooney | stephenfin: when the op moves form unsupported to supported ill need the vdpa devices mocked but for these negitive test i dont | |
| 14:33:31 | sean-k-mooney | that sound ok to you | |
| 14:35:28 | sean-k-mooney | actully i might need to mock some fo them they way you are but ill cross that bridge when i come to it | |
| 14:36:03 | sean-k-mooney | i guess i could do api unit tests instead of functional too. | |
| 14:37:17 | stephenfin | sean-k-mooney: go for it | |
| 14:37:33 | stephenfin | though I take offence to the suggestion it's too complex :-P | |
| 14:37:52 | stephenfin | (tbf, I actually thought it was reasonably understandable) | |
| 14:38:34 | sean-k-mooney | well i cant just do fakelibvirt.HostPCIDevicesInfo(num_pfs=1, num_vf=0, num_vdpa=2) | |
| 14:38:42 | sean-k-mooney | like i made work in the unit tests | |
| 14:38:59 | sean-k-mooney | the fact you are manually creating the devices in the test is why is complex | |
| 14:39:13 | stephenfin | oh, well it's easy to add that | |
| 14:39:33 | stephenfin | I just didn't need it since I'd a single test | |
| 14:39:41 | sean-k-mooney | yep that the only thing i was unhappy with really | |
| 14:41:31 | sean-k-mooney | well i didnt review them too closely so there mihgt be something else but that made me go ill look at this when i have more time | |
| 15:04:12 | openstackgerrit | Merged openstack/nova stable/train: [stable-only] gate: Pin CEPH_RELEASE to nautilus in LM hook https://review.opendev.org/c/openstack/nova/+/780014 | |
| 15:17:20 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/stein: [stable-only] gate: Pin CEPH_RELEASE to nautilus in LM hook https://review.opendev.org/c/openstack/nova/+/780277 | |
| 15:17:52 | elod | gibi lyarwood : i've uploaded the release patches: https://review.opendev.org/q/project:openstack/releases+status:open+intopic:nova | |
| 15:18:06 | gibi | will check soon | |
| 15:19:09 | elod | thx o/ let me know if you disagree with the minor version bump | |
| 15:33:36 | lbragstad | dansmith opened up a bug against nova to document what we found https://bugs.launchpad.net/nova/+bug/1918945 | |
| 15:33:38 | openstack | Launchpad bug 1918945 in OpenStack Compute (nova) "Nova API fails with 500s when called with non-project-scoped keystone tokens" [Undecided,New] | |
| 15:33:47 | dansmith | lbragstad: sweet | |
| 15:37:09 | sean-k-mooney | lbragstad: that is kind fo expected depending on what you are calling | |
| 15:37:21 | sean-k-mooney | lbragstad: server create for example should not work wit | |
| 15:37:31 | sean-k-mooney | domain or system tokens | |
| 15:37:35 | lbragstad | sean-k-mooney yeah - i don't expect it to work | |
| 15:37:40 | lbragstad | i just didn't expect a 500 | |
| 15:37:55 | dansmith | sean-k-mooney: it's failing in object field validation, | |
| 15:37:58 | sean-k-mooney | it should be a 400 or 403 | |
| 15:37:59 | dansmith | which is...not the right place :) | |
| 15:38:13 | sean-k-mooney | yep agreed | |
| 15:38:39 | sean-k-mooney | we should be handeling it in the policy check level or similar in the api | |
| 15:39:04 | dansmith | yeah I think we probably need a decorator on anything that might create resources to @require_project or something | |
| 15:39:25 | dansmith | because I think even with something like instance shutdown, we might go to write an instance action record and fail if we have no project :( | |
| 15:39:56 | dansmith | I haven't looked at that yet, but.. there's likely a lot of potential for those types of things | |
| 15:50:39 | gibi | stephenfin: I have couple of things in the func test for the vdpa https://review.opendev.org/c/openstack/nova/+/780112 | |
| 15:50:54 | stephenfin | good timing - I was just fixing the test failure | |
| 15:51:34 | gibi | ohh, I only run the vdpa test locally not the whole suite | |
| 15:51:53 | gibi | good that we have the CI :) | |
| 15:52:09 | stephenfin | yeah, me too /o\ | |
| 15:52:55 | gibi | sean-k-mooney: I will look at the ops blocking patch before I go to bed today. Now I have to step out before the curfew | |
| 15:53:26 | sean-k-mooney | gibi: im still working on it so it may or may not be ready but hopefully will | |
| 15:53:56 | sean-k-mooney | ill push what i have before the end of the day but might need more time to actully get test for all of them | |
| 15:54:44 | sean-k-mooney | dansmith: we proably should be blcoking all isntace actions | |
| 15:54:52 | sean-k-mooney | at least for now | |
| 15:54:58 | dansmith | sean-k-mooney: agreed | |
| 15:55:30 | dansmith | sean-k-mooney: kinda makes me wonder what the point of system-admin is in a lot of cases, if they can't shutdown or migrate instances | |
| 15:55:31 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: Bump the Compute RPC API to version 6.0 https://review.opendev.org/c/openstack/nova/+/761452 | |