| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-06-24 | |||
| 15:28:38 | bauzas | stephenfin: yup, haven't forgotten but I looked at the whole series to refresh my mind | |
| 15:30:03 | dansmith | melwitt: thanks | |
| 15:42:51 | openstackgerrit | Ghanshyam Mann proposed openstack/nova stable/stein: Make greande jobs n-v for EM and oldest stable https://review.opendev.org/737332 | |
| 15:59:48 | bauzas | stephenfin: question before I leave https://review.opendev.org/#/c/714697/10/nova/objects/instance_numa.py@213 | |
| 16:00:39 | bauzas | I do understand this could be silly to have more than one policy, but should we have a kind of nova status upgrade check preventing it ? | |
| 16:37:07 | stephenfin | bauzas: It's not just silly, it's impossible | |
| 16:37:39 | stephenfin | The only way we'd get into that situation is via manual DB modifications, and when that's done all cards are off the table | |
| 16:38:25 | stephenfin | An upgrade check wouldn't resolve anything either, since if someone had done manual DB modifications beforehand, they could do them again after the upgrade check ran | |
| 16:41:32 | sean-k-mooney | bauzas: it was just stored in the wrong place | |
| 16:41:47 | sean-k-mooney | bauzas: the cpu policy has alsways been vm wide | |
| 16:41:52 | sean-k-mooney | not per cell | |
| 16:41:57 | stephenfin | yup. That was my bad | |
| 16:42:04 | stephenfin | I was young and innocent | |
| 16:42:13 | sean-k-mooney | i actully think this predates you | |
| 16:42:22 | stephenfin | nope, I added that | |
| 16:42:45 | sean-k-mooney | :'O how could you :P | |
| 16:42:53 | stephenfin | previously we determined whether policy was in effect by whether cpu_pinning_raw was set or not | |
| 16:43:01 | stephenfin | _that_ was ndipanov | |
| 16:43:11 | sean-k-mooney | ah ok | |
| 16:46:22 | sean-k-mooney | stephenfin: also handnt review that till now but it looks correct | |
| 16:47:05 | stephenfin | thanks | |
| 18:44:48 | sean-k-mooney | melwitt: dansmith shoudl this have a +w https://review.opendev.org/#/c/656998/ | |
| 18:45:33 | sean-k-mooney | dansmith: have you resolved the fact that it needs admin to work unless you own the image | |
| 18:45:57 | sean-k-mooney | https://review.opendev.org/#/c/734184 has the depends on but that earlier patch does not | |
| 18:45:59 | dansmith | sean-k-mooney: no, buried under many other dependent issues right now | |
| 18:46:33 | sean-k-mooney | dansmith: ok so the first two patches in the change have 3 +2s and +w | |
| 18:46:54 | dansmith | that should do it :) | |
| 18:47:03 | dansmith | it works as expected if you own the image, | |
| 18:47:11 | dansmith | and it's off by default, so I think it's okay at the moment from our side anyway | |
| 18:47:28 | dansmith | it just doesn't work with the image we create in devstack | |
| 18:47:32 | sean-k-mooney | right i was wondering if you wanted to hold them untill you fixed the other case but if you ok with them mergeing then cool | |
| 18:47:44 | dansmith | I think they're fine as is, | |
| 18:48:05 | dansmith | it's early in the cycle and if we don't get something resolved (I'm sure we will) then we can add to the docs patch about caveats | |
| 18:48:39 | sean-k-mooney | dansmith: ya if we dont figure something out we can always add a worksaround config option if needed to use admin | |
| 18:48:52 | sean-k-mooney | or document it as you said | |
| 18:48:56 | dansmith | yeah, although that has caveats for quota | |
| 18:49:00 | dansmith | so docs either way | |
| 18:49:47 | sean-k-mooney | ok in that case im going to call it a day o/ | |
| 18:50:14 | dansmith | thanks for looking out :) | |
| 20:17:36 | melwitt | dansmith, gmann: I've got a small fix proposed for a gate bug [that doesn't happen very often] around a InstanceMapping.save() race if you might be interested in reviewing https://review.opendev.org/734400 | |
| 20:21:46 | dansmith | melwitt: why the queued_for_delete=True in the test? | |
| 20:21:56 | dansmith | won't hurt anything but doesn't seem relevant? | |
| 20:23:04 | spatel | sean-k-mooney: does openstack has any config option to tell build 5 machine in single shot? | |
| 20:23:31 | melwitt | dansmith: oh, I was thinking (at the time) I needed something to "save" with it but the uuid would have done it huh? other than that it just mimics the actual bug (update of qfd failed) but not really relevant | |
| 20:23:31 | spatel | I am using terraform and its not letting me build 10 machine but if i say do 5 machine it works | |
| 20:23:55 | dansmith | melwitt: but the copy you're save()ing shouldn't have that set in the real race condition | |
| 20:24:14 | dansmith | the *other* one might, but not the stale one right? | |
| 20:24:23 | dansmith | er, the one that really got deleted | |
| 20:25:51 | melwitt | dansmith: no it shouldn't. what I was thinking was I wanted to make an instance mapping that is not yet qfd = True and then in the test I would set qfd = True and save it but I cheated by not having the mapping be returned by a "get" method 😬 | |
| 20:25:51 | melwitt | dansmith: no it shouldn't. what I was thinking was I wanted to make an instance mapping that is not yet qfd = True and then in the test I would set qfd = True and save it but I cheated by not having the mapping be returned by a "get" method 😬 | |
| 20:26:37 | dansmith | so gonna snip that out for posterity? | |
| 20:26:45 | melwitt | sure | |
| 20:27:09 | dansmith | it sent me on a goose chase trying to figure out why it mattered, so probably good | |
| 20:27:16 | melwitt | oh sorry :( | |
| 20:27:32 | dansmith | my other -1 was going to be "we can't merge patches with only one revision because fairness" so this will fix that too :D | |
| 20:28:09 | dansmith | otherwise seems like an obvious fix | |
| 20:29:06 | melwitt | heh :) | |
| 20:29:56 | openstackgerrit | melanie witt proposed openstack/nova master: Raise InstanceMappingNotFound if StaleDataError is encountered https://review.opendev.org/734400 | |
| 20:30:21 | melwitt | dansmith: done ^ | |
| 20:30:32 | dansmith | beatcha | |
| 20:30:38 | melwitt | danke | |
| 21:04:44 | sean-k-mooney | spatel: yes not supports multi create and no you should not use it | |
| 21:05:21 | sean-k-mooney | unless you meen the per compute node build limit which is 10 | |
| 21:05:59 | spatel | sean-k-mooney: i am talking about in general limit (not per compute) | |
| 21:06:19 | sean-k-mooney | spatel: im pretty sure we dont have an api limit at least not one that would block 5 | |
| 21:06:25 | spatel | one of user trying to create bunch of vm for testing and its failing even i have capacity | |
| 21:06:37 | sean-k-mooney | the api is limited by your quota | |
| 21:07:01 | spatel | sean-k-mooney: must be something else going on (just wanted to check if i missed something here) | |
| 21:07:06 | sean-k-mooney | but if you ask for 300 instance in once request we will spawn them | |
| 21:08:56 | spatel | in my case its 10 vm so i don't think its going to be limit issue, let me debug and see | |
| 21:10:33 | melwitt | I think you _should_ use multi create if you're requesting (anti-)affinity | |
| 21:11:51 | spatel | what is multi create? (my users using terraform to spin up vm, i don't think they are requesting any anti-affinity) | |
| 21:12:44 | sean-k-mooney | melwitt: if you do that and a vm retires it will use the alternate allcoation which are now invalid becaue another vm could have use the same alternate | |
| 21:12:47 | spatel | I have noticed big spike on RabbitMQ so not sure if my rabbitMQ is under heavy load | |
| 21:13:31 | sean-k-mooney | melwitt: so if you realy want anti affintiy and dont want to risk retries causeing the full set to fail you should boot them serially | |
| 21:14:03 | sean-k-mooney | spatel: openstack server create --min=5 --max=10 --flavor whatever... | |
| 21:14:05 | melwitt | sean-k-mooney: why would you expect retries? numa request? | |
| 21:14:29 | sean-k-mooney | melwitt: numa or sriov are the most common | |
| 21:15:02 | sean-k-mooney | melwitt: if you are not using either then ya multi create is fine | |
| 21:15:17 | melwitt | yeah, ok | |
| 21:15:38 | sean-k-mooney | the only reason for retires in that case is an actual failure on the compute node which hopefully isnt common in your cloud | |
| 21:16:55 | sean-k-mooney | spatel: ^ create betwen 5 and 10 server with a given flavor image network excrta | |
| 21:17:41 | spatel | but if we are using terraform then we can't tell them how to limit | |
| 21:17:57 | spatel | openstack server create may handle that well | |
| 21:18:07 | sean-k-mooney | spatel: terraform is likely makeing multiple requests | |
| 21:18:31 | spatel | sean-k-mooney: yes i believe its all parallel fork | |
| 21:19:20 | sean-k-mooney | ya so unless your hitting quota issue there is not limit in how many build requests you can make in parrale in that case | |
| 21:19:32 | sean-k-mooney | ratelimiting of api requests is out of scope of nova | |
| 21:22:57 | spatel | sean-k-mooney: good to know! i think i have something else going on. | |
| 23:18:25 | openstackgerrit | sean mooney proposed openstack/nova master: [WIP] make provider tree and proviers iterable https://review.opendev.org/737594 | |
| #openstack-nova - 2020-06-25 | |||
| 00:33:52 | gmann | melwitt: +A, i was thinking to capture the im-not-found in _do_delete() also but if request reach there it should be a normal delete and not any race. | |
| 00:35:06 | gmann | melwitt: dansmith: elod finally this is ready, stable/stein unblocking gate, please review - https://review.opendev.org/#/c/737332/ | |
| 00:36:01 | gmann | this makes the grenade job as n-v and depends-on fixes the lgeacy multinode jobs for 'virutalenv not found' | |
| 00:36:22 | melwitt | gmann: ah yeah. thanks! | |
| 00:36:48 | melwitt | gmann: ok cool, I was just about to ask if that was ready. I will look tomorrow if others don't beat me to it | |
| 00:37:19 | melwitt | oh, actually that's very easy so I'll look now | |
| 00:37:45 | melwitt | gmann: are we planning to backport that to queens too or? | |
| 00:37:47 | gmann | melwitt: thanks. yeah, and depends-on also merged to good to go | |
| 00:38:49 | gmann | melwitt: yes but stable/rocky and other EM stable gate is still blocked, we need this devstack fix to get in and then backport this. devstack fix which need more work - https://review.opendev.org/#/c/735615/ | |
| 00:39:02 | melwitt | ack | |