| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-01-14 | |||
| 17:42:41 | sean-k-mooney | fungi: that does not cover libvirt verions | |
| 17:42:46 | sean-k-mooney | or qemu | |
| 17:43:01 | sean-k-mooney | we manage that seperatly | |
| 17:43:28 | lyarwood | I didn't need that, just the distros | |
| 17:43:48 | sean-k-mooney | ah then ya latest lts of ubuntu centos and suse | |
| 17:43:48 | lyarwood | or openSUSE sorry | |
| 17:44:39 | sean-k-mooney | fungi: has there been any moemente on updating centos 8 sot centos 8 stream | |
| 17:44:54 | sean-k-mooney | fungi: i assume we will change that in X | |
| 17:45:27 | fungi | sean-k-mooney: i'm not aware of any tc level decision around that yet. discussions were that we only needed to solve it for the cycle in which 8 becomes eol | |
| 17:45:49 | sean-k-mooney | ok so i guess technialy Y then | |
| 17:46:02 | sean-k-mooney | EOL is decemebr | |
| 17:46:06 | fungi | though i'm not entirely sure what it means for stable branches which supported 8 and how we choose to test them | |
| 17:46:25 | sean-k-mooney | ya | |
| 17:46:38 | fungi | since opendev doesn't usually continue to provide images for eol distro versions for very long | |
| 17:46:46 | sean-k-mooney | one option might be to use rocky linux or one of the repelacement for centos | |
| 17:47:41 | sean-k-mooney | i guess we will see how things play out | |
| 17:48:59 | sean-k-mooney | centos 8 stream should sablise once centos 9 stream is a thing but no idea when that will be a thing or how long centos 8 stream will be supported when its released | |
| 17:58:02 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Remove MIN_{LIBVIRT,QEMU}_BLOCKDEV https://review.opendev.org/c/openstack/nova/+/754701 | |
| 17:58:02 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION and NEXT_MIN_{LIBVIRT,QEMU}_VERSION https://review.opendev.org/c/openstack/nova/+/754703 | |
| 17:58:03 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Remove MIN_LIBVIRT_S390X_CPU_COMPARE https://review.opendev.org/c/openstack/nova/+/754704 | |
| 18:05:34 | openstackgerrit | sean mooney proposed openstack/nova stable/train: only wait for plugtime events in pre-live-migration https://review.opendev.org/c/openstack/nova/+/770844 | |
| 18:11:53 | stephenfin | lyarwood: Seeing as you're looking at DB stuff at the moment, think you could take a look at these, my two oldest outstanding patches? :) https://review.opendev.org/c/openstack/nova/+/530905/ https://review.opendev.org/c/openstack/nova/+/530487/ | |
| 18:12:44 | openstackgerrit | sean mooney proposed openstack/nova master: use psycopg2 binary instead of source package https://review.opendev.org/c/openstack/nova/+/770473 | |
| 18:16:30 | sean-k-mooney | stephenfin: why do we need uuid in the pci devices table? | |
| 18:18:04 | sean-k-mooney | oh its for tracking them in placment | |
| 18:18:09 | sean-k-mooney | so its not used currently | |
| 18:21:22 | sean-k-mooney | +1 on both but its also strictly speaking not needed currently unless i missed where its beign used | |
| 18:21:36 | sean-k-mooney | in anycase time for dinner | |
| 19:35:17 | artom | melwitt, so, for the record, I am looking at https://review.opendev.org/c/openstack/nova/+/769614 :) | |
| 19:35:35 | artom | But it's in an area of hardware.py that I'm not really familiar with, so there's a learning curve | |
| 19:38:33 | dansmith | lyarwood: since you've brought it up a couple times, I think this is going to fix the multistore test hang/fail: https://review.opendev.org/c/openstack/tempest/+/770850 | |
| 19:38:37 | dansmith | also gibi ^ | |
| 19:56:08 | qqmber | hi there | |
| 22:41:37 | openstackgerrit | Ghanshyam proposed openstack/nova master: Reuse code from oslo lib for JSON policy migration https://review.opendev.org/c/openstack/nova/+/763487 | |
| 22:46:46 | openstackgerrit | Ghanshyam proposed openstack/placement master: [goal] Deprecate the JSON formatted policy file https://review.opendev.org/c/openstack/placement/+/764394 | |
| 22:54:24 | dansmith | melwitt: replied | |
| 22:55:03 | melwitt | dansmith: danke | |
| 22:55:59 | melwitt | I was coincidentally looking at some different gate failures involving presumable stress on node resources and wanted to see if there are similarities | |
| 22:56:43 | dansmith | well, this was a pretty standard OOM once I realized that, | |
| 22:57:09 | dansmith | but it's unusual that the OOM is caused by the test runner, so I hadn't seen an OOM manifest itself as a killing of one worker before | |
| 22:58:52 | melwitt | I see now, in syslog of the example you linked. literal oom-killer messages. what I've been looking at today is different | |
| 23:00:06 | dansmith | this one blows up so fast that it doesn't even register in dstat, | |
| 23:00:15 | dansmith | which is what took me a bit to realize it was an oom | |
| 23:00:28 | dansmith | it goes from like 45M to 1.5g in a couple seconds locally | |
| 23:00:33 | melwitt | wow | |
| 23:01:09 | melwitt | how were you able to tell it's the test runner? something in the logs? | |
| 23:01:45 | dansmith | well, the oom was a python process, and one worker (which is python) just never reports its results, | |
| 23:02:21 | dansmith | then reproing locally (with plenty of memory) I see one test runner blip to 1.5g, before exiting normally, which is the same size as the victim from the killer | |
| 23:02:42 | dansmith | I don't oom because I have lots of memory locally, but I could see that balloon, and if it happens at the wrong time in the gate, it'll trigger and die | |
| 23:03:06 | melwitt | I see, thanks | |
| 23:05:30 | melwitt | nice find. I've been looking at a lot of gate failures lately that seem to be outside of openstack and trying to find hints about what could be causing them | |
| 23:21:49 | openstackgerrit | melanie witt proposed openstack/nova stable/stein: Change default num_retries for glance to 3 https://review.opendev.org/c/openstack/nova/+/764963 | |
| 23:24:43 | openstackgerrit | melanie witt proposed openstack/nova stable/stein: Use fair locks in resource tracker https://review.opendev.org/c/openstack/nova/+/770657 | |
| 23:49:38 | sean-k-mooney | melwitt: actully regarding ^ | |
| 23:50:00 | sean-k-mooney | i realised that while the upper-constatis allows it im not sure lower constartis on stien does | |
| 23:50:50 | sean-k-mooney | https://github.com/openstack/nova/blob/stable/stein/lower-constraints.txt#L79 oslo.concurrency==3.26.0 | |
| 23:51:19 | sean-k-mooney | it need 3.29.0 | |
| 23:51:42 | sean-k-mooney | are we allowed to bump it like that in a backport? | |
| 23:54:02 | sean-k-mooney | for what its worth i think its a good thing to backport just dont know what stable policy allows | |
| 23:54:06 | melwitt | oh yeah, we aren't. so I guess I was right originally, that it couldn't be backported | |
| 23:54:34 | melwitt | sigh | |
| 23:55:15 | sean-k-mooney | well with the patch on gerrit if peopel want to use it it should be eaiser for them | |
| 23:56:13 | sean-k-mooney | to do the backport you would have to add the fair lock form oslo to noava | |
| 23:56:32 | sean-k-mooney | and just use that instead | |
| 23:57:04 | melwitt | yeah, I doubt it's worth going that far | |
| 23:57:51 | sean-k-mooney | ya | |
| #openstack-nova - 2021-01-15 | |||
| 03:29:21 | openstackgerrit | Brin Zhang proposed openstack/nova master: Cyborg shelve/unshelve support https://review.opendev.org/c/openstack/nova/+/729563 | |
| 08:04:06 | gibi | dansmith: thanks for finding out reasone of the multistore failure an proposing a fix. nice catch! | |
| 08:17:31 | lyarwood | morning | |
| 08:18:04 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Remove MIN_{LIBVIRT,QEMU}_BLOCKDEV https://review.opendev.org/c/openstack/nova/+/754701 | |
| 08:18:04 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION and NEXT_MIN_{LIBVIRT,QEMU}_VERSION https://review.opendev.org/c/openstack/nova/+/754700 | |
| 08:18:05 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Remove MIN_LIBVIRT_S390X_CPU_COMPARE https://review.opendev.org/c/openstack/nova/+/754704 | |
| 09:11:34 | openstackgerrit | wushiming proposed openstack/os-vif master: Drop lower-constraints job https://review.opendev.org/c/openstack/os-vif/+/770907 | |
| 09:17:59 | nightmare_unreal | Hello , is there a way to force a VM to boot from volume. For e.g. during VM creation i want to enforce that it should be from volume only | |
| 09:19:21 | kashyap | lyarwood: Morning; for the NEXT_ version bump, we also traditionally give a heads-up on the openstack-discuss list -- wanna do that too? (E.g. one of my past emails: http://lists.openstack.org/pipermail/openstack-discuss/2019-November/010907.html) | |
| 09:22:29 | openstackgerrit | wushiming proposed openstack/python-novaclient master: Drop lower-constraints job https://review.opendev.org/c/openstack/python-novaclient/+/770912 | |
| 09:24:43 | lyarwood | kashyap: yup can do | |
| 09:25:14 | kashyap | Thank ya. | |
| 09:35:56 | songwenping_ | sean-k-mooney: Hi, what do you means about the vgpu spec: https://review.opendev.org/c/openstack/nova-specs/+/750116/17#L97, should we use rpc.cast the cyborg-agent to create a new mdev? | |
| 09:37:28 | lyarwood | kashyap: sent | |
| 09:39:43 | kashyap | lyarwood: Just read the mail in the archives; looks fine, thx. | |
| 09:41:59 | lyarwood | cool thanks | |
| 10:20:21 | songwenping_ | gibi: hi, about your comment in line https://review.opendev.org/c/openstack/nova/+/622336/20/nova/api/openstack/compute/remote_consoles.py#186, `if svc.version < |
|
| 10:21:14 | brinzhang0 | gibi: cyborg shelve/unshelve updated alex_xu's concern, pls re-check again, thanks https://review.opendev.org/c/openstack/nova/+/729563 | |
| 10:21:30 | gibi | songwenping_: I would go with http 400 | |
| 10:21:38 | gibi | brinzhang0: ack | |
| 10:22:38 | songwenping_ | gibi: get | |
| 10:23:06 | brinzhang0 | gibi: yes, the same as songwenping_'s concern , IMO 400 is more suitable | |
| 10:24:21 | brinzhang0 | gibi: thanks for check the cyborg shelve/unshelve patch ^^ | |
| 11:29:47 | openstackgerrit | Lee Yarwood proposed openstack/nova master: api: Log os-resetState as an instance action https://review.opendev.org/c/openstack/nova/+/770926 | |
| 11:43:24 | gibi | bauzas: are your concerns answered in https://review.opendev.org/c/openstack/nova/+/729563 ? I'm holding the +W until ack it | |
| 12:36:00 | sean-k-mooney | songwenping__: i was suggesting that gets shoudl not create teh mdev ever. the cyborg agent should create the mdevs for all bound arqs on start up. ideally it would set the binding state to "provisioning" on start up also and we would have anohter state "unknown" that the api would report if the cyborg agent misses its heart beat | |
| 12:37:12 | sean-k-mooney | songwenping__: so on start up nova would see 1 of 3 states, bound if the cyborg agent started first and hadn completed bininding, unknown if the cyborg agent has not heartbeat to the cyborg conductor yet | |
| 12:37:36 | sean-k-mooney | or provisioning if the agent is in the process of creating the mdev | |
| 12:38:32 | sean-k-mooney | if cyborg was to send the same bining complete event when it change teh status form provisioning to bound as it does during normal arq binding | |
| 12:38:45 | sean-k-mooney | the process in nova awould be the same | |
| 12:39:34 | sean-k-mooney | e.g. set up event handeler, check if its already bound if not wait for binding complete event, if it si bound cancel the even waiter and proceed with boot | |
| 12:40:55 | sean-k-mooney | gibi: bauzas does that sound like a resonable approch to ye instead of have get arq sometimes do an rpc call to the cybrog agent to create mdevs | |
| 12:41:37 | bauzas | gibi: will look at https://review.opendev.org/c/openstack/nova/+/729563 | |
| 12:42:28 | gibi | bauzas: thanks | |