| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-09-04 | |||
| 09:12:59 | brinzhang | gibi: can you check this cherr-pick patch https://review.opendev.org/#/c/749701/ | |
| 09:13:24 | brinzhang | it was backported to ussuri | |
| 09:13:26 | bauzas | gibi: yeah that's an alternative, I was thinking of looking directly at git | |
| 09:13:43 | bauzas | but then you need to just awk or some other script | |
| 09:13:49 | bauzas | just use* awk | |
| 09:17:24 | brinzhang | yes, if we want to get every project, if there is a script that will be better | |
| 09:18:38 | brinzhang | gibi: how to get this release number (19.0.0.0rc1)? form https://releases.openstack.org/ussuri/#nova seems cannot get it | |
| 09:20:35 | gibi | I sorry I mixed up the numbers 20 is train, 21 is ussuri | |
| 09:20:37 | brinzhang | s/form/from | |
| 09:21:02 | gibi | so you need 20.0.0.0rc1..stable/ussuri | |
| 09:21:12 | brinzhang | ack | |
| 09:21:18 | gibi | the rc1 tag is added when we branch out the release | |
| 09:21:34 | gibi | that is the point when master was separated from stable/ussuri | |
| 09:21:39 | brinzhang | so the train is 19.0.0.0rc1, the stein is 18.0.0.0rc1? | |
| 09:22:09 | gibi | train is 20, ussuri is 21 | |
| 09:22:35 | brinzhang | yeah, rocky is 18, stein is 19, train is 20, ussuri is 21 | |
| 09:22:39 | gibi | stein is 19 | |
| 09:22:42 | gibi | yepp | |
| 09:23:12 | brinzhang | very thankful | |
| 09:23:45 | gibi | please note that every project has its own numbering as they had their first release at different cycle | |
| 09:23:56 | gibi | and also in the past openstack used different tag names | |
| 09:25:51 | brinzhang | yeah, I think https://releases.openstack.org/ussuri is correctly | |
| 09:26:25 | brinzhang | just need to change the realse name | |
| 09:26:35 | brinzhang | just need to change the realse name in the url | |
| 09:43:54 | brinzhang | gibi: I find we dont necessary to checkout to the stable/ussuri, in master branch, we also can get the bug | |
| 09:44:17 | brinzhang | of course, we should point the correct branch number | |
| 09:44:55 | gibi | brinzhang: yes, git allows to query the tree without checking out anything | |
| 09:45:08 | brinzhang | \o | |
| 11:11:48 | noonedeadpunk | folks, need some help with figuring out the best way to separate hosts. Let me describe scenario | |
| 11:13:29 | noonedeadpunk | We have group of hosts that have ephemeral storages saved on NFS (/var/lib/nova/instances eventually). And we have another group of hosts with ceph. And we want all new VMs to be created on these ceph hosts, while leave old ones to be operating normally for a while | |
| 11:14:29 | noonedeadpunk | I'm aware about isolate aggregates, but then we will need to hide all current flavors, set some metadata on them, and create new set of flavors | |
| 11:14:51 | noonedeadpunk | Which will result that ppl won't be able to do retype | |
| 11:15:25 | noonedeadpunk | Another thing that come up to mind were AZ. But again, newly created VMs won't be guaranteed not to move to old AZ | |
| 11:16:15 | noonedeadpunk | Didn't look into cells yet tbh, but we eventually want to get rid of first AZ pretty soon, so don't want to overcomplicate things | |
| 11:19:15 | noonedeadpunk | Not usre if setting default_schedule_zone will set AZ for all newly created instances | |
| 11:19:53 | noonedeadpunk | Ok, found it, disregard please:) | |
| 11:19:59 | noonedeadpunk | https://docs.openstack.org/nova/latest/admin/availability-zones.html#implications-for-moving-servers | |
| 11:36:30 | lyarwood | sean-k-mooney: think I've finally reproduced the focal detach issue btw | |
| 11:36:57 | lyarwood | sean-k-mooney: had to do a complete tempest run on a small 8gb devstack instance | |
| 11:50:23 | sean-k-mooney | oh ok | |
| 11:51:04 | sean-k-mooney | any idea what is the cause | |
| 11:51:16 | sean-k-mooney | lyarwood: ^ | |
| 11:59:08 | lyarwood | sean-k-mooney: still working my way through the logs, honestly just feels like the guestOS is just too slow in detaching the device | |
| 12:03:33 | lyarwood | sean-k-mooney: I'm going to wire up the libvirt events for this and see if they show the disk eventually being detached later on or something | |
| 12:06:35 | openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP/DNM libvirt: Increase device detach wait times https://review.opendev.org/749827 | |
| 12:06:36 | openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP/DNM libvirt: Start emitting DeviceRemovedEvent and DeviceRemovalFailedEvent events https://review.opendev.org/749929 | |
| 12:07:21 | sean-k-mooney | lyarwood: i wonder if this is realted to the pc machine type too | |
| 12:07:38 | sean-k-mooney | lyarwood: we handel pci hotplug differently in q35 | |
| 12:07:48 | sean-k-mooney | the virtio-block interfaces we use by default | |
| 12:08:07 | sean-k-mooney | all are hotpulged and unpluged when we attach and detach them | |
| 12:08:07 | lyarwood | sean-k-mooney: the machine type should be the same as earlier releases no? | |
| 12:08:18 | sean-k-mooney | it is | |
| 12:08:49 | sean-k-mooney | but for the pc machine type we are usin ahci interupt to do the hot unplug for the detach | |
| 12:09:03 | sean-k-mooney | but we use the native pcie support in q35 | |
| 12:09:12 | sean-k-mooney | the ahci appoch is slower and more error prone | |
| 12:09:31 | sean-k-mooney | lyarwood: so im jsut aggreeing really that this coudl be a slow guest | |
| 12:09:37 | lyarwood | ah right | |
| 12:09:38 | sean-k-mooney | or at least its taking a while | |
| 12:09:46 | sean-k-mooney | q35 might mitigate that | |
| 12:09:55 | sean-k-mooney | __might__ | |
| 12:10:00 | sean-k-mooney | its really just a guess | |
| 12:10:17 | lyarwood | it could be that the newer versions of QEMU are consuming more resources on the host leading to instances being slower than previous releases | |
| 12:10:31 | sean-k-mooney | maybe | |
| 12:10:33 | lyarwood | but agreed that q35 should make this better | |
| 12:10:43 | sean-k-mooney | i wonder if the issue would go away if we used virtio-scsi too | |
| 12:11:16 | sean-k-mooney | if nothing else if we confrimed that that could be a workaround for the guest | |
| 12:11:18 | sean-k-mooney | *ci | |
| 12:11:56 | sean-k-mooney | im guessing we need a retry loop or something else to handel it more robustly however | |
| 12:12:03 | sean-k-mooney | if i can help let me know | |
| 12:15:10 | lyarwood | there is a loop at the moment that I'd like to replace with the events listed above instead eventually | |
| 12:15:24 | lyarwood | that still has corner cases but it would be better than this imho | |
| 12:16:48 | sean-k-mooney | ya the events would be better | |
| 12:52:52 | openstackgerrit | Merged openstack/nova stable/ussuri: Add a lock to prevent race during detach/attach of interface https://review.opendev.org/749033 | |
| 13:12:51 | bauzas | dansmith: when you're up, remember me, we can't just change a method name in an o.vo object ? | |
| 13:13:04 | bauzas | because of the compatibility | |
| 13:13:53 | bauzas | dansmith: like, I'd like to rename https://github.com/openstack/nova/blob/master/nova/objects/request_spec.py#L538 | |
| 13:14:13 | bauzas | to be, say, ensure_network_information(ctxt, instance) | |
| 13:14:53 | bauzas | ah no | |
| 13:14:57 | bauzas | now I remember | |
| 13:15:11 | bauzas | unless you say it's a remotable method, you can do it | |
| 13:16:00 | bauzas | like https://github.com/openstack/nova/blob/master/nova/objects/compute_node.py#L256 | |
| 13:16:25 | bauzas | kk, all good | |
| 13:28:17 | dansmith | bauzas: sure you can change a remotable one all you want :) | |
| 13:28:22 | dansmith | er a NON-remotable on | |
| 13:28:30 | bauzas | yup, now I remember | |
| 13:28:53 | bauzas | only decorated methods need to have a major version to be removed or renamed | |
| 13:29:51 | bauzas | that's what happens when you stay too much in libvirt :) | |
| 13:35:41 | dansmith | heh | |
| 13:42:12 | sean-k-mooney | dansmith: added you to this https://review.opendev.org/#/c/747395/ | |
| 13:42:31 | sean-k-mooney | dansmith: its an oslo messigning change to move the heartbeat to a real thread | |
| 13:42:42 | dansmith | oh boy | |
| 13:42:48 | sean-k-mooney | i know you had concens about that and wsgi before | |
| 13:43:06 | sean-k-mooney | mainly breaking the wsgi server thread lifecycle | |
| 13:43:50 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add type hints to 'nova.virt.libvirt.utils' https://review.opendev.org/742865 | |
| 13:43:51 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add support for resize and cold migration of emulated TPM files https://review.opendev.org/639934 | |
| 13:47:13 | dansmith | sean-k-mooney: yeah, I just added native threading support to glance's API so that they could do the things they need to do in wsgi mode and it seems to be fine | |
| 13:47:26 | dansmith | dug into what uwsgi expects you to do for threading more so I have a better handle on it | |
| 13:53:50 | sean-k-mooney | ok just wanted to keep you in the loop incase that could cause issues there is a FFE request on the mailing list related to it | |
| 13:58:37 | dansmith | ugh, well, *rushing* that doesn't seem like a good plan | |
| 13:59:05 | dansmith | oh, I see, | |