| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-06-17 | |||
| 19:22:53 | sean-k-mooney | hum not sure if good pr or pr blunder https://www.benjerry.ie/flavours/netflix-and-chillld-ice-cream | |
| 19:28:03 | melwitt | thanks for the video model chat o/ | |
| 19:36:30 | mnaser | is there reasoning behind what has ended up being validated in the api layer and what doesn't get validated in the api layer? | |
| 19:37:03 | mnaser | i haven't replicated this yet (but in theory i can imagine it's possible): end user tried to create a vm where image size > disk size when doing bfv | |
| 19:37:16 | mnaser | and they don't really get much more of a helpful error (and their request gets refused too obviously) | |
| 19:37:35 | mnaser | but only after it fails, in what i assume nova-compute trying to ask cinder to create a disk that's impossible to create | |
| 19:53:43 | melwitt | mnaser: generally we validate whatever we can in the api layer to fail fast and avoid the behavior you're describing (fail later on nova-compute). within reason though, as some checks would involve rpc call to the compute host and we avoid that to some degree | |
| 19:54:24 | melwitt | mnaser: I found this part where we validate the image_min_disk against the volume disk size, but that might be different than just the image size as in your example? https://github.com/openstack/nova/blob/master/nova/compute/api.py#L681 | |
| 19:56:15 | mnaser | melwitt: oh interesting, you're right, i wonder if this is a case where the user did not have min_disk specified.. | |
| 19:56:58 | mnaser | melwitt: oh but we don't do image_size > dest_size | |
| 19:57:15 | mnaser | unlike https://github.com/openstack/nova/blob/master/nova/compute/api.py#L693-L695 | |
| 19:58:01 | mnaser | so i _think_ the scenario is no min_disk but no checks on image_size / dst_size | |
| 19:58:24 | melwitt | right thats what I was saying I wonder if thats different | |
| 19:59:05 | melwitt | actually don't know why it's different but I'm not a bfv expert | |
| 19:59:28 | mnaser | melwitt: i think it is, min_size is taken from a glance property which is user-provided (unless a snapshot is taken) | |
| 19:59:34 | mnaser | so i assume user here uploaded an image without min_size | |
| 20:01:06 | melwitt | yeah I mean I don't know why we wouldn't do the same checks for the volume case | |
| 20:01:15 | melwitt | like is that intentional or an oversight | |
| 20:02:55 | mnaser | melwitt: yeah -- in my opinion its an oversight but indeed, maybe someone knows better :) | |
| 20:02:59 | mnaser | anyhow | |
| 20:03:04 | mnaser | ill have a patch up shortly | |
| 20:05:39 | mnaser | melwitt: i swear i saw you mention a while back a key to click before copying from github which gave you a commit hash'd url instead of master | |
| 20:05:56 | melwitt | it's 'y' | |
| 20:09:09 | mnaser | oh awesome, thanks | |
| 20:09:49 | openstack | Launchpad bug 1883969 in OpenStack Compute (nova) "Nova doesn't fail at API layer when image_size > volume_size with BFV" [Undecided,New] | |
| 20:09:49 | mnaser | https://bugs.launchpad.net/nova/+bug/1883969 | |
| 20:13:08 | melwitt | sweet mnaser++ | |
| 20:14:25 | openstackgerrit | Mohammed Naser proposed openstack/nova master: bfv: check if image_size > volume_size in API https://review.opendev.org/736370 | |
| 20:14:40 | mnaser | melwitt: ^ voila, the unit test i wrote passed locally but i didnt run the whole suite | |
| 20:14:44 | mnaser | it makes sense to meâ„¢ | |
| 20:14:44 | mnaser | it makes sense to me™ | |
| 20:20:08 | openstackgerrit | Mohammed Naser proposed openstack/nova master: bfv: check if image_size > volume_size in API https://review.opendev.org/736370 | |
| 20:36:35 | melwitt | mnaser: cool I'll check it | |
| 22:09:24 | gmann | sean-k-mooney: mnaser and to make listing flavor better (instead of need of --private or --all explicitly ) we should add the admin as default tenant access in private flavor. currently what all admin can list the flavors by default and what all it can delete is not matching. | |
| 22:09:55 | gmann | also --private and --all is all admin things and for non-admin --private is just ignored - https://github.com/openstack/nova/blob/6ed556f4adfd8a98fae2e91204f1ce0484f5dd9d/nova/api/openstack/compute/flavors.py#L106 | |
| 22:13:42 | gmann | if we add admin as default tenant access in private flavor then i think we can solve it in backward compatible way but it will be interop issue so that still need microversion. | |
| #openstack-nova - 2020-06-18 | |||
| 03:37:13 | openstackgerrit | Jinsheng Zhang proposed openstack/nova-specs master: Add nova-support-multiple-boot-volume-with-boot-order-selection spec https://review.opendev.org/736422 | |
| 04:05:12 | openstackgerrit | Jinsheng Zhang proposed openstack/nova-specs master: fix "Line limited" error https://review.opendev.org/736425 | |
| 05:07:20 | openstackgerrit | Jinsheng Zhang proposed openstack/nova-specs master: Add nova-support-multiple-boot-volume-with-boot-order-selection spec https://review.opendev.org/736422 | |
| 05:43:46 | openstackgerrit | Jinsheng Zhang proposed openstack/nova-specs master: Add nova-support-multiple-boot-volume-with-boot-order-selection spec fix document error https://review.opendev.org/736422 | |
| 07:50:29 | gibi | sean-k-mooney, melwitt: re: resize without changing the flavor, the context on the PTG was that changing a resource request of a port might also needs a move operation without changing the flavor | |
| 07:51:28 | gibi | sean-k-mooney, melwitt: I'll try to propose a spec for discussion during this cycle so your input about machine_type change and video mode change being similar use case helps. | |
| 07:52:45 | gibi | sean-k-mooney, melwitt: however I think this spec will be targeted to W not V due to time constraints and discussion needs | |
| 08:00:25 | sean-k-mooney | gibi: ah yes i could not recall what it was in relation too | |
| 08:01:48 | sean-k-mooney | gibi: i think there are other usecases too such as recreatign the vm with the same image and flavor but just updating the embeded copy to pick up change to extra_specs and image metadata without actually imaging the disk | |
| 08:03:24 | gibi | but still cold migrating the VM if the current host is not good for the updted extra_spec / image metadata? | |
| 08:03:35 | sean-k-mooney | gibi: yes | |
| 08:03:48 | sean-k-mooney | i would guess in generall it would be a move opertation | |
| 08:03:49 | gibi | OK, that seems like the same operation that I would need for the qos update | |
| 08:04:13 | sean-k-mooney | it might be vaild to use the same host but likely the schduler would not select the same host | |
| 08:04:30 | sean-k-mooney | it might but basically the same as same host resize | |
| 08:05:11 | sean-k-mooney | i guess we could always add a weigher to make it prefer the same host | |
| 08:05:44 | sean-k-mooney | but thats not really important | |
| 08:08:54 | gibi | yeah, I can imagine that the end user needs an to update these data if it does not cause downtime (cold migration) of the VM | |
| 08:09:13 | gibi | * needs an option to update | |
| 08:09:28 | gibi | but same host resize is still downtime today | |
| 08:09:57 | sean-k-mooney | gibi: currently you need to do a db edit and a hard reboot | |
| 08:10:42 | sean-k-mooney | the downstream issue that melwitt was looking at is related to the fact that in rhel 8.2 the default we use in nova "cirrus" was deprecated | |
| 08:10:44 | openstack | sean-k-mooney: Error: Error getting bugzilla.redhat.com bug #1651994: NotPermitted | |
| 08:10:44 | sean-k-mooney | https://bugzilla.redhat.com/show_bug.cgi?id=1651994 | |
| 08:11:01 | sean-k-mooney | oh thats private well that is what it was tracking | |
| 08:11:43 | sean-k-mooney | anyway 16.0 which is based on train was released on 8.1 16.1 will be on 8.2 | |
| 08:12:31 | sean-k-mooney | so customer are now in the situation where the default model that nova will select is deprecaed and there is no way via the api to change this on existing instance other than rebuild | |
| 08:13:08 | sean-k-mooney | we wont be able to back port this new command or whatever it will be but it might help in the future | |
| 08:39:31 | openstackgerrit | Wenping Song proposed openstack/nova master: delete sub resource provider when delete resource provider https://review.opendev.org/719163 | |
| 09:54:40 | openstackgerrit | Wenping Song proposed openstack/nova master: delete sub resource provider when delete resource provider https://review.opendev.org/719163 | |
| 10:38:26 | arne_wiebalck | One of the main points for user adoption probably is that whenever sth needs to change, this should be feasible without having the Ironic admin to re-clean nodes or the Nova admin to add new flavors (with 5000 nodes in Ironic we have around 150 resource classes and hence 150 flavors). This is why we came up with the idea of a "kickstart" driver where Ironic uses a kickstart/preseed file to do the | |
| 10:38:26 | arne_wiebalck | the whole discussion. | |
| 10:38:26 | arne_wiebalck | to deploy. However, other users (like the Ceph team) want to partition/RAID their physical instances in a specific way as needed for their service. There is no convenient way to express this is via Nova/Ironic at the moment. So, they reinstall these instances after initial deployment through Nova/Ironic once more with kickstart. This double installation is what we'd like to get rid of, and this triggered | |
| 10:38:26 | arne_wiebalck | Let me add a little background info: physical instances at CERN are all done via Nova and Ironic. The main user is OpenStack itself and we rely on Ironic's software RAID and standard cloud images | |
| 10:38:26 | arne_wiebalck | TheJulia: dansmith sean-k-mooney Sorry, I missed the discussion yesterday. | |
| 10:38:27 | arne_wiebalck | deployment (and skip the usual image deployment) as it would provide the user with the same flexibility as there is now. But since we would like to keep Nova in the mix (for various reasons), we may need some tooling help on the Nova side. | |
| 12:36:20 | aarents | Hi nova, | |
| 12:38:02 | aarents | sean-k-mooney: dansmith Let me know if it needs further amend on https://review.opendev.org/#/c/736169/ https://review.opendev.org/#/c/734776/ thanks! | |
| 12:39:07 | openstackgerrit | Alexandre Arents proposed openstack/nova master: libvirt: ensure disk_over_commit is not negative https://review.opendev.org/719008 | |
| 12:39:29 | sean-k-mooney | ^ that should be done by the config option | |
| 12:39:33 | sean-k-mooney | we set min 0 i think | |
| 12:40:41 | sean-k-mooney | oh that is not the ratio | |
| 12:41:09 | sean-k-mooney | for that to be negitive the size on disk would have to be larger then the virtual size | |
| 12:41:39 | aarents | sean-k-mooney: yes | |
| 12:41:57 | sean-k-mooney | does it being negitive break something | |
| 12:43:06 | sean-k-mooney | ah i see | |
| 12:43:18 | aarents | It just mislead calcuation of available_disk_least on which rely disk_filter | |
| 12:47:43 | sean-k-mooney | ya clamping the value shoudl be fine. | |
| 13:12:54 | rmart04 | Hey all, I'm wondering if anyone could help me. Not strictly dev specific, but I'm having trouble with NUMA information being passed through to my virtual machines. /sys/bus/pci/devices*/numa_node always = -1. I'm running Rocky on C7, with numa_nodes=2 and cpu_sockets=2. | |
| 13:13:08 | rmart04 | and cpu pinning policy set to dedicated | |
| 13:14:17 | sean-k-mooney | is the -1 on the host or in the guest | |
| 13:14:23 | sean-k-mooney | if its in the guest that is expected | |
| 13:14:26 | rmart04 | In the guest | |
| 13:14:52 | sean-k-mooney | so we do not currently create a pcie root complex per numa node | |
| 13:15:13 | sean-k-mooney | since there is only one pci root all devices are childerne of that root | |
| 13:15:42 | sean-k-mooney | wehn we do passthough we dont affiniteis the pci device to the virutal numa node of the guest | |
| 13:15:59 | sean-k-mooney | so it is reported as -1 meaning no numa affinity in the guest | |
| 13:16:55 | sean-k-mooney | to change that we would likely have to use the q35 machine type and create a pcie root complete per numa node then add the passthough deivice to the correct pci root | |
| 13:17:45 | sean-k-mooney | that has other implciation mainliy that on move operation either we have to allow the toploty to change or we have to limit the host we can select to maintain the current toplogy | |
| 13:18:14 | sean-k-mooney | if we allow the toplogy to change the virtual pci address of the devices in the guest would also change | |
| 13:19:16 | sean-k-mooney | rmart04: but yes if you have a multi numa node guest this can result in cross numa traffic worse case twice because you dont know the numa affinity of the device | |
| 13:20:18 | openstackgerrit | Merged openstack/nova master: libvirt: Mark e1000e VIF as supported https://review.opendev.org/734777 | |
| 13:22:32 | rmart04 | OK, appreciate all the info SeanKMooney. I guess another way around this is to split the host into two guests, one on each numa node with their associated pci-passthrough devices (GPUs). Currently I appear to be blocked on this by my older kernel. 3.10. I bump into an issue allocating memory from the second NUMA node for the second machine. I believe this is fixed in 4.14. | |
| 13:23:06 | rmart04 | qq, you mention the q35 machine type, what type do we use by default? | |