| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-11-10 | |||
| 13:33:40 | opendevreview | Balazs Gibizer proposed openstack/nova stable/xena: Define new functional test tox env for placement gate to run https://review.opendev.org/c/openstack/nova/+/817256 | |
| 13:35:05 | opendevreview | Balazs Gibizer proposed openstack/placement stable/xena: Use 'functional-without-sample-db-tests' tox env for placement nova job https://review.opendev.org/c/openstack/placement/+/817255 | |
| 13:35:44 | gibi | gmann: ^^ added the missing cherry-pick lines in these ^^ | |
| 14:25:39 | gmann | gibi: thanks | |
| 15:05:11 | opendevreview | Balazs Gibizer proposed openstack/nova stable/xena: Fix interference in db unit test https://review.opendev.org/c/openstack/nova/+/817240 | |
| 15:25:57 | opendevreview | Stephen Finucane proposed openstack/nova master: Use unittest.mock instead of third party mock https://review.opendev.org/c/openstack/nova/+/714676 | |
| 15:43:34 | lyarwood | Just when you thought the nova codebase was safe from stephenfin ;) | |
| 15:44:09 | lyarwood | Can we give you the Friday nick of SirChangeALot? | |
| 15:44:20 | stephenfin | someone complained about us still using it recently. blame that person | |
| 15:44:28 | stephenfin | whose name I can't remember | |
| 15:44:36 | lyarwood | ^_^ | |
| 15:44:43 | lyarwood | I'll queue that up later in the week | |
| 15:44:57 | dasm | "it's them, not us" :) | |
| 15:49:24 | sean-k-mooney | lyarwood: for what its worth its really pretty tirvial | |
| 15:49:45 | lyarwood | yeah I know I was only jesting | |
| 15:49:54 | sean-k-mooney | most of it was find replace and one or two other changes which are ease to fined by the change size beside the file | |
| 15:50:20 | sean-k-mooney | it made me sad you cant sort the list of files in the gerrit review by changed lines | |
| 15:50:32 | sean-k-mooney | it would be a nice feature for that type of change | |
| 15:53:01 | EugenMayer | holds that data | |
| 15:53:01 | EugenMayer | Ah nova backup process was queued, but it was queue since i had not enough space. I removed the queued images in the ui (glance images), but now instances show up a task `image backup` and i cannot see how to remove it. Is that a nove tasks i can list (cannot find anything with the nova cli) or is that rather a glance task? A little confused who | |
| 15:54:43 | EugenMayer | `glance task-list` does not show up those tassks | |
| 16:00:40 | EugenMayer | is `nova reset-state --active` the way to solve this | |
| 16:41:05 | opendevreview | Alexey Stupnikov proposed openstack/nova master: Test aborting queued live migration https://review.opendev.org/c/openstack/nova/+/776250 | |
| 17:37:13 | sean-k-mooney | gibi: good to see the neutron sepc is apporved ill try to rereview https://review.opendev.org/c/openstack/nova-specs/+/787458 this week but i belive its largely the same as v9 and i was ok with the overall design a that point. | |
| 17:37:53 | sean-k-mooney | by the way when was/is the spec review day | |
| 17:38:25 | sean-k-mooney | ah the 16th | |
| 17:38:40 | sean-k-mooney | i tought i migth have missed it ill be looking at specs tomorrow in anycase | |
| 17:40:03 | sean-k-mooney | EugenMayer: reste-state shoudl be used with caution generally as a last resort before do ing a direct db modification | |
| 17:40:28 | sean-k-mooney | if its stuck in the image backup ste a hard reboot might be a better first attempt | |
| 17:40:53 | sean-k-mooney | reststate may leave the instnace in an unclean state | |
| 17:56:50 | opendevreview | Merged openstack/nova master: db: Remove models that were moved to the API database https://review.opendev.org/c/openstack/nova/+/812149 | |
| 17:57:22 | EugenMayer | sean-k-mooney that is what happened, reset state was clearing the ui, but it stuck nevertheless. Tried soft reboot, was not able to, so i rebooted the compute | |
| 17:58:00 | EugenMayer | i cannot say that openstack shines in this areas to be honest. Not used to have such undefined, oncontrolled states created by UI interactions alone | |
| 17:58:31 | EugenMayer | in general, tasks cannot be stopped, cannot be seen and cannot be controlled, while they can stuck. This is the one area i'am kind of frustrated about | |
| 17:58:56 | sean-k-mooney | EugenMayer: that is because they are not independent tasks for the most poart | |
| 17:59:30 | sean-k-mooney | there are a limit set of operation that are modeled as task like live migration that can be aborted | |
| 17:59:45 | sean-k-mooney | but most api action are not premetable once issued | |
| 18:02:01 | sean-k-mooney | EugenMayer: creating a backup for example is not really a task its an imperitive action you have taken that shoudl either complete or put the vm in an error state | |
| 18:02:18 | sean-k-mooney | that error state can be then recovered form vai a hard-reboot or other operation | |
| 18:02:50 | sean-k-mooney | where we can recover form a failed operation nova shoudl and mark the instnace event in error | |
| 18:02:52 | opendevreview | Merged openstack/nova master: objects: Remove 'bandwidth' fields from notifications https://review.opendev.org/c/openstack/nova/+/812148 | |
| 18:03:01 | opendevreview | Merged openstack/nova master: db: Remove models for removed services, features https://review.opendev.org/c/openstack/nova/+/812150 | |
| 18:03:11 | EugenMayer | actually controlling it. And if the task creating that image never reports back and most probably moves the server task/state further, it is stuck forever | |
| 18:03:11 | EugenMayer | it's what happens if 'microservice and self contained' is the main principal as an arch without controller the echo that comes with it. In an image backup, you invoke via nova, which then invokes a glance task, set some second state or whatever somewhere. So we have a glance task and state, a second task and state for the server and nothing that is | |
| 18:04:31 | opendevreview | Dan Smith proposed openstack/nova master: Log instance event wait times https://review.opendev.org/c/openstack/nova/+/817030 | |
| 18:04:38 | opendevreview | Merged openstack/nova master: db: Remove nova-network models https://review.opendev.org/c/openstack/nova/+/812151 | |
| 18:04:42 | EugenMayer | if you decouple that much, the hard thing becomes cascading and controlling events and tasks which involve several subsystem. And for this part, it seems that is where openstack has nothing in place. Nova tasks in nove - great. if it involves other parts, things get stuck very fast. Be it neutron, the storage or similar. | |
| 18:05:27 | EugenMayer | Anyway, sorry. A little frustrated. I get over it :) | |
| 18:05:36 | sean-k-mooney | we do have some recovery logic but asa an operator you really should not be managing any of that internal state | |
| 18:06:22 | sean-k-mooney | https://docs.openstack.org/api-ref/compute/#servers-actions-servers-os-instance-actions is where you can list the action performed on an instance by the way | |
| 18:06:28 | EugenMayer | (i would love to not manage it, but it becomes my duty if queued stuck because it waits to claim storage space, which does not happen and then it rolls forever) | |
| 18:07:33 | sean-k-mooney | it should actully fail if there is not enouch space. is it failing on the image upload | |
| 18:07:41 | sean-k-mooney | or the creation of the inital snapshot | |
| 18:08:56 | EugenMayer | sean-k-mooney interesting, i have see that API but assumed this was rather 'list the actions one can do'. I guess 'nova instance-action-list' is not that, but 'openstack server event list esxi` | |
| 18:09:39 | EugenMayer | well i can assure you, it did not fail but has been queued. The task was started using `nova backup ...` | |
| 18:10:29 | sean-k-mooney | qued in glance | |
| 18:10:50 | EugenMayer | Yes, but 'instance backup' in nova/compute | |
| 18:11:31 | EugenMayer | i deleted the image in glance (not stopping the task, if that is possible at all), which then stuck the compute instance in 'instance backup' forever | |
| 18:12:08 | EugenMayer | A different thing, trying to host an ESXi as a VM on openstack KVM. The sole purpose of that VM is using ESXi native tools to convert qcow2 disks to _proper_ vmdks | |
| 18:12:27 | sean-k-mooney | instance backup... so its no in image uploading or image pendign upload | |
| 18:13:03 | sean-k-mooney | EugenMayer: right we would not expect deleting the image in glance to stop the task | |
| 18:13:12 | EugenMayer | I have run this under proxmox with ethernet e1000 but under openstack it starts with virtio, which seems not to be supported by ESXi thus the VM has no interface when booted. Can i switch the interface type? | |
| 18:13:12 | sean-k-mooney | that could infact break our cleanup code | |
| 18:13:24 | EugenMayer | sean-k-mooney that i understood - after that :) | |
| 18:13:28 | sean-k-mooney | you can specify e1000 | |
| 18:13:39 | EugenMayer | via xml only? | |
| 18:13:55 | sean-k-mooney | you set hw_vif_model=e1000 in the glance image metadata | |
| 18:14:15 | sean-k-mooney | https://github.com/openstack/glance/blob/master/etc/metadefs/compute-libvirt-image.json#L111 | |
| 18:14:31 | EugenMayer | that is pure meta-data,right? | |
| 18:15:02 | sean-k-mooney | it will change the xml that is generated | |
| 18:15:13 | sean-k-mooney | that is how you change the xml in a supported way | |
| 18:15:37 | sean-k-mooney | that and flaovr extra specs but devicve models are genrealy only tunable via the image | |
| 18:16:08 | EugenMayer | ah the flavor can stear that? That's fine, i create a dedicated flavor via terraform anyway | |
| 18:16:22 | sean-k-mooney | a flaovr cannot change that option | |
| 18:16:27 | sean-k-mooney | it can change some others | |
| 18:16:38 | EugenMayer | so it must be an image option only, ok | |
| 18:16:39 | sean-k-mooney | everything in that file is setable via the glance image only | |
| 18:16:50 | sean-k-mooney | yes the image used for the root disk | |
| 18:16:59 | sean-k-mooney | so your esxi image in this case | |
| 18:17:18 | sean-k-mooney | openstack is a cloud plathform rather then virtualisation solution | |
| 18:17:54 | opendevreview | Julia Kreger proposed openstack/nova master: Ironic - Handle instance host on rebalance https://review.opendev.org/c/openstack/nova/+/813897 | |
| 18:17:57 | sean-k-mooney | so you create high levle templatees via the flavor and imave that you then instnaciate many copies of ranter then creatin indivicual instances | |
| 18:18:10 | EugenMayer | i see, i try to adopt that. Can i manipualte an existing glance image or how would i introduce this? | |
| 18:18:28 | sean-k-mooney | modifying an exsiting image will not affect instnace already using it | |
| 18:18:32 | sean-k-mooney | but you can update it | |
| 18:18:44 | EugenMayer | sure, i will redo the instance. That is what i do with terraform anyhhow | |
| 18:18:53 | sean-k-mooney | genreeally if it widely used in production you will not modify it but its really a case by case decision | |
| 18:18:58 | EugenMayer | ok so it is API only i guess | |
| 18:19:30 | EugenMayer | is this an md-property? | |
| 18:19:51 | sean-k-mooney | when we create an image we "snapshot" or copy the exra specs form the flavor and metadta form the image and store a perinstance copy | |
| 18:20:01 | sean-k-mooney | that copy is then used for the lifetime of the instnace | |
| 18:20:28 | mloza | hi folks, i have two kind of computes, first one is configured to store boot disk in ceph and second is configured to store boot disk locally on the host. Both works as intendend but second one can be bypass if the user tries to boot the instance from a volume | |
| 18:20:32 | sean-k-mooney | yes these are image metadata properties | |
| 18:21:01 | sean-k-mooney | mloza: correct that is expect | |
| 18:21:28 | sean-k-mooney | if the user boots from a cinder volume that storage will always be used | |
| 18:21:58 | sean-k-mooney | so if you have a cinier providing storage via say iscsi | |
| 18:22:10 | sean-k-mooney | then the boot form volume instnace can boot on either host by default | |
| 18:22:26 | EugenMayer | sean-k-mooney: glance image-update bbcb16b8-8c84-4b01-92e8-6d66938460bf --property hw_vif_model=e1000 .. is taht what you mean by updating the image? | |
| 18:22:37 | sean-k-mooney | EugenMayer: yes exactly | |
| 18:22:58 | sean-k-mooney | if you create a new vm with that image then it should have e1000 nics instead of virtio | |