| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-09-19 | |||
| 17:05:38 | openstack | Launchpad bug 1703954 in OpenStack Compute (nova) "Attach/Detach encrypted volume problems with real paths" [Undecided,Incomplete] | |
| 17:07:23 | lyarwood | mriedem_afk: I'll post something in the morning to retry _get_device_link but in the longer term I think I need to sort out the symlinking in the encryptors such that it's safe to provide them with a real block device path and they don't do anything stupid. | |
| 17:07:41 | lyarwood | mriedem_afk: anyway both tasks for the morning \o_ | |
| 17:36:20 | mriedem | lyarwood: ack thanks for investigating, i'll open a bug (i found https://bugs.launchpad.net/os-brick/+bug/1820007 but looks old/different) and start tracking in e-r | |
| 17:36:21 | openstack | Launchpad bug 1820007 in os-brick "Failed to attach encrypted volumes after detach: volume device not found at /dev/disk/by-id" [Undecided,New] | |
| 17:36:29 | mriedem | it also only failed on ovh nodes so maybe those are slower here, idk | |
| 17:37:51 | mriedem | actually a lot of the details in that bug look the same so i'll just re-use that | |
| 18:10:15 | openstackgerrit | Walter A. Boring IV (hemna) proposed openstack/nova stable/pike: WIP: Avoid redundant initialize_connection on source post live migration https://review.opendev.org/683008 | |
| 18:35:25 | openstackgerrit | Merged openstack/nova master: Trigger real BuildAbortException during migrate with bandwidth https://review.opendev.org/682876 | |
| 20:15:17 | openstackgerrit | Merged openstack/nova master: Validate CPU config options against running instances https://review.opendev.org/680107 | |
| 20:15:26 | openstackgerrit | Merged openstack/nova master: objects: Add 'NUMACell.pcpuset' field https://review.opendev.org/680108 | |
| 20:15:36 | openstackgerrit | Merged openstack/nova master: hardware: Differentiate between shared and dedicated CPUs https://review.opendev.org/671800 | |
| 21:56:57 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add evacuate vs rebuild contributor doc https://review.opendev.org/683245 | |
| 21:57:03 | mriedem | efried_pto: you asked for it ^ dansmith your review on that would also be helpful due to your blog post | |
| 21:57:06 | mriedem | *the* blog post | |
| 22:00:59 | dansmith | mriedem: https://www.youtube.com/watch?v=gVpzrxjiAtc | |
| 22:01:29 | mriedem | coincidentally i'm wearing that same outfit | |
| 22:01:38 | mriedem | just went to the scarf store last night | |
| 22:01:47 | dansmith | I don't doubt it | |
| 22:01:58 | dansmith | coincidentally, I'm wearing nothing *but* that scarf | |
| 22:02:22 | mriedem | man, i can't imagine the amount of coke flowing through these guys in this video | |
| 22:03:15 | dansmith | yeah keith looks like he hasn't slept in two weeks | |
| 22:28:34 | openstackgerrit | Dustin Cowles proposed openstack/nova master: Provider Config File: YAML file loading and schema validation https://review.opendev.org/673341 | |
| 22:28:35 | openstackgerrit | Dustin Cowles proposed openstack/nova master: Provider Config File: Function to further validate and retrieve configs https://review.opendev.org/676029 | |
| 22:28:36 | openstackgerrit | Dustin Cowles proposed openstack/nova master: WIP: Provider Config File: Merge provider configs to provider tree https://review.opendev.org/676522 | |
| 23:15:41 | openstackgerrit | Eric Fried proposed openstack/nova master: libvirt: Enable driver configuring PMEM namespaces https://review.opendev.org/679640 | |
| #openstack-nova - 2019-09-20 | |||
| 00:21:22 | openstackgerrit | sean mooney proposed openstack/nova-specs master: resubmit image metadata prefiltering spec for ussuri https://review.opendev.org/683258 | |
| 08:59:52 | luyao | efried_pto, stephenfin: Hi, could you help review the vpmems doc if you get time, since you both are familiar with the vpmems series. :D I think I get the content close which is focusing on the current functionality. https://review.opendev.org/#/c/680300. | |
| 09:02:32 | luyao | And the patch 'objects: use all_things_equal from objects.base' is ready to merge , need +W on it, https://review.opendev.org/#/c/681397/13 | |
| 09:07:35 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: Add a prelude for the Train release https://review.opendev.org/683327 | |
| 09:59:32 | AdamMork | Hello, friends! I have a openstack contains 2 controllers an 4 compute node's. I need to enable AES-NI for one or more vm. Openstack deployed via kolla and work on docker containers(nova container, neutron container etc. on controller). I read manual https://software.intel.com/en-us/articles/openstack-epa-feature-breakdown-and-analysis . On the | |
| 09:59:33 | AdamMork | compute node (in bios i enable AES-NI) . But in vm use command (cat /proc/cpuinfo | grep aes) i not found aes support. How to configure nova to enable aes instructions? | |
| 10:28:35 | AdamMork | ok! How i can modify Nova conf "The Nova* libvirt driver takes its configuration information from a section in the main Nova file /etc/nova/nova.conf." I have docker containers and how to find /etc/nova/nova.conf.? should i connect to nova container? | |
| 11:28:54 | AdamMork | ok. I see more manuals and understand : in globals.yml need uncomment :node_custom_config: "/etc/kolla/config" and overrides basic config for nova (As of now kolla only supports config overrides for ini based configs.) https://github.com/openstack/kolla-ansible/blob/master/doc/source/admin/advanced-configuration.rst THX | |
| 12:54:46 | mriedem | gibi: i saw your resize reschedule rpc pin bug, do you know if that's a new regression in train? | |
| 12:55:28 | gibi | mriedem: not yet. I'm about to push a reproduction patch, then I will look into when the fault was introduced | |
| 12:55:57 | gibi | it is clear that the problem is with the legacy request spec in prep_resize ending up in the conductor during a re-schedule with rpc pinned to 5.0 | |
| 12:58:38 | gibi | mriedem: this is where it blows https://github.com/openstack/nova/blob/9b2e00e015f22b2d876cd3c239af8e139040c8c8/nova/conductor/manager.py#L327 | |
| 12:58:51 | mriedem | i know that in compute rpc api 5.1 we send the RequestSpec to compute https://opendev.org/openstack/nova/src/branch/master/nova/compute/rpcapi.py#L833 | |
| 12:58:58 | mriedem | but backlevel if to the dict form if we can't | |
| 12:59:13 | mriedem | gibi: ah yeah - that's my patch | |
| 12:59:15 | mriedem | remember? | |
| 12:59:34 | mriedem | https://review.opendev.org/#/c/680762/ | |
| 13:00:03 | mriedem | so if you have a recreate we can lay that on top | |
| 13:00:43 | gibi | mriedem: ohh you have a fix for it. cool. Yeah I will push a functional repro soon and the you can rebase | |
| 13:00:50 | gibi | or I can rebase | |
| 13:00:55 | gibi | your series | |
| 13:01:00 | mriedem | you can rebase it, | |
| 13:01:06 | mriedem | also, i've duplicated your bug against 1843090 | |
| 13:01:17 | gibi | mriedem: thanks. I will update my patch accordingly | |
| 13:01:42 | gibi | mriedem: I reached this bug while tried to create a func test for the bandwidth case when rpc in pinned | |
| 13:02:06 | gibi | mriedem: it seems there will also be extra issues | |
| 13:02:07 | mriedem | yeah it was this patch that made me think of it in your bw series for prep_resize where you were using the request spec | |
| 13:03:36 | gibi | mriedem: btw do you know about a bug regading booting a instance with --availablity-zone az:node and then migrate it. I saw that in this case nova does not try to re-schedule the migration if the first dest host fails in prep_resize | |
| 13:03:57 | gibi | it seem the that filter_properties are not populated for re-schedule | |
| 13:04:17 | gibi | if I boot the server with the new host parameter then the re-schedule work | |
| 13:05:18 | mriedem | gibi: that's working as designed https://opendev.org/openstack/nova/src/branch/master/nova/scheduler/utils.py#L801 | |
| 13:05:44 | mriedem | using az::node will set force_nodes | |
| 13:06:07 | mriedem | do you see "Re-scheduling is disabled due to forcing a host" in the logs? | |
| 13:06:13 | gibi | mriedem: but forcing the host during the boot allows me to migrate it without dest host specified, but it does not allow nova to re-schedule | |
| 13:06:27 | gibi | during that migrate | |
| 13:07:06 | gibi | mriedem: I have to go back and re-create the situation as I saw this while I created the func test for the rpc pin bug. | |
| 13:07:26 | mriedem | gibi: which release? https://review.opendev.org/#/q/I3f488be6f3c399f23ccf2b9ee0d76cd000da0e3e | |
| 13:07:47 | gibi | mriedem: master :) | |
| 13:07:54 | mriedem | oh nvm that's ignore_hosts | |
| 13:08:46 | mriedem | so i think force_hosts/nodes gets persisted for some reason, i'm not sure why, but for every move operation we call this to basically unset those fields https://opendev.org/openstack/nova/src/branch/master/nova/objects/request_spec.py#L692 | |
| 13:08:55 | mriedem | it would be simpler if we just didn't persist those values | |
| 13:09:24 | mriedem | gibi: so maybe we're calling ^ *after* populate_retry | |
| 13:10:03 | mriedem | yup https://opendev.org/openstack/nova/src/branch/master/nova/conductor/tasks/migrate.py#L293 | |
| 13:11:29 | gibi | mriedem: so is the order of pupulate and rest wrong there? | |
| 13:11:53 | mriedem | it appears so | |
| 13:11:54 | gibi | s/pupulate/populate_retry | |
| 13:12:07 | gibi | mriedem: OK I will create a reproduction for that as well | |
| 13:12:10 | mriedem | i'm checking if that's a regression | |
| 13:13:37 | mriedem | gibi: looks like it's been that way since newton https://review.opendev.org/#/c/284974/18/nova/conductor/tasks/migrate.py | |
| 13:14:02 | gibi | mriedem: then nobody cares :D | |
| 13:14:42 | mriedem | ha | |
| 13:14:51 | gibi | mriedem: btw there are re-schedule func tests that uses az:node boot, so those will start behaving differently if we fix this | |
| 13:14:52 | mriedem | or nobody just ever thought to ask if that's wrong | |
| 13:15:19 | mriedem | dansmith: bauzas: can you think of any rason why we persist RequestSpec.force_hosts/nodes? | |
| 13:15:31 | mriedem | like many of the other request spec fields, persisting that seems to only cause headaches | |
| 13:15:34 | bauzas | PEBKAC ? | |
| 13:16:00 | bauzas | I thought we have a method for this | |
| 13:16:13 | mriedem | i understand that when request spec was originally written everything was persisted, but we've rolled back a lot of that piece by piece as we find problems | |
| 13:16:24 | mriedem | bauzas: yeah, but that's kind of ... dumb, right? | |
| 13:16:30 | bauzas | mriedem: https://github.com/openstack/nova/blob/master/nova/objects/request_spec.py#L692 | |
| 13:16:36 | mriedem | we could just not persist the damn field and then we wouldn't have to explicitly reset for every move operation | |
| 13:16:38 | bauzas | mriedem: yeah, this | |
| 13:17:09 | bauzas | mriedem: when alaski provided the persistence for the spec, his and me forgot to discuss about this :( | |
| 13:17:15 | bauzas | him* | |
| 13:17:22 | bauzas | so we created this method | |
| 13:18:21 | mriedem | ok but going forward, to avoid new ways of shooting off our toes, we could just stop persisting force_hosts/nodes and only rely on that method as a workaround for existing request specs | |
| 13:19:39 | mriedem | these are the fields i count that we've changed to not be persisted on the requestspec: ignore_hosts, requested_destination, retry, network_metadata, requested_resources | |
| 13:20:08 | mriedem | the latter 2 are newer and were not persisted since they were introduced, but the first 3 were all retroactive | |
| 13:20:39 | mriedem | oh and the instance_group members/hosts https://github.com/openstack/nova/blob/master/nova/objects/request_spec.py#L614 | |
| 13:21:20 | mriedem | which reminds me, if any stable core cares about pike and ocata https://review.opendev.org/#/q/topic:bug/1830747+status:open | |
| 13:21:37 | bauzas | mriedem: cool with this | |
| 13:22:10 | bauzas | mriedem: I was also thinking about some way to say which specific fields shouldn't be persisted | |
| 13:22:31 | bauzas | mriedem: and sure, will look at the stable changes | |
| 13:23:40 | mriedem | gibi: as for existing functional resize reschedule tests using az::node, i'd have to see the impacts to assess really | |