| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-09-19 | |||
| 22:01:38 | mriedem | just went to the scarf store last night | |
| 22:01:47 | dansmith | I don't doubt it | |
| 22:01:58 | dansmith | coincidentally, I'm wearing nothing *but* that scarf | |
| 22:02:22 | mriedem | man, i can't imagine the amount of coke flowing through these guys in this video | |
| 22:03:15 | dansmith | yeah keith looks like he hasn't slept in two weeks | |
| 22:28:34 | openstackgerrit | Dustin Cowles proposed openstack/nova master: Provider Config File: YAML file loading and schema validation https://review.opendev.org/673341 | |
| 22:28:35 | openstackgerrit | Dustin Cowles proposed openstack/nova master: Provider Config File: Function to further validate and retrieve configs https://review.opendev.org/676029 | |
| 22:28:36 | openstackgerrit | Dustin Cowles proposed openstack/nova master: WIP: Provider Config File: Merge provider configs to provider tree https://review.opendev.org/676522 | |
| 23:15:41 | openstackgerrit | Eric Fried proposed openstack/nova master: libvirt: Enable driver configuring PMEM namespaces https://review.opendev.org/679640 | |
| #openstack-nova - 2019-09-20 | |||
| 00:21:22 | openstackgerrit | sean mooney proposed openstack/nova-specs master: resubmit image metadata prefiltering spec for ussuri https://review.opendev.org/683258 | |
| 08:59:52 | luyao | efried_pto, stephenfin: Hi, could you help review the vpmems doc if you get time, since you both are familiar with the vpmems series. :D I think I get the content close which is focusing on the current functionality. https://review.opendev.org/#/c/680300. | |
| 09:02:32 | luyao | And the patch 'objects: use all_things_equal from objects.base' is ready to merge , need +W on it, https://review.opendev.org/#/c/681397/13 | |
| 09:07:35 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: Add a prelude for the Train release https://review.opendev.org/683327 | |
| 09:59:32 | AdamMork | Hello, friends! I have a openstack contains 2 controllers an 4 compute node's. I need to enable AES-NI for one or more vm. Openstack deployed via kolla and work on docker containers(nova container, neutron container etc. on controller). I read manual https://software.intel.com/en-us/articles/openstack-epa-feature-breakdown-and-analysis . On the | |
| 09:59:33 | AdamMork | compute node (in bios i enable AES-NI) . But in vm use command (cat /proc/cpuinfo | grep aes) i not found aes support. How to configure nova to enable aes instructions? | |
| 10:28:35 | AdamMork | ok! How i can modify Nova conf "The Nova* libvirt driver takes its configuration information from a section in the main Nova file /etc/nova/nova.conf." I have docker containers and how to find /etc/nova/nova.conf.? should i connect to nova container? | |
| 11:28:54 | AdamMork | ok. I see more manuals and understand : in globals.yml need uncomment :node_custom_config: "/etc/kolla/config" and overrides basic config for nova (As of now kolla only supports config overrides for ini based configs.) https://github.com/openstack/kolla-ansible/blob/master/doc/source/admin/advanced-configuration.rst THX | |
| 12:54:46 | mriedem | gibi: i saw your resize reschedule rpc pin bug, do you know if that's a new regression in train? | |
| 12:55:28 | gibi | mriedem: not yet. I'm about to push a reproduction patch, then I will look into when the fault was introduced | |
| 12:55:57 | gibi | it is clear that the problem is with the legacy request spec in prep_resize ending up in the conductor during a re-schedule with rpc pinned to 5.0 | |
| 12:58:38 | gibi | mriedem: this is where it blows https://github.com/openstack/nova/blob/9b2e00e015f22b2d876cd3c239af8e139040c8c8/nova/conductor/manager.py#L327 | |
| 12:58:51 | mriedem | i know that in compute rpc api 5.1 we send the RequestSpec to compute https://opendev.org/openstack/nova/src/branch/master/nova/compute/rpcapi.py#L833 | |
| 12:58:58 | mriedem | but backlevel if to the dict form if we can't | |
| 12:59:13 | mriedem | gibi: ah yeah - that's my patch | |
| 12:59:15 | mriedem | remember? | |
| 12:59:34 | mriedem | https://review.opendev.org/#/c/680762/ | |
| 13:00:03 | mriedem | so if you have a recreate we can lay that on top | |
| 13:00:43 | gibi | mriedem: ohh you have a fix for it. cool. Yeah I will push a functional repro soon and the you can rebase | |
| 13:00:50 | gibi | or I can rebase | |
| 13:00:55 | gibi | your series | |
| 13:01:00 | mriedem | you can rebase it, | |
| 13:01:06 | mriedem | also, i've duplicated your bug against 1843090 | |
| 13:01:17 | gibi | mriedem: thanks. I will update my patch accordingly | |
| 13:01:42 | gibi | mriedem: I reached this bug while tried to create a func test for the bandwidth case when rpc in pinned | |
| 13:02:06 | gibi | mriedem: it seems there will also be extra issues | |
| 13:02:07 | mriedem | yeah it was this patch that made me think of it in your bw series for prep_resize where you were using the request spec | |
| 13:03:36 | gibi | mriedem: btw do you know about a bug regading booting a instance with --availablity-zone az:node and then migrate it. I saw that in this case nova does not try to re-schedule the migration if the first dest host fails in prep_resize | |
| 13:03:57 | gibi | it seem the that filter_properties are not populated for re-schedule | |
| 13:04:17 | gibi | if I boot the server with the new host parameter then the re-schedule work | |
| 13:05:18 | mriedem | gibi: that's working as designed https://opendev.org/openstack/nova/src/branch/master/nova/scheduler/utils.py#L801 | |
| 13:05:44 | mriedem | using az::node will set force_nodes | |
| 13:06:07 | mriedem | do you see "Re-scheduling is disabled due to forcing a host" in the logs? | |
| 13:06:13 | gibi | mriedem: but forcing the host during the boot allows me to migrate it without dest host specified, but it does not allow nova to re-schedule | |
| 13:06:27 | gibi | during that migrate | |
| 13:07:06 | gibi | mriedem: I have to go back and re-create the situation as I saw this while I created the func test for the rpc pin bug. | |
| 13:07:26 | mriedem | gibi: which release? https://review.opendev.org/#/q/I3f488be6f3c399f23ccf2b9ee0d76cd000da0e3e | |
| 13:07:47 | gibi | mriedem: master :) | |
| 13:07:54 | mriedem | oh nvm that's ignore_hosts | |
| 13:08:46 | mriedem | so i think force_hosts/nodes gets persisted for some reason, i'm not sure why, but for every move operation we call this to basically unset those fields https://opendev.org/openstack/nova/src/branch/master/nova/objects/request_spec.py#L692 | |
| 13:08:55 | mriedem | it would be simpler if we just didn't persist those values | |
| 13:09:24 | mriedem | gibi: so maybe we're calling ^ *after* populate_retry | |
| 13:10:03 | mriedem | yup https://opendev.org/openstack/nova/src/branch/master/nova/conductor/tasks/migrate.py#L293 | |
| 13:11:29 | gibi | mriedem: so is the order of pupulate and rest wrong there? | |
| 13:11:53 | mriedem | it appears so | |
| 13:11:54 | gibi | s/pupulate/populate_retry | |
| 13:12:07 | gibi | mriedem: OK I will create a reproduction for that as well | |
| 13:12:10 | mriedem | i'm checking if that's a regression | |
| 13:13:37 | mriedem | gibi: looks like it's been that way since newton https://review.opendev.org/#/c/284974/18/nova/conductor/tasks/migrate.py | |
| 13:14:02 | gibi | mriedem: then nobody cares :D | |
| 13:14:42 | mriedem | ha | |
| 13:14:51 | gibi | mriedem: btw there are re-schedule func tests that uses az:node boot, so those will start behaving differently if we fix this | |
| 13:14:52 | mriedem | or nobody just ever thought to ask if that's wrong | |
| 13:15:19 | mriedem | dansmith: bauzas: can you think of any rason why we persist RequestSpec.force_hosts/nodes? | |
| 13:15:31 | mriedem | like many of the other request spec fields, persisting that seems to only cause headaches | |
| 13:15:34 | bauzas | PEBKAC ? | |
| 13:16:00 | bauzas | I thought we have a method for this | |
| 13:16:13 | mriedem | i understand that when request spec was originally written everything was persisted, but we've rolled back a lot of that piece by piece as we find problems | |
| 13:16:24 | mriedem | bauzas: yeah, but that's kind of ... dumb, right? | |
| 13:16:30 | bauzas | mriedem: https://github.com/openstack/nova/blob/master/nova/objects/request_spec.py#L692 | |
| 13:16:36 | mriedem | we could just not persist the damn field and then we wouldn't have to explicitly reset for every move operation | |
| 13:16:38 | bauzas | mriedem: yeah, this | |
| 13:17:09 | bauzas | mriedem: when alaski provided the persistence for the spec, his and me forgot to discuss about this :( | |
| 13:17:15 | bauzas | him* | |
| 13:17:22 | bauzas | so we created this method | |
| 13:18:21 | mriedem | ok but going forward, to avoid new ways of shooting off our toes, we could just stop persisting force_hosts/nodes and only rely on that method as a workaround for existing request specs | |
| 13:19:39 | mriedem | these are the fields i count that we've changed to not be persisted on the requestspec: ignore_hosts, requested_destination, retry, network_metadata, requested_resources | |
| 13:20:08 | mriedem | the latter 2 are newer and were not persisted since they were introduced, but the first 3 were all retroactive | |
| 13:20:39 | mriedem | oh and the instance_group members/hosts https://github.com/openstack/nova/blob/master/nova/objects/request_spec.py#L614 | |
| 13:21:20 | mriedem | which reminds me, if any stable core cares about pike and ocata https://review.opendev.org/#/q/topic:bug/1830747+status:open | |
| 13:21:37 | bauzas | mriedem: cool with this | |
| 13:22:10 | bauzas | mriedem: I was also thinking about some way to say which specific fields shouldn't be persisted | |
| 13:22:31 | bauzas | mriedem: and sure, will look at the stable changes | |
| 13:23:40 | mriedem | gibi: as for existing functional resize reschedule tests using az::node, i'd have to see the impacts to assess really | |
| 13:23:59 | bauzas | mriedem: efried_pto: btw. https://review.opendev.org/#/c/683327/ | |
| 13:24:02 | mriedem | but if we ignore the forced host/node to boot the server during resize scheduling, it seems we should allow rescheduling to alternates | |
| 13:26:12 | gibi | mriedem: sure. we will see when I managed to write the reproduction and reorder the populate_retry call | |
| 13:27:04 | mriedem | stephenfin: are you working on an admin guide docs patch for pcpu? | |
| 13:27:12 | stephenfin | mriedem: yup | |
| 13:27:16 | stephenfin | atm, in fact | |
| 13:27:39 | stephenfin | should have something ready to go before EOD | |
| 13:35:59 | mriedem | bauzas: notes on the prelude patch | |
| 13:36:06 | bauzas | cool | |
| 13:40:26 | mriedem | weee https://zuul.opendev.org/t/openstack/build/e3f767dbd32247ffa5aa7daa79e0e5af/log/job-output.txt#32748 | |
| 13:40:31 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Func test for migrate reschedule with pinned compute rpc https://review.opendev.org/683385 | |
| 13:41:21 | gibi | mriedem: reproduction for 1843090 ^^ | |
| 13:41:52 | gibi | mriedem: now I'm going to do the rebase of your fix on it | |
| 13:42:15 | mriedem | gibi: when you rebase you might as well strike whatever word efried_pto didn't like from my comment | |
| 13:42:23 | gibi | mriedem: sure thing | |
| 13:52:39 | mriedem | gibi: comments in your functional recreate patch | |
| 13:54:59 | gibi | mriedem: thanks. regarding the request_spec uglyness. I can cook up something that is test only. But sooner or later we (I) need to go back and fix this uglyness as it is now needed in two places (as noted) | |