| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-09-18 | |||
| 11:02:11 | stephenfin | I thought an instance was either persistent or transient | |
| 11:02:24 | stephenfin | *domain | |
| 11:03:45 | openstack | bugzilla.redhat.com bug 1861142 in openstack-nova "[OSP 13.0.z] Unable to detach volume from instance when previously removed from the inactive config" [High,Verified] - Assigned to lyarwood | |
| 11:03:45 | lyarwood | stephenfin: https://bugzilla.redhat.com/show_bug.cgi?id=1861142 for an example btw | |
| 11:05:23 | stephenfin | fun | |
| 11:05:27 | lyarwood | so much | |
| 11:13:58 | sean-k-mooney | how would people feel about backporting droping the persistent domain? is that doable or no? | |
| 11:14:12 | sean-k-mooney | im not sure how that and rolling upgrades would interact | |
| 11:14:15 | stephenfin | let's get it in master before we even think about | |
| 11:14:16 | stephenfin | that | |
| 11:15:06 | sean-k-mooney | ya im just wondering for some of the downstream bugs. | |
| 11:15:20 | sean-k-mooney | although lyarwood you are fixign that bug via the events right | |
| 11:19:51 | lyarwood | We could for new domains | |
| 11:20:03 | lyarwood | we would just need to retain support for persisted domains for a while | |
| 11:20:07 | lyarwood | but that's the same in master | |
| 11:20:22 | lyarwood | actually I don't know if we could ever drop that support really | |
| 11:20:33 | sean-k-mooney | we should be able too | |
| 11:20:37 | lyarwood | unless we wait for major OS changes | |
| 11:20:41 | lyarwood | like bionic to focal | |
| 11:20:44 | lyarwood | that would need LM anyway | |
| 11:21:20 | lyarwood | that would need an upgrade check on the computes | |
| 11:21:42 | sean-k-mooney | so if all new domains are trasiant. and we make all move operations transitant and all hard reboots convert them to transient | |
| 11:21:44 | lyarwood | so ops could be told something like `you're upgrading Nova to a version without support for persistent domains, LM instances from this compute host now before moving` | |
| 11:21:55 | sean-k-mooney | then i think we only need 1 or 2 releases to drop it | |
| 11:22:01 | lyarwood | yeah that's cool | |
| 11:22:26 | sean-k-mooney | we would want a nova status check or something | |
| 11:22:40 | sean-k-mooney | to say hay these vms still need to be moved | |
| 11:23:21 | lyarwood | yup | |
| 11:25:10 | sean-k-mooney | from a down stream persective it woudl be nice if this was in wallaby | |
| 11:25:33 | sean-k-mooney | since the upgrade to wallaby will be an FFU and will involve migrations | |
| 11:25:46 | lyarwood | yup | |
| 11:26:04 | kashyap | sean-k-mooney: I'm on the Focal box you gave me; I'm doing a bunch of tests in it, I hope it won't disappear under my feet? :-) | |
| 11:27:19 | sean-k-mooney | kashyap: not unless you or lyarwood delete it | |
| 11:28:48 | sean-k-mooney | i wont be doing any maintance on it until kolla releases victoria or i hit a kernel issue that requires a reboot | |
| 11:29:01 | kashyap | sean-k-mooney: Ah-ha, thank you | |
| 11:29:02 | sean-k-mooney | its the same cloud i do all my dev on too | |
| 11:29:15 | kashyap | And thanks for doing the gratis maintenance of this cloud for us | |
| 11:29:33 | kashyap | I feel like I should be paying you out in a beverage of your choice, for using your electricity remotely | |
| 11:30:10 | sean-k-mooney | its going to be running anyway | |
| 11:30:29 | sean-k-mooney | at some point im probaly going to redploy a limited third party ci on it again | |
| 11:32:02 | vinay_m | stephenfin : gone through bug comments ,discussed like it would be sphinx version related issue | |
| 11:32:07 | sean-k-mooney | when i do ill proably try and replace fort nebula for multi numa testing of the whitebox plugin | |
| 11:33:28 | vinay_m | i would like to explore how a search ref link working ,added in nova/doc/source/index.rst , how can i ? | |
| 11:34:25 | sean-k-mooney | vinay_m: you can propose a patch via gerrit and the docs job will render a preview sight | |
| 11:34:31 | sean-k-mooney | you can also build the docs locally | |
| 11:34:35 | sean-k-mooney | using tox | |
| 11:34:40 | sean-k-mooney | if you donwload the repos | |
| 11:35:04 | sean-k-mooney | i think that woudl be enouch to test the search capablity | |
| 11:40:29 | openstackgerrit | Lee Yarwood proposed openstack/nova master: zuul: Introduce nova-evacuate https://review.opendev.org/744883 | |
| 11:40:30 | openstackgerrit | Lee Yarwood proposed openstack/nova master: zuul: Replace nova-live-migration with zuulv3 jobs https://review.opendev.org/752557 | |
| 11:42:26 | lyarwood | gmann: https://review.opendev.org/#/c/752641/ would you mind taking a look at this if you have time | |
| 11:54:19 | openstackgerrit | Lee Yarwood proposed openstack/nova-specs master: WIP/DNM - Image defined ephemeral storage encryption https://review.opendev.org/752284 | |
| 11:54:52 | ygk_12345 | hi all | |
| 11:55:03 | ygk_12345 | i am facing a unique problem with rocky | |
| 11:56:14 | ygk_12345 | when I create an instance, it is having 4 vnics inside and they are getting ips from four dhcp tenant networks. but after sometime the vm is lossing its two network interfaces | |
| 11:57:00 | ygk_12345 | so when I rebooth it then, then the vnics are coming back but again after 100 secs they are disappearing again . is this a knownw issue ? that image worked earlier | |
| 12:01:25 | sean-k-mooney | it sounds like you have a neutron issue | |
| 12:02:06 | sean-k-mooney | ygk_12345: if neutron is retruning incosistent data then the force refesh of the netwrok info cache will result in the cache being populated with that inconsitent data | |
| 12:02:35 | sean-k-mooney | we have seen this downstream with contrail and aci before | |
| 12:02:49 | sean-k-mooney | i have not see it be an issue with intree network backends | |
| 12:03:04 | ygk_12345 | sean-k-mooney I see these messages in the compute node where that instance is | |
| 12:03:07 | ygk_12345 | Received unexpected event network-vif-plugged-86f1034d-837d-4e67-ad5e-63d9642a0b2a for instance with vm_state active and task_state None. | |
| 12:03:34 | ygk_12345 | Received unexpected event network-vif-unplugged-86f1034d-837d-4e67-ad5e-63d9642a0b2a for instance with vm_state active and task_state None. | |
| 12:04:14 | sean-k-mooney | what neutron backend are you using | |
| 12:04:20 | ygk_12345 | ovs | |
| 12:04:25 | ygk_12345 | ovs | |
| 12:04:27 | sean-k-mooney | with iptables or contrack | |
| 12:04:29 | ygk_12345 | ovs | |
| 12:04:46 | sean-k-mooney | for the firewall driver | |
| 12:04:56 | ygk_12345 | how to check it ? | |
| 12:05:13 | sean-k-mooney | /etc/neutron/plugins/ml2/ml2_conf.ini | |
| 12:05:22 | ygk_12345 | sean-k-mooney let me check | |
| 12:06:00 | ygk_12345 | sean-k-mooney firewall_driver = openvswitch | |
| 12:06:07 | sean-k-mooney | ok that is conntrack | |
| 12:06:44 | ygk_12345 | sean-k-mooney so whats the workaround for this issue ? | |
| 12:07:00 | sean-k-mooney | you have not provided enough info to know what the issue is yet | |
| 12:07:10 | ygk_12345 | why i s the vif intermittently loosing connection and dropping ? | |
| 12:07:43 | sean-k-mooney | those event wont casue the vif to be removed form the guest | |
| 12:08:02 | ygk_12345 | sean-k-mooney so what logs to check in particulaer ? | |
| 12:08:49 | sean-k-mooney | well first you need to look at the instance action logs for the affected instance and determin that no api action was performend on the guest | |
| 12:08:52 | ygk_12345 | sean-k-mooney its happening only with a particular image | |
| 12:09:18 | sean-k-mooney | the image cant affect this unless you are using the watchdog and the vm hung and was rebooted | |
| 12:09:26 | ygk_12345 | where can I find those logs ? | |
| 12:10:09 | sean-k-mooney | openstack server event list <uuid> | |
| 12:11:15 | ygk_12345 | no actions except two reboots which I did | |
| 12:11:28 | ygk_12345 | create,reboot,reboot | |
| 12:11:58 | sean-k-mooney | ok if there are no action at the time of the events you next need to look at the neutron agent logs | |
| 12:12:17 | sean-k-mooney | basically you need to check why the agent updated the port status | |
| 12:12:20 | ygk_12345 | which agent in particular ? | |
| 12:12:32 | ygk_12345 | dhcp ? | |
| 12:12:43 | sean-k-mooney | if it was devstack it would be devstack@q-agt but its the neutron ovs l2 agent | |
| 12:12:55 | ygk_12345 | ok | |
| 12:13:31 | sean-k-mooney | if nova is not perfroming any instance actions then the port vif:host-id should not be updating | |
| 12:14:05 | ygk_12345 | how can I track the port status of an instance in the ovs logs ? | |
| 12:14:14 | sean-k-mooney | so the only thing that would cause those event to be emitted is a change in state form the ovs agent | |
| 12:14:52 | sean-k-mooney | you need to have the neutron port uuid if the agent is in debug mode it will print the uuid of the port it is processign at differnt times | |
| 12:15:02 | ygk_12345 | ok let me check | |
| 12:15:14 | sean-k-mooney | what you are looking for is if the port is listed in teh added or deleted list of ports around teh time of the events | |
| 12:15:33 | sean-k-mooney | that would indicate the port being removed or added to the ovs bridge | |
| 12:15:50 | sean-k-mooney | simialrly if the admin state on the newon port was changed it would also show up there | |