| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-04-16 | |||
| 18:55:37 | imacdonn | Last time I tried, it was required... whether or not it *should* ...... | |
| 18:56:05 | melwitt | thanks y'all | |
| 19:09:35 | mriedem | imacdonn: well i'm unable to recreate your issue in a functional test but i found a new regression, 500 in the api | |
| 19:18:04 | cdent | mriedem: you're so good at that | |
| 19:20:13 | imacdonn | mriedem: hmm, I was just pondering if maybe the marker instance gets deleted when the last "real" instance for a project is deleted ... in testing that, I just got a "ClientException: Unknown Error (HTTP 504)" - not sure if related | |
| 19:20:21 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add regression test for bug 1825034 https://review.openstack.org/653098 | |
| 19:20:22 | openstack | bug 1825034 in OpenStack Compute (nova) "listing deleted servers from the API fails after running fill_virtual_interface_list online data migration" [High,Confirmed] https://launchpad.net/bugs/1825034 | |
| 19:20:22 | mriedem | imacdonn: would be interested to know why my recreate steps for *your* bug don't hit here ^ | |
| 19:21:27 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add regression test for bug 1825034 https://review.openstack.org/653098 | |
| 19:21:28 | openstack | bug 1825034 in OpenStack Compute (nova) "listing deleted servers from the API fails after running fill_virtual_interface_list online data migration" [High,Confirmed] https://launchpad.net/bugs/1825034 | |
| 19:21:56 | mriedem | imacdonn: the marker instance is soft deleted as soon as it's created | |
| 19:22:27 | mriedem | https://github.com/openstack/nova/blob/master/nova/objects/virtual_interface.py#L308 | |
| 19:23:25 | imacdonn | mriedem: ah, right, so it probably requires archival to have happened to reproduce my original problem | |
| 19:23:35 | mriedem | that's what my functional test does | |
| 19:23:39 | mriedem | but it doesn't hit your issue | |
| 19:24:10 | imacdonn | does it create the two null rows in security_groups ? | |
| 19:24:25 | gmann | looking for review on these 2 specs - https://review.openstack.org/#/c/603969/ https://review.openstack.org/#/c/547850/ | |
| 19:24:48 | gmann | mriedem: would you like to give second round review on this (API cleanup) - https://review.openstack.org/#/c/603969 | |
| 19:24:49 | mriedem | imacdonn: that i don't know - this is also sqlite so i'm not sure if sqlite is more strict about null values in constraints than mysql (that would be funny if it is) | |
| 19:25:15 | mriedem | gmann: it's in the queue somewhere | |
| 19:25:36 | gmann | ok, thanks | |
| 19:26:47 | imacdonn | mriedem: I wouldn't be surprised if sqlite observes null when checking for unique constraint (which we already established that mysql does not) | |
| 19:29:08 | imacdonn | mriedem: OTOH https://sqlite.org/faq.html#q26 seems to suggest otherwise | |
| 19:31:49 | mriedem | imacdonn: i added this to the test and the test passes http://paste.openstack.org/show/749386/ | |
| 19:33:50 | imacdonn | mriedem: is this test running with the admin context ? | |
| 19:34:09 | imacdonn | mriedem: i.e. the one that has no project_id | |
| 19:34:25 | mriedem | ctxt.project_id is an admin context with no project_id, | |
| 19:34:31 | mriedem | self.api.project_id is a non-null value | |
| 19:35:46 | melwitt | gmann: those are in my queue too | |
| 19:39:16 | gmann | melwitt: thanks. | |
| 19:39:41 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add post-test wrinkle to list deleted servers before archive https://review.openstack.org/653131 | |
| 19:44:55 | imacdonn | mriedem: FWIW, I can reproduce my original problem with this sequence: 1) create an instance 2) run migrations 3) archive 4) run migrations | |
| 19:46:01 | mriedem | imacdonn: that's what my functional test does though | |
| 19:46:07 | mriedem | but let me try in devstack | |
| 19:46:19 | mriedem | the neutron fixture in the functional test is likely not creating any virtual interface records | |
| 19:46:21 | imacdonn | mriedem: yeah, except sqlite vs. mysql .... or .... ? | |
| 19:46:26 | mriedem | sure | |
| 19:46:29 | mriedem | i'll spin up a devstack | |
| 19:46:51 | mriedem | also need to get up and stretch the legs and get some coffee | |
| 19:48:29 | mriedem | sean-k-mooney: dansmith: btw that mitaka->newton regression mentioned yesterday is a thing, i reported a bug https://bugs.launchpad.net/nova/+bug/1825018 | |
| 19:48:30 | openstack | Launchpad bug 1825018 in OpenStack Compute (nova) "security group driver gets loaded way too much in the api" [Low,Triaged] | |
| 20:28:26 | mnaser | mriedem: backporting fixed it cleanly. | |
| 20:28:54 | mnaser | I have an abandoned backport if anyone wants to cherry pick cause it applies cleanly right now | |
| 20:29:33 | mnaser | Left a comment too so if someone finds the bug and sees the proposed but then abandoned patch, they’ll see some useful info there | |
| 20:30:01 | openstackgerrit | melanie witt proposed openstack/nova master: Add get_counts() to InstanceMappingList https://review.openstack.org/638072 | |
| 20:30:01 | openstackgerrit | melanie witt proposed openstack/nova master: Count instances from mappings and cores/ram from placement https://review.openstack.org/638073 | |
| 20:30:02 | openstackgerrit | melanie witt proposed openstack/nova master: Use instance mappings to count server group members https://review.openstack.org/638324 | |
| 20:30:02 | openstackgerrit | melanie witt proposed openstack/nova master: Add get_usages_counts_for_quota to SchedulerReportClient https://review.openstack.org/653145 | |
| 20:30:03 | openstackgerrit | melanie witt proposed openstack/nova master: Set [quota]count_usage_from_placement = True in nova-next https://review.openstack.org/653146 | |
| 20:30:16 | mriedem | mnaser: ah so the heal task fixed up the network info cache? | |
| 20:30:20 | mriedem | on that rocky cloud | |
| 20:30:35 | mnaser | Yep. Just watched them slowly get repopulated | |
| 20:30:39 | mriedem | nice | |
| 20:30:45 | mnaser | Had to apply on all computes tho which was annoying but yeah | |
| 20:31:01 | mriedem | btw, another reason to not backport that data migration, i just found this https://bugs.launchpad.net/nova/+bug/1824435 | |
| 20:31:03 | openstack | Launchpad bug 1824435 in OpenStack Compute (nova) stein "fill_virtual_interface_list migration fails on second attempt" [High,Triaged] | |
| 20:31:05 | mnaser | I probably will redeploy again without it.. if they disappear again.. will raise question | |
| 20:31:17 | mnaser | Yeah I saw that too as well. Never ran into that though.. yet :X | |
| 20:31:46 | mriedem | mnaser: run this as admin on your stein cloud: openstack server list --all-projects --deleted | |
| 20:32:17 | mnaser | o should I be worried about that | |
| 20:32:25 | mriedem | running the command or the bug? | |
| 20:32:34 | mnaser | The command | |
| 20:32:38 | mriedem | it's read-only | |
| 20:32:46 | mnaser | As in is there some magical bug we’re about to discover | |
| 20:32:59 | mriedem | heh i've already discovered the bug | |
| 20:33:01 | mriedem | i think you already have it | |
| 20:33:18 | mnaser | But I can do that later, I also can do it both on a cloud that has db archive enabled and disabled too | |
| 20:33:59 | mriedem | sure the workaround is to archive, but you have to do it after running that migration every time | |
| 20:39:10 | efried | aspiers: Still hanging around? | |
| 20:41:05 | mriedem | imacdonn: efried: yup, recreated http://paste.openstack.org/show/749391/ | |
| 20:41:20 | efried | woot, in a func test? | |
| 20:41:28 | mriedem | no, devstack | |
| 20:41:38 | openstackgerrit | melanie witt proposed openstack/nova master: Count instances from mappings and cores/ram from placement https://review.openstack.org/638073 | |
| 20:41:38 | openstackgerrit | melanie witt proposed openstack/nova master: Set [quota]count_usage_from_placement = True in nova-next https://review.openstack.org/653146 | |
| 20:41:39 | openstackgerrit | melanie witt proposed openstack/nova master: Use instance mappings to count server group members https://review.openstack.org/638324 | |
| 20:48:20 | efried | mriedem: Well, if you can do it in devstack, you can at least mock it in a func test I guess. | |
| 20:49:19 | mriedem | not necessarily - func test is using sqlite | |
| 20:49:22 | mriedem | devstack is using mysql | |
| 20:51:09 | mriedem | but https://sqlite.org/faq.html#q26 suggests sqlite and mysql have the same behavior about how nulls are handled in unique constraints | |
| 20:51:25 | mriedem | efried: but my functional test is doing the same steps https://review.openstack.org/#/c/653098/2/nova/tests/functional/regressions/test_bug_1825034.py | |
| 20:59:43 | imacdonn | mriedem: I think it'd be interesting to see if your sqlite db has the duplicate rows after the migration is run the first time ... is it possible to query that? Not sure what conditions the tests run under.... | |
| 21:00:47 | mriedem | i've added an assertion for that locally and it's just returning 1 security group for the null project_id | |
| 21:01:39 | imacdonn | OK, so that FAQ is wrong .. or we're misinterpreting it | |
| 21:04:52 | openstackgerrit | Dakshina Ilangovan proposed openstack/nova-specs master: Resource Management Daemon - Last Level Cache https://review.openstack.org/651233 | |
| 21:05:30 | openstackgerrit | Dustin Cowles proposed openstack/nova master: Introduces the openstacksdk to nova https://review.openstack.org/643664 | |
| 21:12:55 | openstackgerrit | Dakshina Ilangovan proposed openstack/nova-specs master: Resource Management Daemon - Last Level Cache https://review.openstack.org/651233 | |
| 21:22:05 | aspiers | efried: back | |
| 21:22:18 | aspiers | although it's getting late-ish here | |
| 21:24:48 | cfriesen | anyone here know OVMF? Looks like centos 7.6 has modified the OVMF-20180508-3 rpm to no longer contain the file /usr/share/OVMF/OVMF_CODE.fd that nova looks for in nova/virt/libvirt/driver.py. Instead it now seems to be named /usr/share/OVMF/OVMF_CODE.secboot.fd | |
| 21:24:57 | openstackgerrit | Dakshina Ilangovan proposed openstack/nova-specs master: Resource Management Daemon - Last Level Cache https://review.openstack.org/651233 | |
| 21:28:51 | mnaser | mriedem: is there a patch/fix for the `openstack server list --all-projects --deleted` thing? | |
| 21:35:46 | imacdonn | mnaser, discussion at https://bugs.launchpad.net/nova/+bug/1825034 , I suppose | |
| 21:35:46 | openstack | Launchpad bug 1825034 in OpenStack Compute (nova) stein "listing deleted servers from the API fails after running fill_virtual_interface_list online data migration" [High,Confirmed] | |
| 21:38:01 | mriedem | mnaser: i don't have a fix no | |
| 21:38:05 | mriedem | the workaround is to archive | |
| 21:38:34 | mriedem | i put some thoughts into the bug report but they kind of all suck | |
| 21:38:50 | mnaser | yeah, I went through it, none are really ideal | |
| 21:39:06 | mnaser | is --deleted every supposed to actually return data? | |
| 21:39:15 | mriedem | yeah | |
| 21:39:18 | mriedem | until you archive | |