| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-04-16 | |||
| 20:30:01 | openstackgerrit | melanie witt proposed openstack/nova master: Count instances from mappings and cores/ram from placement https://review.openstack.org/638073 | |
| 20:30:01 | openstackgerrit | melanie witt proposed openstack/nova master: Add get_counts() to InstanceMappingList https://review.openstack.org/638072 | |
| 20:30:02 | openstackgerrit | melanie witt proposed openstack/nova master: Add get_usages_counts_for_quota to SchedulerReportClient https://review.openstack.org/653145 | |
| 20:30:02 | openstackgerrit | melanie witt proposed openstack/nova master: Use instance mappings to count server group members https://review.openstack.org/638324 | |
| 20:30:03 | openstackgerrit | melanie witt proposed openstack/nova master: Set [quota]count_usage_from_placement = True in nova-next https://review.openstack.org/653146 | |
| 20:30:16 | mriedem | mnaser: ah so the heal task fixed up the network info cache? | |
| 20:30:20 | mriedem | on that rocky cloud | |
| 20:30:35 | mnaser | Yep. Just watched them slowly get repopulated | |
| 20:30:39 | mriedem | nice | |
| 20:30:45 | mnaser | Had to apply on all computes tho which was annoying but yeah | |
| 20:31:01 | mriedem | btw, another reason to not backport that data migration, i just found this https://bugs.launchpad.net/nova/+bug/1824435 | |
| 20:31:03 | openstack | Launchpad bug 1824435 in OpenStack Compute (nova) stein "fill_virtual_interface_list migration fails on second attempt" [High,Triaged] | |
| 20:31:05 | mnaser | I probably will redeploy again without it.. if they disappear again.. will raise question | |
| 20:31:17 | mnaser | Yeah I saw that too as well. Never ran into that though.. yet :X | |
| 20:31:46 | mriedem | mnaser: run this as admin on your stein cloud: openstack server list --all-projects --deleted | |
| 20:32:17 | mnaser | o should I be worried about that | |
| 20:32:25 | mriedem | running the command or the bug? | |
| 20:32:34 | mnaser | The command | |
| 20:32:38 | mriedem | it's read-only | |
| 20:32:46 | mnaser | As in is there some magical bug we’re about to discover | |
| 20:32:59 | mriedem | heh i've already discovered the bug | |
| 20:33:01 | mriedem | i think you already have it | |
| 20:33:18 | mnaser | But I can do that later, I also can do it both on a cloud that has db archive enabled and disabled too | |
| 20:33:59 | mriedem | sure the workaround is to archive, but you have to do it after running that migration every time | |
| 20:39:10 | efried | aspiers: Still hanging around? | |
| 20:41:05 | mriedem | imacdonn: efried: yup, recreated http://paste.openstack.org/show/749391/ | |
| 20:41:20 | efried | woot, in a func test? | |
| 20:41:28 | mriedem | no, devstack | |
| 20:41:38 | openstackgerrit | melanie witt proposed openstack/nova master: Set [quota]count_usage_from_placement = True in nova-next https://review.openstack.org/653146 | |
| 20:41:38 | openstackgerrit | melanie witt proposed openstack/nova master: Count instances from mappings and cores/ram from placement https://review.openstack.org/638073 | |
| 20:41:39 | openstackgerrit | melanie witt proposed openstack/nova master: Use instance mappings to count server group members https://review.openstack.org/638324 | |
| 20:48:20 | efried | mriedem: Well, if you can do it in devstack, you can at least mock it in a func test I guess. | |
| 20:49:19 | mriedem | not necessarily - func test is using sqlite | |
| 20:49:22 | mriedem | devstack is using mysql | |
| 20:51:09 | mriedem | but https://sqlite.org/faq.html#q26 suggests sqlite and mysql have the same behavior about how nulls are handled in unique constraints | |
| 20:51:25 | mriedem | efried: but my functional test is doing the same steps https://review.openstack.org/#/c/653098/2/nova/tests/functional/regressions/test_bug_1825034.py | |
| 20:59:43 | imacdonn | mriedem: I think it'd be interesting to see if your sqlite db has the duplicate rows after the migration is run the first time ... is it possible to query that? Not sure what conditions the tests run under.... | |
| 21:00:47 | mriedem | i've added an assertion for that locally and it's just returning 1 security group for the null project_id | |
| 21:01:39 | imacdonn | OK, so that FAQ is wrong .. or we're misinterpreting it | |
| 21:04:52 | openstackgerrit | Dakshina Ilangovan proposed openstack/nova-specs master: Resource Management Daemon - Last Level Cache https://review.openstack.org/651233 | |
| 21:05:30 | openstackgerrit | Dustin Cowles proposed openstack/nova master: Introduces the openstacksdk to nova https://review.openstack.org/643664 | |
| 21:12:55 | openstackgerrit | Dakshina Ilangovan proposed openstack/nova-specs master: Resource Management Daemon - Last Level Cache https://review.openstack.org/651233 | |
| 21:22:05 | aspiers | efried: back | |
| 21:22:18 | aspiers | although it's getting late-ish here | |
| 21:24:48 | cfriesen | anyone here know OVMF? Looks like centos 7.6 has modified the OVMF-20180508-3 rpm to no longer contain the file /usr/share/OVMF/OVMF_CODE.fd that nova looks for in nova/virt/libvirt/driver.py. Instead it now seems to be named /usr/share/OVMF/OVMF_CODE.secboot.fd | |
| 21:24:57 | openstackgerrit | Dakshina Ilangovan proposed openstack/nova-specs master: Resource Management Daemon - Last Level Cache https://review.openstack.org/651233 | |
| 21:28:51 | mnaser | mriedem: is there a patch/fix for the `openstack server list --all-projects --deleted` thing? | |
| 21:35:46 | openstack | Launchpad bug 1825034 in OpenStack Compute (nova) stein "listing deleted servers from the API fails after running fill_virtual_interface_list online data migration" [High,Confirmed] | |
| 21:35:46 | imacdonn | mnaser, discussion at https://bugs.launchpad.net/nova/+bug/1825034 , I suppose | |
| 21:38:01 | mriedem | mnaser: i don't have a fix no | |
| 21:38:05 | mriedem | the workaround is to archive | |
| 21:38:34 | mriedem | i put some thoughts into the bug report but they kind of all suck | |
| 21:38:50 | mnaser | yeah, I went through it, none are really ideal | |
| 21:39:06 | mnaser | is --deleted every supposed to actually return data? | |
| 21:39:15 | mriedem | yeah | |
| 21:39:18 | mriedem | until you archive | |
| 21:39:20 | mnaser | I don't remember the openstack api returning deleted records | |
| 21:39:24 | mnaser | but til I guess | |
| 21:39:34 | melwitt | I think that might be the only case when it does | |
| 21:39:38 | mriedem | there is no guarantee that you'll get results because it depends on how the cloud is setup to archive | |
| 21:39:42 | mnaser | of course | |
| 21:39:49 | mnaser | I just didn't know there was an actual api way | |
| 21:40:36 | mriedem | right, so the only way to hit this i think is to filter on all_tenants and deleted, which at least thankfully is admin-only | |
| 21:40:36 | melwitt | and I agree all of the options suck. I kind of liked the last option mriedem put on the bug but that doesn't help if the migration was only run once (and completed). or maybe the migration could hard delete the marker instance itself if it completed. still, if multiple runs are needed it sucks | |
| 21:40:40 | mriedem | could break some internal tools | |
| 21:40:44 | mriedem | but shouldn't break external users | |
| 21:43:08 | mnaser | tbh | |
| 21:43:18 | mnaser | all_tenants+deleted will probably hurt a lot in a bigger environment anyways | |
| 21:43:40 | mriedem | as in spanking your dbs? | |
| 21:44:30 | imacdonn | or just making a lot of output | |
| 21:45:07 | mnaser | both | |
| 21:45:12 | imacdonn | up to api.max_limit, I guess | |
| 21:45:15 | mnaser | hits the db hard, hits tons of apis hard too | |
| 21:45:26 | melwitt | yeah, was gonna say max_limit save the day | |
| 21:45:57 | mnaser | yeah we have "don't do it™" rule but that might be a good stop gap | |
| 21:51:05 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Exclude fake marker instance when listing servers https://review.openstack.org/653158 | |
| 21:51:09 | mriedem | well here is one option ^ | |
| 21:52:20 | imacdonn | the fact that the fake UUID is defined in virtual_interface makes it slightly more icky :/ | |
| 21:53:35 | mriedem | mnaser: btw, remind me to bring this up the next time you ask that the online data migrations use markers to be more efficient :) | |
| 21:53:50 | mnaser | mriedem: bahaha | |
| 21:54:03 | mnaser | I think I liked the idea jaypipes proposed of using different migration repos like keystone does | |
| 21:54:10 | mnaser | but I think that means you can't batch them | |
| 21:54:11 | melwitt | lol, touche | |
| 21:54:47 | dansmith | mnaser: the point of that suggestion was to *not* batch them | |
| 21:54:59 | dansmith | mnaser: it also doesn't really work well for translations we have to do in python, which is a lot of them | |
| 21:55:15 | mnaser | ah yes I see | |
| 21:55:18 | mnaser | its a lot of rebuilding data | |
| 21:55:19 | dansmith | it just helps us avoid re-running them on presumed idempotentcy | |
| 21:55:28 | mnaser | rather than drop column for add column | |
| 21:55:31 | dansmith | and we could solve that pretty easily with something else | |
| 21:55:39 | mnaser | a .... marker? | |
| 21:55:40 | mnaser | :-P | |
| 21:55:46 | dansmith | doesn't have to be in-band | |
| 21:55:57 | mriedem | speaking of online data migrations, want to drop this old one now? https://review.openstack.org/#/c/651001/ | |
| 21:56:03 | mnaser | yeah | |
| 21:56:06 | dansmith | just a feature flag sort of "I've converted all the flavors, stop asking me" | |
| 21:56:08 | dansmith | problem is, | |
| 21:56:13 | mriedem | we could also be better about following up and cleaning these things up | |
| 21:56:28 | dansmith | if you end up with some older services by accident, you create old data, and stop running the migrations afterwards with no way of cleaning them up | |
| 21:56:37 | dansmith | mriedem: yup | |