| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-04-11 | |||
| 16:06:27 | mriedem | i couldn't reproduce the original bug in the unit test | |
| 16:08:53 | stephenfin | mriedem: Yup, that looks good to me. Good find with those click docs | |
| 16:13:10 | mriedem | need a stable core to hit these https://review.openstack.org/#/q/topic:bug/1821824+branch:stable/stein | |
| 16:15:20 | lyarwood | mriedem: have the branch open to review now btw | |
| 16:54:56 | openstackgerrit | Merged openstack/nova master: devstack: Remove 'tempest-dsvm-tempest-xen-rc' https://review.openstack.org/650018 | |
| 17:14:07 | openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif master: Remove IP proxy methods https://review.openstack.org/643115 | |
| 17:14:13 | openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif master: Refactor functional base test classes https://review.openstack.org/643101 | |
| 17:14:33 | lyarwood | mriedem: https://review.openstack.org/#/q/status:open+topic:bug/1803961 - Would you mind taking a look at this again when you have time. I'm going to suggest that we land and backport this over the competing cinder fix for the time being. | |
| 17:17:47 | openstackgerrit | Merged openstack/nova master: trivial: Remove dead nova.db functions https://review.openstack.org/649570 | |
| 17:48:46 | mriedem | lyarwood: couldn't get jgriffith or another cinder person to look at https://review.openstack.org/#/c/637224/ ? | |
| 17:51:53 | lyarwood | mriedem: I did a while ago and they pointed me towards https://review.openstack.org/#/c/638995/ but that has now stalled and I'm thinking it might just be easier to fix and backport in Nova given the cinder change is touching lots of different backends. | |
| 17:52:23 | lyarwood | mriedem: I can ask again to get confirmation that using migration_status is okay with them as a workaround until ^ lands. | |
| 18:22:20 | mriedem | melwitt: thinking out loud about quota and cross-cell resize and your placement change, when a user resizes a server today, placement will track vcpu/ram usage against both the source and dest node, but the /limits API will only show vcpu/ram usage for the new flavor since that is counted from the instance right? | |
| 18:22:57 | mriedem | so i think if i'm resizing from a flavor with vcpu=2 to vcpu=4, usage from placement for that project will say a total of 6, but the compute limits API would say 4 | |
| 18:23:49 | mriedem | i'm not necessarily saying that's wrong, but is that accurate? | |
| 18:24:41 | melwitt | yeah, it's definitely not going to say 6, but I don't remember if it will say 2 or 4 before the resize is confirmed | |
| 18:24:57 | mriedem | placement would say 6 | |
| 18:25:14 | melwitt | yeah | |
| 18:25:53 | melwitt | looking at the code to see when the new flavor is saved to the Instance object vcpus and memory_mb attributes | |
| 18:25:59 | mriedem | once the server is resized the limits api would say 4 https://github.com/openstack/nova/blob/03322bb517925a9f5a04ebdb41c3fd31e7962440/nova/objects/instance.py#L1535 | |
| 18:26:07 | melwitt | those attributes are what's counted today | |
| 18:26:38 | mriedem | it's finish_resize on the dest host https://github.com/openstack/nova/blob/03322bb517925a9f5a04ebdb41c3fd31e7962440/nova/compute/manager.py#L4632 | |
| 18:26:49 | mriedem | same method that changes the instance status to VERIFY_RESIZE | |
| 18:27:44 | melwitt | ah ok | |
| 18:28:34 | mriedem | maybe this is part of why we were talking about adding consumer types to placement so nova could ask for usage for 'instances' to filter out the usage tracked by the migration record holding the old flavor usage | |
| 18:30:22 | mriedem | anyway, it's a thing for cross-cell resize because while the instance is in VERIFY_RESIZE status we'll have the instance in both the source and target dbs and if we're counting from the dbs we don't want to double count the instance, so i need to filter out the hidden one, | |
| 18:30:30 | mriedem | but that got me thinking about how vcpus/ram will be counted | |
| 18:31:15 | melwitt | yeah. food for thought | |
| 18:31:30 | mriedem | i'm assuming no one wants to think about that or eat that food though | |
| 18:31:37 | melwitt | of course not | |
| 18:33:13 | openstackgerrit | Merged openstack/nova stable/rocky: doc: Fix openstack CLI command https://review.openstack.org/648425 | |
| 18:33:21 | openstackgerrit | Merged openstack/nova stable/rocky: doc: Capitalize keystone domain name https://review.openstack.org/650601 | |
| 18:33:40 | melwitt | this was brought up before in earlier discussions about counting from placement I think, and IIRC some (dansmith?) thought if placement is consuming 6 resources at that point in time, it makes sense for quota usage counting to reflect that as well | |
| 18:34:13 | dansmith | it depends on the resource and the direction | |
| 18:34:27 | melwitt | when do the old allocations go away from placement? VERIFY_RESIZE or after CONFIRM_RESIZE? | |
| 18:34:28 | dansmith | ideally we would only consume max(old_cpus, new_cpus) from quota | |
| 18:34:36 | dansmith | (and placement | |
| 18:34:41 | dansmith | because that's all we need to revert | |
| 18:34:51 | dansmith | but potentially we need to claim sum(old_disk, new_disk) depending | |
| 18:35:03 | melwitt | I see | |
| 18:35:04 | dansmith | and always both if we're not on the same node of course | |
| 18:35:49 | mriedem | right it's a mess and resize to same host doesn't help https://bugs.launchpad.net/nova/+bug/1790204 | |
| 18:35:51 | openstack | Launchpad bug 1790204 in OpenStack Compute (nova) "Allocations are "doubled up" on same host resize even though there is only 1 server on the host" [High,Triaged] | |
| 18:36:14 | mriedem | melwitt: the old allocations go away on confirm | |
| 18:36:24 | melwitt | ack | |
| 18:36:55 | mriedem | it's probably fair to say that our internal resource tracking and quota usage reporting during resize have just never aligned | |
| 18:37:13 | mriedem | the resource tracker would always report usage for the old flavor on the source node while resized to save room for a revert, | |
| 18:37:20 | mriedem | but the quota usage wouldn't reflect that | |
| 18:37:47 | mriedem | i don't know if the old pre-counting reservations stuff held some quota during a resize or not | |
| 18:38:17 | mriedem | i.e. did we hold a reservation on the target host and then /limits + reserved would report 6 in this scenario rather than 4, idk | |
| 18:38:22 | mriedem | no one would probably notice | |
| 18:44:23 | melwitt | mriedem: doesn't look like it. from this, you would fail quota check if you didn't have room to revert https://github.com/openstack/nova/blob/stable/ocata/nova/compute/api.py#L3201 | |
| 18:48:10 | mriedem | that code is all black magic to me, i'd have to setup an ocata devstack to see what happens in the API | |
| 18:48:30 | mriedem | definitely not a priority, was just thinking about it while i write a functional test for this for my cross-cell series | |
| 18:48:32 | melwitt | it does reserve for the new flavor here though at the beginning of the resize https://github.com/openstack/nova/blob/stable/ocata/nova/compute/api.py#L3351 | |
| 18:50:05 | melwitt | if it's an upsize | |
| 18:53:31 | melwitt | so from what I can tell, it will consume quota usage for the new flavor as soon as the resize starts | |
| 18:53:57 | melwitt | only for an upsize | |
| 18:54:21 | melwitt | otherwise it will consume for the old flavor still | |
| 18:55:22 | mriedem | ok so pre-pike the RT would report usage for both the old and new flavor and /limits API would show usage for the old and new flavor (assuming upsize), and starting in pike we'll only report usage for the new flavor | |
| 18:55:28 | mriedem | which could also be a downsize | |
| 18:55:44 | mriedem | maybe i drop vcpu but bump disk or something | |
| 19:00:37 | melwitt | no, I think the limits API would show the usage for only the new flavor because it would be upsize_delta + current usage (old flavor) | |
| 19:00:43 | melwitt | pre-pike | |
| 19:01:34 | melwitt | sorry I wasn't clear that it reserves the delta between old and new | |
| 19:21:11 | mriedem | incoming | |
| 19:21:47 | openstackgerrit | Merged openstack/nova stable/rocky: Add functional regression test for bug 1669054 https://review.openstack.org/649325 | |
| 19:21:48 | openstack | bug 1669054 in OpenStack Compute (nova) rocky "RequestSpec.ignore_hosts from resize is reused in subsequent evacuate" [Medium,In progress] https://launchpad.net/bugs/1669054 - Assigned to Matt Riedemann (mriedem) | |
| 19:22:04 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Improve CinderFixtureNewAttachFlow https://review.openstack.org/639382 | |
| 19:22:04 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix ProviderUsageBaseTestCase._run_periodics for multi-cell https://review.openstack.org/641179 | |
| 19:22:05 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove unused context parameter from RT._get_instance_type https://review.openstack.org/641792 | |
| 19:22:05 | openstack | bug 1818914 in OpenStack Compute (nova) "Hypervisor resource usage on source still shows old flavor usage after resize confirm until update_available_resource periodic runs" [Low,In progress] https://launchpad.net/bugs/1818914 - Assigned to Matt Riedemann (mriedem) | |
| 19:22:05 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add functional recreate test for bug 1818914 https://review.openstack.org/641521 | |
| 19:22:06 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add Migration.cross_cell_move and get_by_uuid https://review.openstack.org/614012 | |
| 19:22:06 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Update usage in RT.drop_move_claim during confirm resize https://review.openstack.org/641806 | |
| 19:22:07 | openstackgerrit | Matt Riedemann proposed openstack/nova master: DNM: Add instance hard delete https://review.openstack.org/650984 | |
| 19:22:07 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add InstanceAction/Event create() method https://review.openstack.org/614036 | |
| 19:22:08 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add TargetDBSetupTask https://review.openstack.org/627892 | |
| 19:22:08 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add Instance.hidden field https://review.openstack.org/631123 | |
| 19:22:09 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Execute TargetDBSetupTask https://review.openstack.org/633853 | |
| 19:22:09 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add CrossCellMigrationTask https://review.openstack.org/631581 | |
| 19:22:10 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_dest compute method https://review.openstack.org/633293 | |
| 19:22:10 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add can_connect_volume() compute driver method https://review.openstack.org/621313 | |
| 19:22:11 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_source compute method https://review.openstack.org/634832 | |
| 19:22:11 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add PrepResizeAtDestTask https://review.openstack.org/627890 | |
| 19:22:12 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add PrepResizeAtSourceTask https://review.openstack.org/627891 | |
| 19:22:12 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add nova.compute.utils.delete_image https://review.openstack.org/637605 | |
| 19:22:24 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Revert cross-cell resize from the API https://review.openstack.org/638048 | |
| 19:22:24 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add revert_snapshot_based_resize conductor RPC method https://review.openstack.org/638047 | |
| 19:22:25 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add archive_deleted_rows wrinkle to cross-cell functional test https://review.openstack.org/651650 | |
| 19:22:25 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Confirm cross-cell resize while deleting a server https://review.openstack.org/638268 | |
| 19:22:26 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add cross-cell resize policy rule and enable in API https://review.openstack.org/638269 | |
| 19:22:26 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add CrossCellWeigher https://review.openstack.org/614353 | |
| 19:22:35 | efried | sean-k-mooney: you still awake? | |
| 19:36:40 | melwitt | efried: fyi, patch to add a few post-release items to the ptl guide https://review.openstack.org/651009 | |
| 19:40:55 | efried | melwitt: ack, on the radar, thank you. | |
| 19:41:52 | efried | oh, that was easy. +2 | |
| 20:00:09 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: [DNM: extra logs] Revert resize: wait for external events in compute manager https://review.openstack.org/644881 | |
| 20:08:10 | mriedem | heh i didn't realize there was a ptl guide | |
| 20:14:28 | openstackgerrit | Merged openstack/python-novaclient master: Add test for console-log and docs for bug 1746534 https://review.openstack.org/651827 | |