Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-12-10
21:36:43 sean-k-mooney in that case it would seam to indicate taht OSP13 may have backported the stein change but unless that closed a bug im not sure why they would have
22:00:21 mriedem cfriesen: i've got something you might like to lose sleep over
22:00:36 cfriesen oh, goody
22:00:45 mriedem so RequestSpec has a limits field
22:00:55 mriedem which if we ever update that thing, it's on accident
22:01:09 mriedem e.g. if we resize from flavor1 to flavor2 and it's persisted, it's on accident,
22:01:19 mriedem and if we revert the resize back to flavor1, we definitely don't wipe out the limits
22:01:59 mriedem looking at something like unshelve, the limits passed down to compute to do the claim (which would only be used for pci/numa claims now) possibly does not reflect the actual flavor being used
22:02:01 openstackgerrit Jack Ding proposed openstack/nova master: [WIP] Flavor extra spec and image properties validation https://review.openstack.org/620706
22:03:15 mriedem actually unshelve might be safe here,
22:03:39 mriedem i was thinking unshelve would use stale limits https://github.com/openstack/nova/blob/08d617084e5aa69ada0898d674022621d130aef3/nova/conductor/manager.py#L813
22:04:02 mriedem but those get overwritten in the filter_properties dict here based on the selected host from the scheduler https://github.com/openstack/nova/blob/08d617084e5aa69ada0898d674022621d130aef3/nova/conductor/manager.py#L841
22:04:12 mriedem god it's a mess
22:04:18 mriedem tl;dr probably shouldn't ever trust RequestSpec.limits
22:04:26 mriedem and we probably shouldn't even persist it
22:04:45 cfriesen was just going to ask why it's persisted. :)
22:04:49 mriedem because limits are totally based on configured filters at any given time you scheduled
22:04:52 mriedem dansmith: ^
22:05:18 mriedem cfriesen: everything was persisted with the request spec and we've just be winding that back case by case ever since
22:05:55 mriedem because it's a turducken of original request / last flavor used but also as a glorified parameter bag to send requests to the scheduler
22:07:41 mriedem limits are just everywhere. the rpc cast to build_and_run_instance contains a (1) limits parameter, a (2) nested limits entry within filter_properties and (3) the RequestSpec.limits
22:07:44 mriedem it's *awesome*
22:09:10 cfriesen what the heck...NUMATopologyLimits contains cpu_allocation_ratio, ram_allocation_ratio, and "network_metadata"? how does that make any sense?
22:10:14 mriedem the network_metadata is the physnet/numa node stuff stephenfin added in rocky
22:10:22 cfriesen okay, so not crazy
22:12:40 cfriesen so it sounds like either we need to persist the limits properly and update them properly when they change, or else not persist them and create them before calling the scheduler. that sound right?
22:13:19 mriedem *create them *after* calling the scheduler, but yes
22:13:22 mriedem those are the options really
22:13:33 mriedem either never trust them across requests, or make sure they are always updated
22:13:40 mriedem i'd opt to never trust them
22:14:10 mriedem i.e. we can only revert the flavor in the request spec here https://github.com/openstack/nova/blob/08d617084e5aa69ada0898d674022621d130aef3/nova/compute/api.py#L3489 because the instance stashes the old_flavor
22:14:16 mriedem we don't stash the old limits anywhere
22:15:55 mriedem anyway, i just needed to dump my brain here while i was looking at this limits code to see if i could re-use it
22:20:02 cfriesen you've probably jinxed me now, and our test guys will be firing off an email with a limit-related failure
22:20:19 mriedem misery loves company
22:21:53 mriedem this is likely also semi broken depending on how you choose to manage allocation rations now, either via config (what this would use) or via placement api https://github.com/openstack/nova/blob/08d617084e5aa69ada0898d674022621d130aef3/nova/scheduler/filters/numa_topology_filter.py#L71
22:21:59 mriedem *ratios
22:22:33 mriedem if i'm managing VCPU/MEMORY_MB allocation ratios via the placement api, ^ will not reflect what is actually being used
22:23:17 mriedem that might have been true before placement if you were only managing allocation ratios in aggregate
22:27:37 dansmith mriedem: is there anything in reqspec that wasn't an ill-conceived steaming pile of poo?
22:28:49 mriedem umm
22:29:14 mriedem well, it is the only place we track that is_bfv thing!
22:29:30 mriedem otherwise it's caused probably more harm than it was worth
22:30:54 mriedem oh yeah and because of that numa filter limits those are per-host,
22:31:00 mriedem so RequestSpec.limits is really per operation
22:31:54 openstack bug 1806064 in OpenStack Compute (nova) rocky "Volume remains in attaching/reserved status, if the instance is deleted after TooManyInstances exception in nova-conductor" [Medium,In progress] https://launchpad.net/bugs/1806064 - Assigned to s10 (vlad-esten)
22:31:54 openstackgerrit Merged openstack/nova stable/rocky: Add functional regression test for bug 1806064 https://review.openstack.org/623931
22:31:56 mriedem alex also found out that we never actually update the numa/pci requests in the reqspec when we resize to another flavor that might have different numa/pci reqs https://review.openstack.org/#/c/621077/
22:32:02 mriedem just lots of fun
22:43:22 openstackgerrit Michael Still proposed openstack/nova master: Move nova.libvirt.utils away from using nova.utils.execute(). https://review.openstack.org/619703
22:43:22 openstackgerrit Michael Still proposed openstack/nova master: Remove utils.execute() from quobyte libvirt storage driver. https://review.openstack.org/619702
22:43:23 openstackgerrit Michael Still proposed openstack/nova master: Remove final users of utils.execute() in libvirt. https://review.openstack.org/619705
22:43:23 openstackgerrit Michael Still proposed openstack/nova master: Imagebackend should call processutils.execute directly. https://review.openstack.org/619704
22:43:24 openstackgerrit Michael Still proposed openstack/nova master: Remove utils.execute() from the hyperv driver. https://review.openstack.org/620008
22:43:24 openstackgerrit Michael Still proposed openstack/nova master: Remove the final user of utils.execute() from virt.images https://review.openstack.org/620007
22:43:25 openstackgerrit Michael Still proposed openstack/nova master: Move a generic bridge helper to a linux_net privsep file. https://review.openstack.org/620010
22:43:25 openstackgerrit Michael Still proposed openstack/nova master: Remove utils.execute() from virt.disk.api. https://review.openstack.org/620009
22:43:26 openstackgerrit Michael Still proposed openstack/nova master: Move some linux network helpers to use privsep. https://review.openstack.org/621398
22:43:26 openstackgerrit Michael Still proposed openstack/nova master: Move bridge creation to privsep. https://review.openstack.org/620180
22:43:27 openstackgerrit Michael Still proposed openstack/nova master: Move interface enabling to privsep. https://review.openstack.org/621528
22:43:27 openstackgerrit Michael Still proposed openstack/nova master: Move simple execute call to processutils. https://review.openstack.org/621527
22:43:28 openstackgerrit Michael Still proposed openstack/nova master: Move setting mac addresses for network devices to privsep. https://review.openstack.org/621529
22:43:29 openstackgerrit Michael Still proposed openstack/nova master: create_veth_pair is unused, remove it. https://review.openstack.org/624226
22:43:29 openstackgerrit Michael Still proposed openstack/nova master: Move binding ips to privsep. https://review.openstack.org/622151
22:43:29 openstackgerrit Michael Still proposed openstack/nova master: Move interface disabling to privsep. https://review.openstack.org/622150
22:43:30 openstackgerrit Michael Still proposed openstack/nova master: Move create_tap_dev into privsep. https://review.openstack.org/624228
22:43:30 openstackgerrit Michael Still proposed openstack/nova master: Create specialist set_macaddr_and_vlan helper. https://review.openstack.org/624227
22:44:26 mriedem so much for ever getting zuul to run tests on your change now
23:03:07 mriedem dansmith: i figured out how i'm getting 1 server out of the API when listing and there is a copy in both cells: https://github.com/openstack/nova/blob/08d617084e5aa69ada0898d674022621d130aef3/nova/compute/api.py#L2684
23:04:02 dansmith mriedem: oh is that the filter for the BR if the instance is already created?
23:04:09 mriedem yeah i think that's what it's for
23:04:14 mriedem laski added it
23:04:36 dansmith yeah, well, at least it makes sense
23:49:44 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: Cross-cell resize https://review.openstack.org/603930
23:58:12 mriedem jangutter: i'll star that offload spec but it's going to be greek to me
23:58:19 mriedem and i'll have to come back to it, done for the day
#openstack-nova - 2018-12-11
00:45:28 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (4) https://review.openstack.org/574106
00:45:37 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (5) https://review.openstack.org/574110
00:45:50 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (6) https://review.openstack.org/574113
00:46:01 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (7) https://review.openstack.org/574974
00:46:15 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (8) https://review.openstack.org/575311
00:56:24 openstackgerrit Merged openstack/nova stable/rocky: Create BDMs/tags in cell with instance when over-quota https://review.openstack.org/623932
01:23:36 openstackgerrit Merged openstack/nova master: Move nova-cells-v1 to experimental queue https://review.openstack.org/623538
01:38:08 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (9) https://review.openstack.org/575581
01:38:16 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (10) https://review.openstack.org/576017
01:38:25 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (11) https://review.openstack.org/576018
01:38:34 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (12) https://review.openstack.org/576019
01:38:44 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (13) https://review.openstack.org/576020
02:35:16 daikk115 Hi all, I got the error during live migration between two same CPU compute hosts (different BIOS versions lead to different CPU models: SandyBridge-IBRS and SandyBridge).
02:35:27 daikk115 How can I deal with this problem?
04:00:52 openstackgerrit Yikun Jiang proposed openstack/nova master: Remove live_migration_progress_timeout config https://review.openstack.org/619142
04:00:52 openstackgerrit Yikun Jiang proposed openstack/nova master: Add live migration timeout action https://review.openstack.org/619143
06:12:44 openstackgerrit Fan Zhang proposed openstack/nova master: Log disk transfer stats in live migration monitor. https://review.openstack.org/619395
06:46:56 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (14) https://review.openstack.org/576027
06:47:05 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (15) https://review.openstack.org/576031
06:47:16 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (16) https://review.openstack.org/576299
06:47:26 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (17) https://review.openstack.org/576344
06:47:37 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (18) https://review.openstack.org/576673
07:10:01 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (19) https://review.openstack.org/576676
07:10:11 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (20) https://review.openstack.org/576689

Earlier   Later