Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-05-13
15:50:49 gibi correct (shit, another edge case)
15:52:08 dansmith so here's a couple of less impactful options:
15:52:56 dansmith 1. Each time we cache an image, or run the periodic, we generate a disk allocation for the compute node uuid which consumes inventory according to how much space the cache is using
15:53:04 dansmith 2. Same as above, but adjust the reserved amount
15:53:50 dansmith both cases need to consider the case where the _base is not on the same filesystem as the instances, but there's less synchronization involved, and we're not spraying tons of new allocations into place
15:54:34 dansmith we could also make it a workaround that you opt into in the short term to see how it goes, because cleanup from it would be much easier (just nuke one $cnuuid allocation)
15:55:18 dansmith and we could make compute node startup nuke that allocation if present and the workaround is disabled (or the self-correcting reserved amount)
15:55:54 gibi right, I would use allocation instead of reserved as allocation has a consumer attached
15:56:16 gibi reserved would be a sum of configured + detected
15:56:30 gibi which is math, that I dont like :)
15:56:31 dansmith hopefully glance gives us enough information to be able to increase that allocation before we start the download, so we know "oh sorry, inventory says we don't have room for this base image, so fail()"
15:56:52 dansmith I know it would, but I think reserved would be more obvious to an operator
15:57:00 dansmith even though it's a composite value
15:57:20 dansmith just because listing allocations are a bunch of meaningless-to-the-human UUIDs
15:57:46 dansmith I'm not arguing for that, I'm just saying there're benefits both ways
15:57:57 gibi I see. yes it is a tradeoff
15:58:37 dansmith I gotta get on a call
15:58:41 dansmith food for thought
15:59:42 gibi dansmith: thanks! I appreciate your help
16:00:40 gibi I've checked. glance give use the physical size of the image on the API. Which is good for ensuring we have still disk for the download
16:02:14 gibi also I think even if nova is converting a qcow2 glace image to raw locally (due to force_raw_images config) the resulting raw file is sparse
16:02:34 gibi but it might dependent on the host OS + file system support
16:26:39 gibi dansmith: if we start allocating / reserving the cache disk usage in placement then do we still need a pre-filter as well? For me it is OK to simply let the boot fail on the compute side if the extra DISK_GB resource for the cache cannot be allocated?
16:27:02 gibi s/?//
16:27:21 gibi anyhow documented your idea in the bug report
16:27:55 dansmith gibi: weighing aside, any compute with enough space for an instance but not enough for the instance+image will become a magnet for new builds, which will all fail
16:28:42 dansmith so, yeah, we can just pretend that's not worth solving, but it also sucks because avoiding that is what we're trying to solve with placement
16:31:22 gibi OK, I can imagine this as a two step solution then. First the allocation / reservation management in the compute then second the pre-filter that uses instance + image DISK_GB request but allocate only the instance disk for the instance_uuid in placement
16:32:52 dansmith perhaps
16:40:04 openstackgerrit melanie witt proposed openstack/nova master: DNM Try out running sphinx-build in parallel for releasenotes https://review.opendev.org/727429
16:48:40 openstackgerrit Balazs Gibizer proposed openstack/nova master: WIP: allow disabling image cache for raw images https://review.opendev.org/727261
17:23:58 openstackgerrit melanie witt proposed openstack/nova master: DNM Try out running sphinx-build in parallel for releasenotes https://review.opendev.org/727429
19:05:50 openstackgerrit Jiri Suchomel proposed openstack/nova master: Add ability to download Glance images into the libvirt image cache via RBD https://review.opendev.org/574301
20:20:37 openstackgerrit melanie witt proposed openstack/nova master: DNM Try out running sphinx-build in parallel for releasenotes https://review.opendev.org/727429
20:54:18 openstackgerrit Sean McGinnis proposed openstack/nova master: DNM: Test making EM branch release notes static https://review.opendev.org/727875
23:15:54 melwitt gmann: do you understand why even after version bump we can get pep8 errors in gate? https://review.opendev.org/727589
23:18:10 gmann melwitt: it would be an error now as hacking min version bump will stop the new checks added in flake8 3.8.0 version.
23:18:28 gmann this one- https://review.opendev.org/#/c/727347/1
23:19:15 melwitt gmann: yeah but... (sorry) I thought bumping the version _stops_ the new checks from getting pulled in>
23:19:26 melwitt s/>/?/
23:19:45 gmann sorry *would not be an error
23:19:53 gmann missing *not*
23:20:31 melwitt ok, makes sense. so why second patch needed? is it nice to have for future flake8 or something>
23:20:36 melwitt gah I keep hitting >
23:21:10 gmann yeah for future but we will not be able to verify it or all error till we have new hacking pulling new checks.
23:21:47 melwitt yeah, ok.
23:21:51 gmann I am going to release the new hacking version 4.0.0 which will pull new checks and then in 727589 patch we will bump hacking version to so that we can verify the fix
23:22:38 gmann I mean fix + new hacking version bump in a single patch.
23:31:57 melwitt gmann: yeah makes sense. +2 on the 3.0.1 patch though I wondered if it's needed to avoid gate failures, would have thought we'd have a gate-failure bug around it. that is, it wasn't clear to me if the patch is needed to fix gate failure
23:34:24 gmann melwitt: I did nova patch earlier before i described the situation in other patches cmt msg like this- https://review.opendev.org/#/c/727576/
23:35:29 gmann melwitt: basically it will 1. fix local run where the latest fixed hacking 3.0.1 is not pulled automatically. in case of gate it is pulled as fresh installation. 2. it will protect if future flake8 3.9.0 version pull other new checks.
23:35:30 melwitt ah I see
23:35:55 melwitt oh ok so not a gate issue
23:36:04 melwitt cool thanks for explaining
23:36:12 melwitt I was confused before heh
23:37:05 gmann otherwise hacking before 3.0.1 is capping "flake8<4.0.0" so flake8 3.8.0 which ic current release and 3.9.0 future release both can be protected by bumping hacking to 3.0.1
23:37:22 melwitt right
23:37:59 melwitt yeah, so needed for future
23:39:46 gmann it is actually confusing as all inbuild deps actually. hacking does protect cap for flake8 and flake8 does protect cap for pycodestyle. but if pycodestyle is pulled as latest by some flake8 plugins then again it fail like flake8-import-order then we need pycodestyle cap explicitly. like tempest case- https://review.opendev.org/#/c/727276/2/test-requirements.txt
23:40:13 gmann so our way of handling the flake8 with plugins is not so great.
23:40:22 melwitt wow. mind blown
23:41:24 gmann i added PR on flake8-import-order but not sure if they will accept or not with our hacky use case :) - https://github.com/PyCQA/flake8-import-order/pull/172
23:41:56 melwitt ah, good luck :)
#openstack-nova - 2020-05-14
00:46:44 openstackgerrit Wenping Song proposed openstack/nova master: delete sub resource provider when delete resource provider https://review.opendev.org/719163
00:49:25 openstackgerrit melanie witt proposed openstack/nova master: Configure latex_engine = 'xelatex' for pdf docs build https://review.opendev.org/727898
01:07:14 openstackgerrit melanie witt proposed openstack/nova master: Configure latex_engine = 'xelatex' for pdf docs build https://review.opendev.org/727898
01:32:15 openstackgerrit Merged openstack/nova-specs master: Re-propose provider-config-file spec for Victoria https://review.opendev.org/725788
01:36:51 openstackgerrit Merged openstack/nova-specs master: Re-proposes 'Proposal for a safer remote console with password authentication' https://review.opendev.org/719720
02:48:28 tony_su gibi, sean-k-mooney : Thanks for reviewing and merging the spec.
02:50:43 tony_su gibi, sean-k-mooney : I will start to rebase related patches and send out for code review once they are ready.
03:03:55 melwitt fyi the openstack-tox-docs job is failing for nova since a recent upper-constraints bump, I have proposed a fix here https://review.opendev.org/727898 I have also sent a ML post
05:56:38 openstackgerrit melanie witt proposed openstack/nova master: Configure latex_engine = 'xelatex' for pdf docs build https://review.opendev.org/727898
07:10:38 openstackgerrit Nalini Varshney proposed openstack/nova master: Add migration to make key field type VARBINARY in aggregate_metadata table, https://review.opendev.org/725522
07:35:02 brinzhang_ sean-k-mooney: https://review.opendev.org/#/c/727589/2 do I need to update this patch? I have not concern stephenfin has already fixed these issue, but he abandoned.
07:49:09 gibi good morning Nova
07:49:29 bauzas whoops my manners
07:49:34 bauzas good morning Nova
07:50:26 brinzhang_ morning nova
07:50:59 brinzhang_ hi gibi, bauzas, I saw you are -1 for https://review.opendev.org/#/c/727140/1, the flake8 error fix patch
07:51:44 gibi brinzhang_: my -1 is about the fact that 3.8.1 added yet another check that needs to be fixed
07:51:48 brinzhang_ I submit a same one, before that I have not concern stephenfin's patch, do I need to continue ?
07:52:24 brinzhang_ gibi: I will see http://paste.openstack.org/show/793434/
07:52:40 bauzas brinzhang_: my concerns was about making sure what we would need to change
07:52:49 gibi I guess stephenfin abandoned the patch as we went a different direction, capping the version
07:53:34 brinzhang_ yeah
07:53:55 brinzhang_ I mean, if there is no sense to fix, I think I would abandon too.
07:56:16 brinzhang_ https://review.opendev.org/#/c/727347/ gmann bumped hacking to 3.0.1, I am not sure whether this can avoid these pep8 error happen again
07:58:30 brinzhang_ gibi: stephenfin and I fixed the issue is reported by https://review.opendev.org/#/c/724260/6, and https://review.opendev.org/#/c/726564/2.
07:59:12 brinzhang_ I found your paste is not easy to fix, may need to change the function name, such as "./nova/tests/unit/api/openstack/compute/microversions.py:34:5: F811 redefinition of unused 'index' from line 29"
07:59:18 brinzhang_ :P
08:02:05 gibi brinzhang_: those redefinitions are intentional and not cause a python bug due to the decorator top of it will generate unique functions dynamically. So I think F811 needs to be ignored in tox.ini
08:03:01 gibi I think it is OK to fix the issues reported by the other new checks along with version bump
08:03:09 gibi but please consult stephenfin too
08:03:54 brinzhang_ gibi: agree
08:04:49 brinzhang_ gibi: yeah, I will try to update the patch to latest, and then consult
08:05:32 gibi OK, cool
08:20:58 openstackgerrit Brin Zhang proposed openstack/nova master: Fix flake8 error from hacking <3.0.1 https://review.opendev.org/727589
08:45:27 bauzas brinzhang_: gibi: FWIW, +Wd https://review.opendev.org/#/c/727347/1
08:45:57 bauzas brinzhang_: but then I don't understand why we should merge https://review.opendev.org/#/c/727589/
08:46:04 gibi bauzas: thanks

Earlier   Later