| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-07-31 | |||
| 14:35:19 | mdbooth | sean-k-mooney: Yep. IIRC that's what we discussed previously. | |
| 14:35:42 | sean-k-mooney | i think the backport is still on hold for the orginal change | |
| 14:36:03 | mriedem | dustinc: in case you haven't seen since i didn't vote, i've got some concerns in the base ironic+sdk patch https://review.opendev.org/#/c/642899/33 | |
| 14:36:15 | sean-k-mooney | i belive my change can be applied to stable/X without breaking anything then the other chagne can be stacked on top | |
| 14:36:16 | mriedem | ugly tracebacks in the logs when nova-compute starts up before ironic-api is available | |
| 14:37:59 | dustinc | mriedem: been a bit distracted but did notice and will be looking into it when I get some time. Much appreciated! | |
| 14:58:40 | prometheanfire | I suspect that the new lxml has some ordering issues or something | |
| 14:58:42 | prometheanfire | http://logs.openstack.org/60/673160/3/check/cross-nova-py36/547e78c/testr_results.html.gz | |
| 15:04:49 | openstackgerrit | Merged openstack/nova master: Fix AttributeError in RT._update_usage_from_migration https://review.opendev.org/667687 | |
| 15:04:59 | openstackgerrit | Merged openstack/nova master: Fix RT init arg order in test_unsupported_move_type https://review.opendev.org/667688 | |
| 15:15:31 | openstackgerrit | ZhengMa proposed openstack/nova master: Check container_format on instance creation https://review.opendev.org/673407 | |
| 15:19:12 | sean-k-mooney | prometheanfire: those xml fragment look identical so ya maybe | |
| 15:25:25 | prometheanfire | kk, testing https://review.opendev.org/673848 | |
| 15:33:59 | mriedem | sean-k-mooney: comments in https://review.opendev.org/#/c/669738/ | |
| 15:34:05 | mriedem | maybe i'm just dumb | |
| 15:36:19 | sean-k-mooney | mriedem: thanks ill chack | |
| 15:37:23 | aspiers | sean-k-mooney: the new filter editor in lnav 0.8.5 is awesome (press tab) | |
| 15:45:49 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/stein: Fix AttributeError in RT._update_usage_from_migration https://review.opendev.org/673856 | |
| 15:57:22 | nafiux | Good morning team, I’m trying to configure allocation ratio for CPU and Memory, but I’m getting this error from nova-scheduler: “Got no allocation candidates from the Placement API. This could be due to insufficient resources or a temporary occurrence as compute nodes start up.” | |
| 15:57:30 | openstackgerrit | Merged openstack/nova master: Introduces the openstacksdk to nova https://review.opendev.org/643664 | |
| 15:57:38 | openstackgerrit | Merged openstack/nova master: Use OpenStack SDK for placement https://review.opendev.org/656023 | |
| 15:58:55 | nafiux | Any hint on how to setup that properly? | |
| 15:59:30 | nafiux | I’m using kolla-ansible for deployment, and I already created the “/etc/kolla/config/nova/localhost/nova.conf” file which contains: cpu_allocation_ratio = 16.0 & ram_allocation_ratio = 5.0 under [DEFAULT] | |
| 16:11:54 | mriedem | which release? do you even have compute nodes reporting into placement? i.e. does "openstack resource provider list" return anything? | |
| 16:12:03 | mriedem | https://docs.openstack.org/osc-placement/latest/ | |
| 16:12:15 | mriedem | https://docs.openstack.org/nova/latest/admin/configuration/schedulers.html#allocation-ratios | |
| 16:15:10 | mriedem | efried_rollin: random musing in https://review.opendev.org/#/c/673706/ but i'll likely abandon that change and open a new bug for the other spurious warning in there | |
| 16:19:26 | nafiux | “openstack resource provider list” isn’t found, should I install any additional python CLI package? | |
| 16:20:09 | nafiux | Oh, I saw… “pip install osc-placement”, let me try. | |
| 16:20:43 | nafiux | Yes, I’m getting a node there | |
| 16:26:34 | Sundar | nit tests to fail. Is this really kosher? | |
| 16:26:34 | Sundar | Hi all, compute/manager.py::build_and_run_instance() (https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L1859) requires the request_spec as an argument. But many unit test cases (e.g. https://github.com/openstack/nova/blob/master/nova/tests/unit/compute/test_compute.py#L9305) call it without the request_spec, which causes some u | |
| 16:37:35 | melwitt | Sundar: that unit test is not doing something ideal by passing {} for those args. in the case where the args are not relevant to what's being tested, it's recommended to use mock.ANY instead. what do you mean by "causes some unit tests to fail"? | |
| 16:45:51 | openstackgerrit | Matt Riedemann proposed openstack/nova master: rt: soften warning case in _remove_deleted_instances_allocations https://review.opendev.org/673873 | |
| 16:46:58 | mriedem | Sundar: first thing is i'd avoid copying unit tests from the test_compute module, | |
| 16:47:14 | mriedem | base your tests off test_compute_mgr if you're looking for existing tests to start from | |
| 16:47:30 | mriedem | test_compute.py is historically running actual services and using the db, so it's more functional than unit-y | |
| 16:47:43 | mriedem | reserve more complicated tests for actual functional tests with fixtures and such | |
| 16:48:10 | mriedem | like mel said, if your test doesn't care about the param, pass a Mock, else pass an actual nova.objects.RequestSpec object, not a dict | |
| 16:48:17 | mriedem | request specs as dicts are super legacy | |
| 17:09:27 | Sundar | mriedem, melwitt: Sorry, was in a call. I am referring to existing Nova unit tests e.g. https://github.com/openstack/nova/blob/master/nova/tests/unit/compute/test_compute.py#L9305. My patches do not anything with flavors etc. but are failing due to this issue: http://logs.openstack.org/44/631244/25/check/openstack-tox-py27/f76bf93/job-output.txt.gz | |
| 17:09:28 | Sundar | #_2019-07-31_10_21_40_684847 | |
| 17:09:38 | Sundar | mriedem, melwitt: Sorry, was in a call. I am referring to existing Nova unit tests e.g. https://github.com/openstack/nova/blob/master/nova/tests/unit/compute/test_compute.py#L9305. My patches do not anything with flavors etc. but are failing due to this issue: http://logs.openstack.org/44/631244/25/check/openstack-tox-py27/f76bf93/job-output.txt.gz | |
| 17:09:39 | Sundar | #_2019-07-31_10_21_40_684847 | |
| 17:10:29 | Sundar | [Yikes, message got duplicated] | |
| 17:10:53 | Sundar | IOW, these are not new test cases I am introducing. | |
| 17:11:55 | melwitt | yeah, so that makes sense considering the code you added. you'll just need to adjust the unit tests to accommodate the new code, like making the test pass Mock objects instead of empty dicts like it is doing right now | |
| 17:14:09 | melwitt | this is a common thing, when adding new code, often existing unit tests will need to be adjusted along with the change. in this particular case, you uncovered a legacy thing (passing {} for objects) that should not be that way | |
| 17:19:33 | Sundar | melwitt: Yes, I know. But there are a lot of unit tests like that. :( Was hoping to get some alternatives. | |
| 17:22:54 | Sundar | melwitt, mriedem: I could adjust my code to check if request_spec is None. Hope that is also acceptable. | |
| 17:27:16 | melwitt | if there's a chance for request_spec to be None in reality, then that would work. if not, it's not really ok to do something in the impl just to get around unit tests that should be fixed. I'm not as well-versed about request_spec as mriedem, he probably would know most whether request_spec could possibly be None in this case | |
| 17:31:08 | Sundar | melwitt: Kinda expected that response, but tried my luck anyway. ;) Thank you. | |
| 17:40:47 | mriedem | Sundar: for tests that hit this code https://review.opendev.org/#/c/631244/25/nova/compute/manager.py@1865 you should be passing a real RequestSpec object. for *existing* tests hitting that code, if they are passing {} or None or something, then they need to be updated to pass either a RequestSpec object or a Mock | |
| 17:41:00 | mriedem | or, to simply mock out _create_and_bind_arqs | |
| 17:41:07 | mriedem | since those existing tests don't care about the details of that method | |
| 17:41:15 | mriedem | and your *new* tests would call _create_and_bind_arqs directly | |
| 17:41:17 | mriedem | with a RequestSpec object | |
| 17:41:32 | mriedem | - Failed: 159 | |
| 17:41:34 | mriedem | yikes | |
| 17:41:51 | mriedem | i'm guessing that's a lot of cumulative failures since it's later in the series | |
| 17:42:06 | mriedem | i.e. https://review.opendev.org/#/c/631243/27 | |
| 17:45:07 | Sundar | mriedem: Thanks. I counted 272 errors of that type over all. Mocking _create_and_bind_arqs is certainly easier. | |
| 17:45:27 | melwitt | code review | |
| 17:45:27 | melwitt | Sundar: a bit of advice, what I would do in this situation is write the impl and get the tempest tests passing (I assume something has been or needs to be set up that runs cyborg in it) first, get some review feedback once tempest coverage is passing, then fix the unit test and func test failures after you get some code review. that way you don't have to go through the work to fix hundreds of tests when the impl might change during | |
| 17:46:43 | mriedem | yeah, or just mock out the thing semi-globally to start, | |
| 17:47:08 | mriedem | e.g. in the patch that is failing tests on _create_and_bind_arqs, just stub that out in the test class setUp | |
| 17:47:41 | mriedem | melwitt: the tricky thing with tempest and this series is there is no tempest coverage with cyborg yet, | |
| 17:47:47 | mriedem | when there is, it'll be a fake cyborg driver, | |
| 17:47:53 | mriedem | but cyborg isn't hooked up with all of that yet, not even placement | |
| 17:47:59 | mriedem | so tempest is a loooong ways off | |
| 17:48:03 | melwitt | oh :\ ok | |
| 17:48:27 | Sundar | melwitt: Makes sense. mriedem: We are setting up tempest: https://review.opendev.org/#/c/670999/ | |
| 17:49:12 | Sundar | But I agree it is tough to predict when it'll land. | |
| 17:49:44 | openstackgerrit | Dustin Cowles proposed openstack/nova master: Provider config file schema and loader https://review.opendev.org/673341 | |
| 17:52:03 | openstackgerrit | Dustin Cowles proposed openstack/nova master: Provider config file schema and loader https://review.opendev.org/673341 | |
| 18:07:44 | mriedem | melwitt: looks like https://review.opendev.org/#/c/668123/3 is ready to go in queens | |
| 18:07:50 | mriedem | lee was wanting that one | |
| 18:09:20 | melwitt | a-ha, cool. will review | |
| 18:10:21 | openstackgerrit | Merged openstack/nova master: Get rid of args to RBDDriver.__init__() https://review.opendev.org/668564 | |
| 18:15:21 | openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Parse the 'os' element from domainCapabilities https://review.opendev.org/673790 | |
| 18:15:21 | openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Handle alternative UEFI firmware binary paths https://review.opendev.org/348394 | |
| 18:23:53 | nafiux | Team, what should I do to properly configure the overcommit of resources? I’m getting this error: “Got no allocation candidates from the Placement API. This could be due to insufficient resources or a temporary occurrence as compute nodes start up.” | |
| 18:30:04 | mriedem | efried_rollin: since i'm not sure how this whole series works, i didn't vote on https://review.opendev.org/#/c/671072/2 but not sure if that should be using a set or list for forbidden_aggregates | |
| 18:30:33 | mriedem | nafiux: you said you listed some resource providers, did you check their inventories to see that they have the allocation_ratio values you configured in nova? you also didn't say what release you were using. | |
| 18:30:47 | mriedem | https://docs.openstack.org/osc-placement/latest/cli/index.html#resource-provider-inventory-list | |
| 18:31:55 | nafiux | Thanks @mriedem. I’m using stein. Let me take a look on that documentation. | |
| 18:33:06 | nafiux | Here what I have: https://pastebin.com/MdXsJYs3 | |
| 18:33:13 | mriedem | nafiux: and https://docs.openstack.org/nova/latest/admin/configuration/schedulers.html#allocation-ratios if you didn't read that yet | |
| 18:33:34 | spatel | sean-k-mooney: are you there? | |
| 18:33:45 | mriedem | yeah so your compute nodes configured ram_allocation_ratio is being reported in the MEMORY_MB inventory for the provider | |
| 18:33:46 | mriedem | | MEMORY_MB | 5.0 | 49139 | 512 | 1 | 1 | 49139 | | |
| 18:33:49 | mriedem | that looks fine | |
| 18:33:56 | sean-k-mooney | spatel: kind of yes | |
| 18:34:08 | mriedem | nafiux: is this a fresh deploy or do you already have some servers on that node? | |
| 18:35:02 | nafiux | It’s a fresh deployment, I don’t have any instance running at this point. I used kolla-ansible. I checked the /etc/nova/nova.conf inside the nova_scheduler container and I was able to see the allocation values: | |
| 18:35:11 | nafiux | docker exec -it nova_scheduler bash | |
| 18:35:20 | spatel | Good, I have machine with 64G and with this setting in grub hugepagesz=2M hugepages=27500 | |
| 18:35:49 | spatel | I have this flavor setting - properties | hw:cpu_policy='dedicated', hw:mem_page_size='large', hw:numa_nodes='2' | |
| 18:35:51 | nafiux | vi /etc/nova/nova.conf, and I can see inside [DEFAULT] the cpu_allocation_ratio = 16.0 and ram_allocation_ratio = 5.0 | |
| 18:36:13 | nafiux | I’m able to launch a server with 16GB of Ram and 8 VPCU, but I’m not able to launch a second one. | |