| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-08-08 | |||
| 15:13:34 | efried | I never understood the original spec | |
| 15:14:01 | efried | but I'll have a look at some point if you get desperate. | |
| 15:14:48 | stephenfin | efried: TBH, it's what I'm doing anyway so it really just needs a sanity check | |
| 15:16:03 | efried | point is, if I do manage to review the code, I'll be doing it against whatever the spec says. So if I rubber-stamp the spec based on the fact that that's what you're coding... https://previews.123rf.com/images/sudowoodo/sudowoodo1708/sudowoodo170800024/84484656-ouroboros-symbol-snake-eating-its-own-tail-modern-alchemy-logo-vector-illustration-.jpg | |
| 15:30:53 | openstackgerrit | Shilpa Devharakar proposed openstack/nova-specs master: Update spec: filtering of alloc candidates by forbidden aggregates https://review.opendev.org/675384 | |
| 15:37:06 | aspiers | how can I debug placement returning no allocation candidates on devstack, when debug is already set to True and showing nothing useful? | |
| 15:37:14 | aspiers | GET /placement/allocation_candidates?limit=1000&required=%21COMPUTE_STATUS_DISABLED&resources=DISK_GB%3A20%2CMEMORY_MB%3A2048%2CVCPU%3A1 | |
| 15:38:27 | aspiers | I've already gone through the three resource classes via openstack resource provider inventory show $rp_uuid | |
| 15:38:51 | aspiers | each one is showing a total and max_unit greater than what is being asked for | |
| 15:38:55 | efried | aspiers: You'll want to show usages | |
| 15:39:09 | aspiers | isn't that what I just did? ^^ | |
| 15:39:15 | efried | You showed inventories | |
| 15:39:39 | efried | usages: "how much of it is consumed" | |
| 15:39:39 | efried | inventories: "how big is it" | |
| 15:39:44 | aspiers | ok | |
| 15:40:34 | aspiers | surely placement should offer to DEBUG-log details of why it's rejecting candidates? | |
| 15:40:49 | efried | that's not nearly as easy as you would think | |
| 15:40:56 | aspiers | "why am I getting NoValidHost" is the #1 FAQ I've ever heard from operators in 7 years of OpenStack | |
| 15:41:02 | efried | ditto | |
| 15:41:03 | aspiers | of course it wouldn't be ;-/ | |
| 15:41:19 | aspiers | otherwise someone would have done it, I guess | |
| 15:41:30 | efried | And this would be an absolutely overwhelming amount of data | |
| 15:41:38 | efried | but mainly | |
| 15:41:38 | aspiers | well sure | |
| 15:41:43 | aspiers | that's why it would be at DEBUG level | |
| 15:41:50 | efried | we reject the vast majority of things based on a database query | |
| 15:42:10 | aspiers | well even logging what's left after that would be a start | |
| 15:42:18 | efried | we do have some debug logs for that | |
| 15:42:29 | efried | and the fact that you're not seeing any means... we've done all the filtering at the db level :) | |
| 15:42:46 | aspiers | ok | |
| 15:43:10 | aspiers | looks like I've used 20 out of 35 DISK_GB and am asking for another 20 | |
| 15:43:17 | efried | that'd do it | |
| 15:43:17 | aspiers | next question is why is it so damn low | |
| 15:43:30 | efried | you can show allocations | |
| 15:43:55 | aspiers | "openstack resource provider allocation show $rp_uuid" shows nothing | |
| 15:44:07 | efried | allocation show takes an instance UUID | |
| 15:44:16 | efried | I think you should be able to do allocation list | |
| 15:44:17 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix rebuild of baremetal instance when vm_state is ERROR https://review.opendev.org/523559 | |
| 15:44:18 | mriedem | melwitt: addressed your comments i think ^ | |
| 15:44:22 | mriedem | efried: or a migration uuid :) | |
| 15:44:37 | edleafe | artom: mriedem: I just looked at the API you were discussing earlier for missing ports, and I think it *should* be a 404, since the resource you are requesting is the port interface. 400 would be correct if you were searching for a server that had that port. | |
| 15:44:48 | mriedem | jroll: TheJulia: could use ironic subteam re-review on this https://review.opendev.org/523559 | |
| 15:45:12 | mriedem | edleafe: both are on the path | |
| 15:45:16 | mriedem | server_id and port_id | |
| 15:45:49 | mriedem | 404 cool for all things on the path then? | |
| 15:45:56 | melwitt | mriedem: that's a blast from the past. will have to reload a lot of context when I go back to review | |
| 15:46:42 | edleafe | mriedem: is this the GET or DELETE call? | |
| 15:46:46 | mriedem | edleafe: anyway, it's essentially dead code anyway | |
| 15:46:48 | mriedem | DELETE | |
| 15:47:07 | mriedem | https://docs.openstack.org/api-ref/compute/#detach-interface | |
| 15:47:44 | edleafe | The call doesn't delete the server. The resource it is concerned with is the port. If that resource doesn't exist, then 404 is correct | |
| 15:47:46 | aspiers | efried: I mean, why is the *total* inventory so damn low, not the usage | |
| 15:47:56 | aspiers | I guess that's in n-cpu | |
| 15:48:04 | efried | yes | |
| 15:48:07 | mriedem | edleafe: ack | |
| 15:48:11 | edleafe | Of course, if you subscribe to the idempotent DELETE school of thought, 204 would be correct | |
| 15:48:20 | efried | aspiers: I assume libvirt driver? | |
| 15:48:21 | mriedem | like i said, it's dead code at this point unless we do a proxy neutronv2.api.show_port call on that port_id | |
| 15:48:29 | openstackgerrit | Shilpa Devharakar proposed openstack/nova-specs master: Update spec: filtering of alloc candidates by forbidden aggregates https://review.opendev.org/675384 | |
| 15:49:05 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: allow getting resource request of every bound ports of an instance https://review.opendev.org/655110 | |
| 15:49:14 | aspiers | efried: virsh shows no pools | |
| 15:49:31 | efried | I don't know what that means | |
| 15:49:37 | aspiers | virsh pool-list | |
| 15:49:38 | efried | look at _get_local_gb_info | |
| 15:49:45 | aspiers | ok thanks | |
| 15:49:51 | efried | what's your [libvirt]images_type? | |
| 15:50:26 | mriedem | artom: you know, could just move this validation to the api https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L6373 | |
| 15:50:34 | sean-k-mooney | aspiers: i dont think openstack uses libvirt storage pools | |
| 15:50:42 | aspiers | sean-k-mooney: yeah that was a guess | |
| 15:50:43 | mriedem | compute is already checking the info cache before doing anything with the port at the neutron level | |
| 15:51:29 | sean-k-mooney | aspiers: mybe we might for lvm image backend? but im pretty sure tis not used for qcow/raw/flat iamge type | |
| 15:51:59 | aspiers | libvirt.images_type = default | |
| 15:52:03 | efried | aspiers: assuming your libvirt.images_type is unset or anything other than lvm/rbd, it'll be looking at the free space on your instances_path | |
| 15:52:12 | aspiers | that makes more sense | |
| 15:54:24 | efried | aspiers: ...which looks like it'll be by default whatever file system your nova installation is | |
| 15:54:57 | efried | if your nova is in /foo/bar/nova, your instances_path will by default be /foo/bar/instances, I think. | |
| 15:55:13 | efried | so | |
| 15:55:23 | efried | df -h /foo/bar ought to tell you how much space it thinks you have. | |
| 15:55:41 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Pass network API to the conducor's MigrationTask https://review.opendev.org/655111 | |
| 15:55:54 | aspiers | yeah it's basic sysadmin / lvm stuff now | |
| 15:56:03 | aspiers | I'll take it from here. thanks for pointers :) | |
| 15:56:07 | efried | good luck | |
| 15:57:19 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: run test_rebuild_server_in_error_state in ironic CI job https://review.opendev.org/675391 | |
| 15:57:19 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix rebuild of baremetal instance when vm_state is ERROR https://review.opendev.org/523559 | |
| 15:58:51 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: run test_rebuild_server_in_error_state in ironic CI job https://review.opendev.org/675391 | |
| 16:15:14 | mriedem | gibi: efried: seeing another notification sample race here https://logs.opendev.org/36/614036/46/check/nova-tox-functional/b505ffb/testr_results.html.gz | |
| 16:16:04 | mriedem | idk if the db stuff in there is just noise | |
| 16:17:19 | mriedem | btw, unrelated to that, i noticed we have a novnc test that fails a lot in multinode jobs http://status.openstack.org/elastic-recheck/#1669468 | |
| 16:19:48 | openstackgerrit | Merged openstack/nova stable/rocky: Avoid logging traceback when detach device not found https://review.opendev.org/674068 | |
| 16:19:56 | openstack | bug 1837955 in OpenStack Compute (nova) rocky "MaxRetriesExceeded sometime fails with messaging exception" [Medium,In progress] https://launchpad.net/bugs/1837955 - Assigned to Matt Riedemann (mriedem) | |
| 16:19:56 | openstackgerrit | Merged openstack/nova stable/rocky: Add functional regression test for bug 1837955 https://review.opendev.org/673536 | |
| 16:22:11 | openstackgerrit | Merged openstack/nova stable/rocky: Cleanup when hitting MaxRetriesExceeded from no host_available https://review.opendev.org/673553 | |
| 16:25:43 | artom | mriedem, ack | |
| 16:26:01 | artom | mriedem, although. Since we should be raising a 400 in the port not found case, technically it's a new microversion | |
| 16:26:27 | artom | So, being the lazy schmuck that I am, do I want to bother? | |
| 16:26:44 | mriedem | artom: you missed edleafe saying it should be a 404 | |
| 16:27:09 | artom | Ah, so I did. | |
| 16:29:45 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Dump versioned notifications when len assertions fail https://review.opendev.org/675404 | |
| 16:29:47 | mriedem | ^ can help with debugging racy versioned notification functional test failures ^ | |
| 16:33:42 | mriedem | this is very weird https://logs.opendev.org/32/634832/43/check/nova-tox-functional-py36/d4f3be5/testr_results.html.gz | |