Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-03-23
15:22:02 bauwser well, okay, I have to disappear in a short few because of kids, but once I'm back, I'll reevaluate the spec
15:22:06 sean-k-mooney[m] well i can ask can i try to port this once it merges to placement
15:22:53 ktibi leakypipes, I checked placement log, i can see only info log with GET on resource_providers/705b7a45-ce71-4455-8349-6cee4c051c0e/inventories
15:23:12 leakypipes ktibi: question for you... it looks like there are no instances in the cell DB for that compute node. were there originally instances on it?
15:23:22 sean-k-mooney[m] ok cool im technically off today but let me know what you decide and ill be back monday
15:24:01 ktibi leakypipes, no when I remove this node, I disable service, evacuate all instances, remove service and shutdown the compute
15:24:32 leakypipes ktibi: k. I'm wondering maybe the placement service for some reason didn't get updated with the removal of those instances...
15:24:50 sean-k-mooney[m] bauwser: if i was able to also port to doing this in placement in rocky that would be the best outcome but im ment to work on someting else internally from april that 10%.
15:24:53 leakypipes ktibi: can you please execute the following SQL query against the nova api database?
15:25:55 leakypipes ktibi: SELECT COUNT(*) FROM allocations AS a JOIN inventories AS i ON a.resource_provider_id = i.resource_provider_id JOIN resource_providers AS rp ON i.resource_provider_id = rp.id WHERE rp.uuid = '91f9e750-351f-463e-aa00-c8843c37560e';
15:26:10 ktibi leakypipes, count : 0
15:26:14 sean-k-mooney[m] o/ ok ill stop working on my day off now... maybe...
15:26:15 leakypipes ktibi: ok, good.
15:26:47 leakypipes ktibi: what about this one? SELECT COUNT(*) FROM inventories AS i ON a.resource_provider_id = i.resource_provider_id JOIN resource_providers AS rp ON i.resource_provider_id = rp.id WHERE rp.uuid = '91f9e750-351f-463e-aa00-c8843c37560e';
15:26:56 leakypipes ktibi: oops, sorry
15:27:12 leakypipes ktibi: what about this one? SELECT COUNT(*) FROM inventories AS i JOIN resource_providers AS rp ON i.resource_provider_id = rp.id WHERE rp.uuid = '91f9e750-351f-463e-aa00-c8843c37560e';
15:27:47 ktibi leakypipes, maybe that can help you http://paste.openstack.org/show/710164/
15:28:13 ktibi leakypipes, count : 0 too
15:28:22 leakypipes k
15:28:47 leakypipes ktibi: got it...
15:28:56 cdent figleaf, fried_rice, leakypipes : just to confirm: I assume we would like the forbidden traits support to show up in the same microversion for /resource_providers and /allocation_candidates, yes?
15:28:59 leakypipes ktibi: your services table has two records for compute06
15:29:11 leakypipes ktibi: delete the one where deleted_at is not NULL.
15:29:15 ktibi yes I see :) I waiting your comfirmation to delete it ^^
15:29:19 fried_rice cdent: Sure, unless there's some reason to split 'em.
15:29:38 cdent fried_rice: only if we want to make the code more split into stages
15:29:52 fried_rice no, not necessary IMO.
15:30:33 fried_rice cdent: In fact, I find myself slightly irked that we can't include multiple features in a single microversion - for reasons of our development process, not for any technical reason.
15:30:50 fried_rice cdent: But in this case, it's the same feature, so we can totally get away with it.
15:31:00 cdent yes and yes
15:31:04 leakypipes ktibi: and then set disabled=0 on the other services table record with copmute06 host name.
15:32:41 leakypipes ktibi: finally, can you show me what the compute_nodes table looks like please? just do this: SELECT id, uuid, hypervisor_hostname FROM compute_nodes;
15:33:10 ktibi leakypipes, ho I can see two compute06
15:34:46 ktibi leakypipes, http://paste.openstack.org/show/710169/
15:35:09 mlavalle rybridges: in that log statement right above L225, could you log aggregate.uuid? The reason I am asking this is https://review.openstack.org/#/c/415031/, which fixes https://bugs.launchpad.net/nova/+bug/1652642
15:35:10 openstack Launchpad bug 1652642 in OpenStack Compute (nova) "Nova API doesn't return aggregate's uuid, which is needed when using the placement API" [High,Fix released] - Assigned to Matt Riedemann (mriedem)
15:35:55 leakypipes ktibi: DELETE FROM compute_nodes WHERE id = 15;
15:36:07 leakypipes ktibi: that *should* fix things up...
15:36:22 ktibi leakypipes, ok I restart nova compute on 06
15:36:44 openstackgerrit melanie witt proposed openstack/nova master: Move _make_instance_list call outside of DB transaction context https://review.openstack.org/555093
15:37:44 mlavalle rybridges: this is the Nova API version you require for aggregates to have a uuid: https://review.openstack.org/#/c/415031/9/nova/api/openstack/api_version_request.py@100
15:37:51 ktibi leakypipes, same error on compute06 : Another thread already created a resource provider with the UUID 91f9e750-351f-463e-aa00-c8843c37560e. Grabbing that record from the placement API.
15:37:59 ktibi maybe need to restart placement ?
15:38:29 leakypipes ktibi: well, that's not an error. :) the error is the "Cannot refresh inventory" thing.
15:38:36 openstackgerrit Matt Riedemann proposed openstack/nova master: Create volume attachment during boot from volume in compute https://review.openstack.org/541420
15:38:38 leakypipes ktibi: no, no need to restart placement.
15:38:57 leakypipes ktibi: what we want to see is inventory records being created in placement.
15:39:04 ktibi leakypipes, ok I'll test to migrate instance to this host
15:42:45 figleaf cdent: yes, one microversion for that makes sense
15:43:07 ktibi leakypipes, ok fail :/
15:43:44 ktibi leakypipes, exception thrown: No valid host was found. Unable to move instance f8e70257-244e-4fa7-b545-0a888b742b84 to host compute06. There is not enough capacity on the host for the instance.
15:44:04 giblet mriedem, leakypipes, mlavalle: is it OK for you If I publish a new version of the bandwidth spec soon, or you are just in the middle of reviewing the current PS?
15:44:22 mlavalle giblet: I'm good
15:45:07 mriedem giblet: i haven't gone back over the comments since i last reviewed it last week
15:45:29 mriedem i could do that quick i guess...
15:46:06 ktibi leakypipes, http://paste.openstack.org/show/710182/
15:46:17 giblet mriedem: I still need about an hour to do the update locally, if that is enough for you. Or you can go back after I published the new rev.
15:46:25 kashyap mriedem: So just checked w/ the folks on -infra, the Gate already does have the "Bionic" (upcoming Ubuntu LTS) images. Do we have any non-voting Nova jobs for it?
15:46:32 kashyap (Near as see, none.)
15:46:36 mriedem kashyap: no
15:46:51 mriedem kashyap: i'm just trying to get us to using the queens UCA right now
15:47:05 mriedem https://review.openstack.org/#/c/554317/
15:47:20 giblet mriedem: there will some new problems described in the update
15:47:39 giblet mriedem: so you might want to read the new rev anyhow
15:48:11 mriedem giblet: i've got about 20 minutes so i'll go over the replies quick
15:48:38 giblet mriedem: OK, cool, thanks
15:52:49 mriedem giblet: yeah i can already see issues with https://review.openstack.org/#/c/502306/17/specs/rocky/approved/bandwidth-resource-provider.rst@100
15:52:52 mriedem so commenting
15:54:51 mriedem and gerrit just shit the bed
15:56:06 giblet gerrit: you are not helpful at all!
15:59:15 giblet mriedem: gerrit is back
15:59:50 mriedem huzzah
15:59:54 bauwser sean-k-mooney[m]: +Wd
16:00:05 openstackgerrit Eric Berglund proposed openstack/nova master: PowerVM Driver: Snapshot https://review.openstack.org/543023
16:01:43 rybridges mlavalle: you might be on to something. When I try to print aggregate.uuid, my log message gets swallowed and i dont see anything getting printed
16:01:49 bauwser hum
16:01:50 bauwser https://review.openstack.org/#/dashboard/?foreach=(project%253Aopenstack%252Fnova-specs)+status%253Aopen+NOT+owner%253Aself+NOT+label%253AWorkflow%3C%253D-1+label%253AVerified%3E%253D1%252Cjenkins+NOT+reviewedby%253Aself+is%253Amergeable&title=Nova+Specs+Review+Inbox+&Needs+final+%252B2=NOT+label%253ACode-Review%3C%253D-1%252Cnova-core+label%253ACode-Review%3E%253D2&Small+Patches=NOT+label%253ACode-Review%3C%253D-1%252Cnova-cor
16:01:51 bauwser delta%253A%3C%253D10&Needs+Feedback+(Changes+older+than+5+days+that+have+not+been+reviewed+by+anyone)=NOT+label%253ACode-Review%3C%253D2+age%253A5d&You+are+a+reviewer%252C+but+haven't+voted+in+the+current+revision=reviewer%253Aself&Passed+Jenkins%252C+No+Negative+Core+Feedback=NOT+label%253ACode-Review%3C%253D-1%252Cnova-core+NOT+message%253A%22Closes-Bug%253A+%22&Wayward+Changes+(Changes+with+no+code+review+in+the+last+5+days)
16:01:51 bauwser abel%253ACode-Review%3C%253D2+NOT+label%253ACode-Review%3C%253D-1%252Cnova-core+age%253A5d
16:02:07 bauwser my gerrit dashboard is just getting me only two specs...
16:02:25 bauwser WTF ?
16:02:37 kashyap Give it a couple of mins to breathe
16:06:53 rybridges mlavalle: I should have the right version though... Everything I am running is just stock stable/ocata built from source. dont even have any patches yet
16:07:02 mriedem giblet: replies in PS17 https://review.openstack.org/#/c/502306/17/
16:07:40 giblet mriedem: looking
16:09:28 giblet mriedem_afk: good points. I will dig it
16:10:00 openstackgerrit Merged openstack/nova-specs master: Reintroduced nic feature based scheduling for rocky https://review.openstack.org/545951
16:16:22 bauwser leakypipes: just a comment on https://review.openstack.org/#/c/545057/5 preventing a +2, just add your thoughts on error handling and I'm +2
16:17:01 rybridges mlavalle: One other thing to note perhaps... When I do openstack aggregate show, I see the ID, but not the uuid in the output. If I want to see the uuid I need to go to the db and look at the aggregates table.
16:17:30 mlavalle rybridges: give me a few minutes and I'll check in my dev system
16:17:43 mlavalle I was going to propose you check with the CLI
16:17:50 mlavalle so you got ahead of me ;-)
16:19:07 rybridges cool thanks! Yea i am taking a look at the CLI now. Even when I run aggregate list with the --debug option i still do not see the uuid from the db anywhere in the response. That is likely related to the problem
16:20:18 rybridges The response body looks like this: {"aggregates": [{"name": "Neutron segment id b9d173e2-fff2-4a33-ac50-312e6f3be914", "availability_zone": null, "deleted": false, "created_at": "2018-03-23T15:59:51.000000", "updated_at": null, "hosts": [], "deleted_at": null, "id": 19, "metadata": {}}]}
16:33:35 cfriesen sahid: for what it's worth, I think your short-term proposal to run isolated emulator threads on 'CONF.cpu_shared_set' is reasonable, though I have one question--what would we do if that set is empty? Allocate a new host CPU like we do now and emit a log?
16:34:50 openstackgerrit Dan Smith proposed openstack/nova master: Add request filter functionality to scheduler https://review.openstack.org/544730
16:34:50 openstackgerrit Dan Smith proposed openstack/nova master: Add aggregates list to Destination object https://review.openstack.org/544729
16:34:51 openstackgerrit Dan Smith proposed openstack/nova master: Make get_allocation_candidates() honor aggregate restrictions https://review.openstack.org/547990
16:34:51 openstackgerrit Dan Smith proposed openstack/nova master: Add AggregateList.get_by_metadata() query method https://review.openstack.org/544728
16:34:52 openstackgerrit Dan Smith proposed openstack/nova master: Add require_tenant_aggregate request filter https://review.openstack.org/545002

Earlier   Later