Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-10-02
16:15:56 openstackgerrit Matthew Booth proposed openstack/nova master: DNM: Add some debug logging https://review.openstack.org/607301
16:18:26 sean-k-mooney artom: try raising right at the point the libvirt driver calls libvir to spawn the instance
16:18:52 gibi mriedem: hi, do you have something for the notification subteam meeting? I don't have any
16:19:14 artom sean-k-mooney, it's not even that. I've booted 10 instances with --min-count, and they've *all* gone to the "wrong" host
16:21:02 sean-k-mooney artom: you didnt hit the auto host down on faiure codepath?
16:21:33 artom sean-k-mooney, well, how would I see that? The hypervisor shows as up, same for the service
16:23:39 dansmith artom: there's a weigher now
16:23:48 dansmith for the fail count,
16:24:02 dansmith but that will only take effect if you have failed once on that compute already
16:24:39 artom dansmith, aha, thank you!
16:24:52 artom That would explain it - now, presumably this fail count is hidden in the DB somewhere.
16:25:29 sean-k-mooney dansmith: didnt you also add something last cycle to auto makre the state as disabled after 10 ish failed boots in addtion to the weigher
16:25:46 dansmith sean-k-mooney: no, the weigher replaced the auto-down thing
16:25:53 sean-k-mooney artom: im assuming openstack compute service list shoul the state as up and statuse as enabled
16:26:29 sean-k-mooney dansmith: ah ok i guess that makes sense we had no auto up right so you had to manully go mark it up
16:27:24 dansmith right
16:27:25 artom dansmith, hrmm, does the fail count get reset on service restart?
16:27:55 artom Btw, this would be a think we should probably expose in the hypervisors API or something
16:28:03 sean-k-mooney artom: if its in memory you would have to restart the schduler
16:28:13 dansmith it's in the host state
16:28:23 mriedem gibi: no, although i do have something i can stick in your ear...
16:28:24 dansmith so we should pull that each time in the filter scheduler, IIRC
16:29:12 mriedem gibi: https://review.openstack.org/#/c/603930/6/nova/conductor/tasks/cross_cell_migrate.py@519 for the cross-cell resize stuff i'm adding a new conductor task and the conductor code relies on that old scheduler_utils.set_vm_state_and_notify method, which sends an error notification but those are all legacy
16:29:35 mriedem https://github.com/openstack/nova/blob/cce3208cc28268e4b50e155c205bcab9f1da2a4b/nova/scheduler/utils.py#L600
16:29:44 sean-k-mooney artom: this is the filter https://github.com/openstack/nova/blob/master/nova/scheduler/weights/compute.py#L24 you could just set the weight multipler to 0 to disable it
16:29:47 mriedem i'm not entirely sure how to make that generic thing useful for versioned notifications
16:29:48 dansmith artom: you can just zero or invert the weight and see if that changes it
16:29:59 gibi mriedem: looking
16:30:18 mriedem artom: see compute_nodes.stats column
16:30:26 mriedem which is a serialized dict i think....
16:30:34 artom mriedem, saw it, it's a JSON blob
16:30:38 mriedem yup
16:30:53 mriedem not a terrible idea to report that out of the os-hypervisors API
16:30:59 artom Restarting sched and/or compute seems to do what I want (ie, effectively reset the failure count)
16:31:10 artom mriedem, yeah eh? It would avoid more losers like me getting lost
16:31:36 gibi mriedem: there is an open (and old) patch about that missing set of notifications https://review.openstack.org/#/c/482629/9/nova/scheduler/utils.py
16:32:10 gibi mriedem: s/old/not reviewed by me for a while/
16:32:12 mriedem oh boy...that's a big one
16:32:48 gibi mriedem: stephenfin was happy with that not so long ago
16:33:35 gibi mriedem: anyhow I made a note to review this sooner than later
16:33:51 mriedem definitely not a high priority, but thanks for noting this exists
16:35:13 gibi mriedem: the burndown chart remembered that it exists not me :)
16:36:21 gibi mriedem: anyhow I will not run the notification subteam meeting this week either then
16:39:42 mriedem gibi: ok
16:39:55 mriedem gibi: do you want to just cancel the meeting?
16:40:07 mriedem i mean, indefinitely
16:41:34 gibi mriedem: that decision is forming slowly in my mind in the past weeks
16:41:46 gibi mriedem: I think it would make sense to cancel it indefinitely
16:44:32 mriedem go for it,
16:44:51 mriedem just have to drop it from http://git.openstack.org/cgit/openstack-infra/irc-meetings/
16:45:06 cdent --meetings++
16:45:28 gibi mriedem: OK, and I drop a mail to ML just for the record
16:50:16 gibi mriedem: fyi https://review.openstack.org/607314
16:53:13 cdent sean-k-mooney: I was "less meetings is good" in reponse to gibi. My syntax very bad.
16:53:54 sean-k-mooney cdent: yes as a sane person that was my assumtion
16:55:16 sean-k-mooney cdent: actully since your about. we still block live migration with config drive to fail right
16:56:06 gibi cdent: :)
16:56:34 cdent sean-k-mooney: that is not an area of expertise for me, but a hazy memory suggests that's the case
16:57:10 openstackgerrit Merged openstack/nova stable/pike: Fix host validity check for live-migration https://review.openstack.org/590263
16:57:18 sean-k-mooney cdent: oh ok i had a vague memory that you were invovled in adding config drive at some point
16:57:41 sean-k-mooney cdent: in either case it does which is what i was expecting
16:57:45 cdent not me, unless I blacked it out
16:58:24 sean-k-mooney cdent: yes that would be a sensable thing to do if you had worked on config drive :P
16:59:44 sean-k-mooney that abit unfiar to config drive as it actully works prettry in limited usescaes but live migration is not one of them
16:59:52 openstackgerrit Elod Illes proposed openstack/nova stable/ocata: Add check for invalid allocation amounts https://review.openstack.org/607320
16:59:53 openstackgerrit Elod Illes proposed openstack/nova stable/ocata: Add check for invalid inventory amounts https://review.openstack.org/607321
17:32:56 cfriesen So what's the process for getting a specless blueprint approved? One of my coworkers opened up https://blueprints.launchpad.net/nova/+spec/support-hpet-on-guest and there's code up as well. Should he be using the runway system once the blueprint is approved? It's just the one commit.
17:50:03 melwitt cfriesen: usually when seeking specless blueprint approval, you can add it to the Open Discussion section of the next nova meeting agenda
17:50:23 cfriesen melwitt: thanks, that works
17:50:30 melwitt using the runway system after approval is good for attracting review attention
17:51:51 gryf artom: unfortunately, I'm afk right now. Which time zone are you in?
17:57:19 openstackgerrit Eric Fried proposed openstack/nova master: Placement: Remove usage of get_legacy_facade() https://review.openstack.org/607336
17:58:24 efried melwitt, cdent: ^
18:43:07 openstackgerrit Sylvain Bauza proposed openstack/nova master: libvirt: implement reshaper for vgpu https://review.openstack.org/599208
18:48:50 artom gryf, GMT-4 (NA east coast)
18:48:54 artom gryf, email?
18:52:06 dansmith mriedem: https://review.openstack.org/#/c/607296
18:55:44 mriedem done
19:07:48 bauzas dansmith: mriedem: a few other people interested in, I finally reworked the reshaper change https://review.openstack.org/599208
19:07:55 bauzas I'll test it on a devstack
19:08:04 bauzas with a machine having pGPUs
19:10:17 mriedem :( this is all half-baked https://review.openstack.org/#/q/topic:bug/1384637+(status:open+OR+status:merged)
19:10:31 mriedem none of that plumbing was ever leveraged by the REST API
19:10:39 openstackgerrit melanie witt proposed openstack/nova-specs master: Update blueprint name so spec matches launchpad https://review.openstack.org/607347
19:15:58 openstackgerrit melanie witt proposed openstack/nova-specs master: Dynamically find releases for move-implemented-specs https://review.openstack.org/592628
19:15:59 openstackgerrit melanie witt proposed openstack/nova-specs master: Add a script for counting blueprints https://review.openstack.org/581914
19:17:26 cfriesen if anyone feels like a spec review, the updated emulated TPM spec is up at https://review.openstack.org/#/c/571111 There are no API changes, the spec is really to allow discussion of the concept.
19:19:19 sean-k-mooney cfriesen: i assume the only enduser fasing change would be an image property or flavor extra spec to enable/request teh vtpm
19:19:43 cfriesen sean-k-mooney: flavor extra spec, yes. the rest is virt driver backend stuff
19:28:59 sean-k-mooney ill review it in detail tomorow but initall feedback is i would proably expect 2 extra_spec argument one to specify the tpm verion and another for the backend type. other then that it would be nice to support this via the image metadata too but after 5 mins skiming it it seams resonable
19:29:55 sean-k-mooney i seam to recall form the PTG there were some live migration requirements too which i did not see explcitly in the spec. if so proabley a good idea to add them
19:33:59 openstackgerrit Artom Lifshitz proposed openstack/nova-specs master: Fail count in API https://review.openstack.org/607352
19:34:13 artom dansmith, mriedem, ^^ really easy spec about the fail count discussion earlier
19:35:05 dansmith hmm
19:35:16 bauzas artom: I thought we said to deprecate os-hypervisors API ?
19:35:22 bauzas at least not adding more to it
19:35:31 artom bauzas, seriously? I had no idea.
19:35:48 artom Nothing in the api-ref about it
19:35:51 dansmith and that fail count is an internal value that can change
19:36:05 mriedem artom wasn't at the ptg when os-hypervisors was discussed...

Earlier   Later