Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-08-14
14:33:26 dansmith if we hit that we'll reschedule and pick another, right?
14:33:45 dansmith but that's kinda what I meant about documenting that fact and recommending an immediate move to RC
14:34:08 dtantsur yeah, I'm wondering how unpleased the people are going to be, if we increase their retry count e.g. twice
14:34:17 dansmith anyway, I'm surprised jay isn't here yet, so I expect he will be soon, let's just hold off a bit as he might have some sneaky idea
14:34:27 dtantsur tripleo maximum is IIRC 30, so it's not a hard failure :)
14:34:42 dansmith maximum what? reschedule?
14:34:48 dtantsur yep
14:34:51 dansmith oof :)
14:35:12 dtantsur the double assignment problem was very frequent some time ago (maybe it's still is)
14:35:25 dtantsur and we have a big issue (well, misfeature) in ironicclient: we retry HTTP Conflict
14:35:29 dansmith you know, I'm not sure why we never solved this problem with a "number of instances == 0" filter instead of some of this other craziness
14:35:55 dansmith like, filter that includes nodes with no instances booted on them
14:36:17 dtantsur yeah.. or extend get_inventory to clearly indicate something like "I cannot accept more nodes, because I won't, go away)
14:36:27 dtantsur yeah, I got the idea
14:36:28 dansmith well, that's what RC does basically
14:36:39 dtantsur well, true :)
14:37:04 dtantsur except that an ironic node can be unavailable for other reasons, e.g. it's being cleaned or in a power management fault
14:37:14 dtantsur which is something we have no way of cleanly expressing
14:37:41 dansmith yeah, I still think that's a different thing than "no inventory" that we should express separately
14:37:44 dansmith hey, look, it's jaypipes
14:37:51 dtantsur wowwowwow
14:38:00 dtantsur jaypipes: welcome to the party, be our guest :D
14:38:12 dansmith jaypipes: I wanna discuss an ironic thing with you.. I have a meeting in 20 minutes, but maybe we could hangout after that?
14:39:44 openstackgerrit Ildiko Vancsa proposed openstack/nova master: Add attachment_complete call to volume/cinder.py https://review.openstack.org/493323
14:39:45 openstackgerrit Ildiko Vancsa proposed openstack/nova master: Implement new attach Cinder flow https://review.openstack.org/330285
14:39:45 openstackgerrit Ildiko Vancsa proposed openstack/nova master: Tweak connection_info translation for the new Cinder attach/detach API https://review.openstack.org/493324
14:42:11 dansmith heh
14:42:22 dtantsur :(
14:49:32 cdent smcginnis: I got run and fetch a car that’s been fixed, but will be back in about 30 mins to look at that grenade thing
14:50:27 smcginnis cdent: Sounds good, thanks.
15:19:12 openstackgerrit Hesam Chobanlou proposed openstack/nova master: add online_data_migrations to nova docs adding cli documentation for online_data_migrations to clarify when the command is complete. https://review.openstack.org/493442
15:23:12 openstackgerrit Hesam Chobanlou proposed openstack/nova master: add online_data_migrations to nova docs adding cli documentation for online_data_migrations to clarify when the command is complete. https://review.openstack.org/493442
15:56:45 dansmith dtantsur: https://hangouts.google.com/call/kls2hwod6nhprahlf5mj5i3qveu
15:56:51 dansmith (if interested)
15:57:00 dtantsur gimme a few minutes
16:00:09 dansmith jaypipes: https://review.openstack.org/#/c/492964/5
16:01:57 smcginnis cdent: We may have figured out the failure, testing now. Devstack config with setting NOVA_USE_MOD_WSGI.
16:34:02 cdent smcginnis: that’s what I was going to look for, so that sounds likely. Took me a lot longer to get back than expected: a nearby music festival was letting out. traffic. very traffic.
16:34:58 smcginnis cdent: I love working from home now and not having to deal with that daily.
16:35:14 cdent yeah, me too, which makes me more sensitive to it when I do...
16:36:31 dtantsur dansmith, jaypipes, this concerns me: https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py#L319-L321
16:36:54 dtantsur it may mean that we'll allow users to get "more resources" by PATCHing node.properties..
16:48:04 dansmith dtantsur: I don
16:48:13 vdrok dtantsur: hrm, so it seems we'll have vcpus=0 and vcpus_used=something during deployment
16:48:28 dansmith dtantsur: don't really know what the whole PATCH thing is, nor the behavior of node.properties
16:48:30 dtantsur vdrok: this has to be fixed, but that's not the biggest problem
16:48:47 dtantsur dansmith: this is where we take inventory from. tl;dr it can be changed from API at any moment
16:49:59 dansmith dtantsur: by the admin yes?
16:50:02 dtantsur yes
16:50:21 dtantsur it's not even absolutely crazy: they can power the instance down, and install more RAM in it..
16:50:21 dansmith dtantsur: so ironic discovers it but admin can override?
16:50:33 dtantsur dansmith: the admin sets it initially, but they can change it
16:51:29 vdrok dtantsur: also if we'll be reporting the resources as used if vcpus=0 in get_inventory, does it mean we'll be doing for maintenance'd nodes too?
16:52:19 vdrok ditto for bad power state
16:52:28 dtantsur yes, this is fine
16:52:38 dtantsur we're mostly concerned about reporting wrong inventory for active nodes
16:54:09 vdrok but then, even if it changes, this is all a part of the same periodic task? like, properties changed, _node_resource sees new value, and uses it in get_inventory
16:54:23 vdrok *and it gets used in
16:55:51 dtantsur right, and the Placement sees free resources to schedule on >_<
16:56:13 dtantsur may be not a huge deal, unless the change all of cpus, memory and disk at the same time
16:56:32 dansmith dtantsur: well, it's not a problem at all once RC is in place right?
16:56:48 dtantsur dansmith: as long as all flavors are using it - yes
16:57:12 dtantsur maybe I should not worry about it too much. just document it as a known issue..
16:57:38 dansmith known issue, which goes away in queens when we require RC for all ironic scheduling
16:57:59 dtantsur right, yeah
16:58:52 dtantsur dansmith: it is fine if my changes will affect get_available_resources result too? I'm not sure what it is for
16:59:03 dtantsur and how it is related to get_inventory
16:59:14 dansmith get_inventory will override the other, AFAIK
16:59:35 dtantsur ah, good
17:08:03 openstackgerrit Chris Dent proposed openstack/nova master: Reset client session when placement endpoint not found https://review.openstack.org/493536
17:08:34 cdent gibi, jaypipes, edleafe ^ finally a test for that. much more boring than original plan, but gets the main
17:18:07 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Update api doc with latest updates in api framework https://review.openstack.org/493041
17:19:39 gmann alex_xu: ^^ update api framework doc, some part i will fix in part2 where i ll mention way to add API step by step
17:20:08 gmann alex_xu: i think once this is merged we should add this in pike also.
17:20:41 gmann this is release wise doc and it should be sync with code for pike repo
18:00:18 cdent gmann: did something change to cause the across the board failures in gate but not check in https://review.openstack.org/#/c/489754/
18:08:02 gmann cdent: ah yea, this got merged before -https://review.openstack.org/#/c/491591/
18:08:18 gmann cdent: we moved is_scheduler_filter_enabled from test.py to common
18:08:33 gmann cdent: if you can change that, i can reapprove
18:08:41 cdent yeah, will do, thanks gmann
18:08:54 gmann same way like https://review.openstack.org/#/c/491591/2/tempest/scenario/test_security_groups_basic_ops.py
18:08:58 gmann cdent: thanks
18:14:34 cdent gmann, done. not tested locally, but I think I got it right
18:15:00 gmann cdent: sure, ll check
18:15:04 gmann thanks
18:18:11 cdent dansmith can you re+W this: https://review.openstack.org/#/c/480924/ Andreas says it’s stuck and needs that
18:18:15 cdent please and thank you
18:26:11 cdent thanks melwitt! hadn’t realized you were around
18:27:12 melwitt np
18:53:46 mnaser cdent https://bugs.launchpad.net/nova/+bug/1661360 is this considered resolved in master now? im asking because i am wondering if its good to drop https://github.com/openstack/puppet-nova/blob/master/manifests/wsgi/apache_api.pp#L113
18:53:47 openstack Launchpad bug 1661360 in tripleo "InstanceNotFound due to missing osapi_compute service version when running nova-api under wsgi" [Critical,Fix released]
18:54:06 cdent one sec mnaser
18:54:08 tasker is there a blueprint to support volume snapshots attached to an image-backed instance?
18:57:39 cdent mnaser: yes, it is safe to deploy under wsgi now, but it is important to use the correct app. the one that is deployed by pbr (via the wsgi_scripts entry point) is the right one. it is: nova-placement-api
18:57:52 cdent mnaser: sorry nova-api-wsgi
18:58:08 mnaser cdent https://github.com/openstack/puppet-nova/blob/master/manifests/params.pp#L52 looks like that's put in place
18:58:12 cdent there’s also one for metadata under wsgi now too: nova-metadata-wsgi
18:58:30 mnaser cdent cool! i'll remove the warning, see how puppet ci reacts to deploying via wsgi and report back if i run into anything, thank you!
18:58:44 cdent mnaser: good luck. I hope it works. it’s much better
18:59:03 mnaser cdent i've been looking forward for it since newton :p

Earlier   Later