Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-01-31
10:23:49 andymccr morning all - im having a few issues with the placement service, was hoping for a bit of guidance. The first is that it seems you can't define which endpoint interface compute connects to placement on. (I put a PR in for that which worked for me but I may be way off the mark: https://review.openstack.org/#/c/426163/ )
10:23:50 openstackgerrit Alex Szarka proposed openstack/nova: Transform instance.add_fixed_ip notification https://review.openstack.org/332876
10:25:09 andymccr secondly, i get a 409 on the first PUT from the compute host, 200 on all GETs but then 400's on subsequent PUTs. The errors are vague though - I'm not too sure what steps I can take to debug that further.
10:29:44 johnthetubaguy andymccr: hmm, interesting
10:29:53 johnthetubaguy andymccr: the endpoint is picked using the keystone service catalog
10:30:02 johnthetubaguy (or should be)
10:30:34 andymccr johnthetubaguy: yeah it is, but it forces "public" afaict because the keystoneauth endpoint_filter doesnt include an interface option when passed by nova
10:30:34 johnthetubaguy andymccr: oh, the interface
10:30:46 johnthetubaguy should we really default to private?
10:31:05 andymccr my pr defaults it to public - so in essence, no change if you don't set it
10:31:33 andymccr internal works for my usecase, but not sure it'd work for everybody - so figured its best to not change things like that midway :) rather make them changeable.
10:32:04 andymccr i imagine in things like role gates/devstack it makes no difference since the endpoints are probably all the same regardless of internal/public
10:32:21 johnthetubaguy its an admin only API (at least by design) so its probably a mute point
10:33:01 andymccr if its admin only perhaps it should default to admin instead?
10:33:32 andymccr from my pov, i need it to not be public or else i have to do some munging when i create endpoints to ensure the public endpoint is created differently to all other public endpoints and uses the internal ip
10:34:24 johnthetubaguy oh, possibly, yeah, default to admin makes more snese
10:35:47 johnthetubaguy in theory non admin users would just get 403 errors I think, but its good to message that better
10:36:23 johnthetubaguy bauzas: cdent: do you have better context on the intended deployment stuff, for our OSA friends?
10:37:09 bauzas johnthetubaguy: sorry, I had an internal problem
10:37:29 bauzas and I'm very busy, so could you please explain me the issue ?
10:37:55 johnthetubaguy bauzas: service catalog, we default to accessing the "public" URL for placement
10:38:05 johnthetubaguy that seems odd, given its an admin only API
10:38:10 andymccr (and you can't override it afaict)
10:38:25 bauzas lemme verify something
10:38:47 bauzas I think nova just doesn't use the public field
10:39:00 johnthetubaguy it seems to, because of how keystone auth works
10:39:19 johnthetubaguy infact it only uses the public field, if I am remembering correctly
10:39:37 bauzas :/
10:41:31 toabctl mriedem, hey. anything I can do to push https://review.openstack.org/#/c/402318/ ?
10:41:33 bauzas johnthetubaguy: AFAIK, we don't have yet policies
10:42:43 johnthetubaguy bauzas: I was wondering about that... uh, oh
10:47:13 johnthetubaguy I think we are OK for policy, looking at this: https://github.com/openstack/nova/blob/a0100d61a9531a3e32f2c72eab54157c32114f5b/nova/api/openstack/placement/handler.py#L175
10:48:28 bauzas agreed
10:50:05 gmann johnthetubaguy: can you look into release notes patch of sort/filter things - https://review.openstack.org/#/c/421760/
10:50:18 gmann johnthetubaguy: i have removed the all_tenant policy bits from there
10:50:22 bauzas johnthetubaguy: when I wrote the PlacementFixture, I saw that it was returning 401s if I wasn't auth'd
10:50:27 johnthetubaguy gmann: ah, we trimmed it down, cool
10:50:29 bauzas now I remember that
10:50:40 gmann yea
10:50:41 johnthetubaguy bauzas: cool
10:50:52 bauzas johnthetubaguy: but that's easily testable thanks to the fixture I wrote, I guess
10:51:29 johnthetubaguy bauzas: there is that other patch I was taking about before, the problem there I think is that some of the code added around allocations is unreachable, and doesn't fix things for instances created after we start reporting custom classes, but I could be reading it all wrong
10:51:46 johnthetubaguy andymccr: sorry, I got fixated on the first issue
10:51:54 johnthetubaguy andymccr: you had a second question
10:52:00 bauzas johnthetubaguy: I briefly looked at the code, but I need time for loading context in mind
10:52:01 johnthetubaguy lots of API errors
10:52:08 johnthetubaguy bauzas: ack
10:52:10 bauzas and I'm awfully busy
10:52:11 bauzas :(
10:52:22 johnthetubaguy bauzas: understood
10:57:31 johnthetubaguy gmann: I worry about advertising the alias
10:58:39 andymccr johnthetubaguy: yeah i'm trying to understand what a 409 would mean in terms of what is actually conflicting - the endpoint seems ok since it's accepting GET requests (responding with a 200), but any PUT requests are returning bad request 400, and the initial put request returns a 409 conflict
10:59:01 andymccr my understanding is the resource already exists, but im wondering where or what i can do to debug that a bit further
10:59:06 johnthetubaguy andymccr: DB configuration is bad, maybe?
10:59:40 andymccr johnthetubaguy: yeah thats what im thinking - so i noticed the nova_placement db that is created isn't populated, perhaps it needs write permissions that i havn't set up properly
11:00:04 johnthetubaguy andymccr: needs a new nova-manage cmd running
11:00:10 andymccr from a nova perspective i just need to run the nova-manage api_db sync command?
11:01:17 johnthetubaguy andymccr: no
11:01:30 gmann johnthetubaguy: i see your point, and agree that can avoid users using those alias. ll update those in respective attribute description.
11:01:34 johnthetubaguy andymccr: oh... well maybe, I can't remember now actually
11:02:01 andymccr johnthetubaguy: i think i must be missing something because it's not quite happening :)
11:02:09 johnthetubaguy gmann: awesome, thank you
11:02:30 johnthetubaguy andymccr: so do you have it configured the same as the API DB in the conf?
11:02:53 openstackgerrit Bob Ball proposed openstack/nova: Fix live migrate with XenServer https://review.openstack.org/424428
11:03:07 andymccr johnthetubaguy: no there is a separate db
11:03:12 andymccr or rather we've setup a separate database
11:03:18 johnthetubaguy andymccr: OK, that sounds good
11:03:34 andymccr with a separate user
11:03:59 andymccr but then the api_db sync doesn't populate the new db - so i wonder if its a permissions issue, although i imagine it would pull the connection string from nova.conf (for the placement db?)
11:04:34 johnthetubaguy andymccr: oh, interesting. it should pull that from nova.conf, but I wonder if that works
11:07:30 johnthetubaguy andymccr: so that is what devstack does: https://github.com/openstack-dev/devstack/blob/master/lib/placement#L143
11:08:37 openstackgerrit Stephen Finucane proposed openstack/nova: Ignore IOError when creating 'console.log' https://review.openstack.org/392643
11:09:15 andymccr johnthetubaguy: hmm yeah, so that pretty much lines up - i tested with --conf-file but let me test that again
11:09:23 sfinucan artom, mdbooth: ^^^
11:09:46 johnthetubaguy andymccr: this is our nova.conf in the gate: http://logs.openstack.org/63/426163/2/check/gate-tempest-dsvm-neutron-full-ubuntu-trusty/a5ece9b/logs/etc/nova/nova.conf.txt.gz
11:10:39 openstack Launchpad bug 1658877 in OpenStack Compute (nova) "live migration failed with XenServer as hypervisor" [High,In progress] - Assigned to Bob Ball (bob-ball)
11:10:39 BobBall mriedem: See my updates to https://bugs.launchpad.net/nova/+bug/1658877 . I'd very much hope we can still aim for a fix in Ocata as changes to the object are needed and I understand these are particularly difficult to backport?
11:11:09 mdbooth sfinucan: You want to add a test for ignoring EPERM? ;)
11:11:19 johnthetubaguy andymccr: did you configure api_database with the placement database, and use the correct conf file when calling api_sync?
11:11:30 sfinucan mdbooth: Damn - you're quick. Had just started fixing that :)
11:12:05 andymccr johnthetubaguy: yes on the config file, but we have a the placement_database section defined.
11:12:26 johnthetubaguy andymccr: something tells me we screwed up, and nova-manage doesn't read that
11:13:01 andymccr johnthetubaguy: interesting, let me test and remove that section
11:13:06 andymccr i assume it'll just default to the api_database then?
11:13:22 johnthetubaguy andymccr: yeah, we will want to not do that, but it would be good to see that fix your issues
11:14:04 andymccr johnthetubaguy: although - i know the puppet team has dedicated configuration for nova_placement database.
11:14:28 andymccr but lets test
11:14:35 andymccr im somewhat sure its db related
11:14:37 johnthetubaguy andymccr: yeah, we should do a separate database, but it would be good to see if that fixes things
11:16:57 openstackgerrit Takashi NATSUME proposed openstack/nova: Avoid forcing translation on logging calls https://review.openstack.org/413876
11:18:26 johnthetubaguy bauzas: have you noticed how you can't have a separate placement database using the placement_database settings? seems in the gate we test with a single shared API database
11:24:18 openstackgerrit Stephen Finucane proposed openstack/nova: Ignore IOError when creating 'console.log' https://review.openstack.org/392643
11:31:34 openstackgerrit Andy McCrae proposed openstack/nova: Allow placement endpoint interface to be set https://review.openstack.org/426163
11:38:54 rpodolyaka johnthetubaguy: yeah, a separate placement db must only be possible with https://review.openstack.org/#/c/362766/
11:40:21 rpodolyaka I wonder if we left placement_database conf group intentionally in this revert - https://review.openstack.org/#/c/360638/ or by mistake
11:42:26 johnthetubaguy rpodolyaka: ah, right, I couldn't see that code when I went digging
11:43:45 johnthetubaguy andymccr: so I think we could make the placement DB separate by having a placement.conf that is used for the nova-manage call and the placement API, where the api-database is configured as the placement database
11:44:04 johnthetubaguy andymccr: seems we are missing some bits on our side for that to work... more sensibly
11:45:10 openstackgerrit Abhishek Kekane proposed openstack/nova: Use is_valid_cidr and is_valid_ipv6_cidr from oslo_utils https://review.openstack.org/320375
11:46:51 andymccr johnthetubaguy: ahh ok - weird. i wonder how the puppet stuff works then - im running a few tests now wrt not using a set placement_db

Earlier   Later