| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-03-16 | |||
| 21:48:30 | mriedem | heh "Lists all running Compute services for a tenant," that's wrong | |
| 21:48:34 | mriedem | services are not per-tenant | |
| 21:48:40 | mriedem | sdague: more api-ref fun ^ | |
| 21:49:04 | mriedem | that should be "for a region" right? | |
| 21:49:11 | mriedem | or a cell i guess | |
| 21:49:28 | mriedem | but i'm not sure we want to start using the c word in the api ref yet | |
| 21:49:29 | dansmith | not a cell | |
| 21:49:34 | dansmith | yeah, don't say that | |
| 21:49:35 | mriedem | true, | |
| 21:49:39 | mriedem | api isn't in the cell | |
| 21:49:42 | dansmith | say for a deployment or region or something | |
| 21:49:45 | mriedem | yeah | |
| 21:50:39 | melwitt | dtp: to see how it's used, look at nova/db/sqlalchemy/models.py at Service to see the data model, then look at nova/objects/service.py to see the object API and you can sort of trace how it's used from using that as a starting point. also look at nova/service.py | |
| 21:51:40 | dtp | ok, thanks | |
| 21:51:52 | mriedem | dansmith: guh, we need to update this at some point too https://docs.openstack.org/ops-guide/arch-scaling.html#cells-and-regions | |
| 21:52:45 | dansmith | hmm, yeah | |
| 21:58:41 | mriedem | dansmith: i've opened a docs bug https://bugs.launchpad.net/openstack-manuals/+bug/1673616 | |
| 21:58:41 | openstack | Launchpad bug 1673616 in openstack-manuals "Scaling in Operations Guide - cells section needs to be updated" [Undecided,New] | |
| 21:58:51 | mriedem | dansmith: you might want to add/update/fix whatever i put in the description | |
| 22:00:12 | dansmith | okay | |
| 22:00:29 | dansmith | looks alright to me at first skim but I'll think on it a bit | |
| 22:01:15 | openstackgerrit | Matt Riedemann proposed openstack/nova master: api-ref: fix description in os-services https://review.openstack.org/446757 | |
| 22:18:05 | openstackgerrit | melanie witt proposed openstack/nova master: Improve descriptions for hostId, host, and hypervisor_hostname https://review.openstack.org/446761 | |
| 22:18:37 | melwitt | mriedem: ^ patch for the host* api-ref stuff | |
| 22:26:39 | openstackgerrit | Octave Orgeron proposed openstack/nova-specs master: Enables MySQL Cluster Support for Nova https://review.openstack.org/446631 | |
| 22:33:55 | mriedem | melwitt: thanks, comments inline | |
| 22:45:56 | cfriesen | is there a known issue in newton where pre_live_migration() can hit an RPC timeout waiting for a really big glance image to download to the destination? | |
| 22:56:52 | openstackgerrit | OpenStack Proposal Bot proposed openstack/nova master: Updated from global requirements https://review.openstack.org/446777 | |
| 23:10:52 | mriedem | easy bug fix https://bugs.launchpad.net/nova/+bug/1673628 | |
| 23:10:52 | openstack | Launchpad bug 1673628 in OpenStack Compute (nova) "api-ref: 'tags' field is not in response parameters docs for "GET /servers/{server_id}"" [Medium,Confirmed] | |
| 23:11:07 | mriedem | cfriesen: that's not a newton specific limitation | |
| 23:11:15 | mriedem | that's a latent issue forever | |
| 23:11:41 | mriedem | cfriesen: see https://review.openstack.org/#/c/419662/1 | |
| 23:13:36 | openstackgerrit | Matt Riedemann proposed openstack/nova-specs master: Repropose tag-instance-when-boot https://review.openstack.org/415315 | |
| 23:13:45 | mriedem | Kevin_Zheng: are you ok with the updates i made to ^? | |
| 23:18:20 | mriedem | melwitt: i think your func test change might be causing a race issue with global state | |
| 23:20:01 | mriedem | yeah you need to use fake.restore_nodes | |
| 23:24:09 | sbezverk | hello, I need to check liveness of nova placement api process, could you recommen any methods to check not just that the socket is opened but also that the process is alive and sane. | |
| 23:24:26 | mriedem | sbezverk: make a curl request to / ? | |
| 23:24:54 | mriedem | that's testing much more than just the process is running | |
| 23:25:01 | sbezverk | mriedem: would it not just check apachie liveness? | |
| 23:25:28 | mriedem | making a curl request to the actual endpoint's root would provide the versions available, which means it's up and running and accepting requests | |
| 23:25:31 | mriedem | and serving a response | |
| 23:25:51 | mriedem | i don't know if a token is required for GET / on the placement endpoint though | |
| 23:26:22 | sbezverk | mriedem: cool, thank you very much. I was not sure if curling for root is sufficient | |
| 23:26:53 | mriedem | yup that will get you the versions available | |
| 23:39:19 | openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Add api-ref for filter/sort whitelist https://review.openstack.org/421760 | |
| 23:39:52 | sbezverk | mriedem: any known issue with placement in ocata you are aware? | |
| 23:39:55 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix functional regression/recreate test for bug 1671648 https://review.openstack.org/446685 | |
| 23:39:56 | openstack | bug 1671648 in OpenStack Compute (nova) ocata "Instances are not rescheduled after deploy fails" [High,In progress] https://launchpad.net/bugs/1671648 - Assigned to Matt Riedemann (mriedem) | |
| 23:40:21 | mriedem | sbezverk: nothing major for placement that i can think of | |
| 23:40:33 | mriedem | sbezverk: we did release some fixes for ocata in 15.0.1 earlier in the week, | |
| 23:40:39 | mriedem | so if you're upgrading, i'd use 15.0.1 | |
| 23:41:13 | sbezverk | mriedem: I brining up at the gate ocata images in kolla-kubernetes project | |
| 23:41:41 | sbezverk | what worked with stable/newton seems broken in ocata, but it might be our issue too.. | |
| 23:42:57 | mriedem | sbezverk: i guess i'd need a more specific example of what's not working | |
| 23:43:56 | Sukhdev | dear Nova folks, after fresh install of devstack, Nova continue to return "No Valid host found" error - can somebody provide some wisdom, please? See the details here - http://paste.openstack.org/show/603034/ | |
| 23:44:29 | sbezverk | mriedem: :) glad you asked. When I start instance I get " Placement API service is not responding." in nova scheduler log | |
| 23:44:59 | mriedem | sbezverk: are you starting the placement service before the nova-scheduler service? | |
| 23:45:04 | sbezverk | mriedem: it looks like connectivity issue but nothing change other than just version of images | |
| 23:45:12 | mriedem | is there a "placement" endpoint in the service catalog? | |
| 23:45:37 | sbezverk | mriedem: yep, service catalog shows placement entries | |
| 23:45:38 | mriedem | is nova.conf for the scheduler configured to use placement in the [placement] section, and are the credentials in there correct? | |
| 23:46:13 | sbezverk | mriedem: all valid points, but as I said nothing changed in the gate job other than image version | |
| 23:46:30 | mriedem | the scheduler didn't use placement in newton, | |
| 23:46:34 | mriedem | so that's a main different with ocata | |
| 23:46:38 | sbezverk | mriedem: was there any change in endpoint registrations between newton and ocata? | |
| 23:46:46 | mriedem | in newton, only the nova-compute service tried connecting to the placement service | |
| 23:46:55 | mriedem | no there was no change in endpoint registration | |
| 23:47:04 | sbezverk | mriedem: AHHH I did not know that | |
| 23:47:17 | mriedem | i'd compare the nova.conf [placement] sections between your nova-compute and nova-scheduler services | |
| 23:47:33 | mriedem | make sure that whatever you did to make it work for nova-compute, that you do that for nova-scheduler too | |
| 23:47:55 | mriedem | sbezverk: https://docs.openstack.org/developer/nova/placement.html#upgrade-notes in case you haven't seen that yet | |
| 23:48:04 | sbezverk | mriedem: if you have 2 minutes I can get it, would be great if you cold take a look at it | |
| 23:48:21 | mriedem | oh boy | |
| 23:48:23 | mriedem | :) | |
| 23:48:42 | mriedem | Sukhdev: is the nova-compute service running and listed as up when you run "nova service-list"? | |
| 23:49:24 | Kevin_Zheng | mriedem: yeah thanks | |
| 23:49:50 | mriedem | Kevin_Zheng: yeah as in you agree with the changes, or yeah as in you will look? | |
| 23:50:10 | Sukhdev | mriedem : yes - see here - http://paste.openstack.org/show/603035/ | |
| 23:50:34 | Kevin_Zheng | agree：） | |
| 23:51:13 | sbezverk | mriedem: http://paste.openstack.org/show/603036/ | |
| 23:51:23 | sbezverk | mriedem: look identical to me :( | |
| 23:52:12 | Kevin_Zheng | thanks alot | |
| 23:53:17 | Sukhdev | mriedem : anything else that I should look for? BTW, I am running the latest code (did git pull 5-10 days ago) | |
| 23:56:24 | mriedem | Kevin_Zheng: yw | |
| 23:56:48 | mriedem | Sukhdev: check the nova-compute logs for errors | |
| 23:57:22 | mriedem | sbezverk: those look the same to me too | |
| 23:57:32 | mriedem | sbezverk: the services all run in individual containers right? | |
| 23:58:39 | mriedem | is there anything in the deployment tooling that has to do anything specific for the compute service wrt placement that the scheduler container isn't doing? | |
| #openstack-nova - 2017-03-17 | |||
| 00:03:08 | sbezverk | mriedem: correct about individual containers | |
| 00:03:42 | sbezverk | mriedem: in paralel I have a job running newton images and it passes wihtout issues | |
| 00:04:19 | sbezverk | mriedem: I will continue debugging to confirm connectivity issue.. curl should tell me that | |
| 00:04:52 | Sukhdev | mriedem : I see lots of time out errors in n-cpu.logs - here is a sample - http://paste.openstack.org/show/603037/ | |
| 00:09:25 | mriedem | Sukhdev: do you have any customizations to the local.conf you used to stack devstack? can you post your local.conf in a paste? | |
| 00:09:44 | mriedem | seems you're having rpc issues, which would mean investigating the oslo.messaging configuration values in nova.conf | |
| 00:13:30 | Sukhdev | mriedem : here is local.conf - http://paste.openstack.org/show/603039/ | |
| 00:13:33 | openstackgerrit | OpenStack Proposal Bot proposed openstack/nova master: Updated from global requirements https://review.openstack.org/446777 | |
| 00:15:18 | Sukhdev | mriedem : here is /etc/nova/nova.conf - http://paste.openstack.org/show/603040/ | |
| 00:22:10 | Sukhdev | mriedem : can you spot anything suspicious in either of those configs? | |