| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-02-28 | |||
| 15:32:40 | dansmith | spotz: got hosts actually registering with their cell db? | |
| 15:32:51 | dansmith | spotz: compute_nodes and services tables in the cell1 db | |
| 15:35:41 | spotz | dansmith: I thought so, but let me go back to the DB vs output from list_cells | |
| 15:36:02 | dansmith | spotz: -v should show you something at least I thought | |
| 15:36:15 | dansmith | spotz: like "checking cell foo" | |
| 15:36:57 | dansmith | yeah, confirmed | |
| 15:37:01 | spotz | dansmith: I have a nova_cell0 DB no cell1 DB | |
| 15:37:09 | dansmith | spotz: well that's a real problem :P | |
| 15:37:31 | spotz | dansmith: So our create_cell didn't work? | |
| 15:38:04 | dansmith | spotz: create_cell just tells nova where the db is for a given cell | |
| 15:38:16 | dansmith | spotz: you should have an apidb, a cell0 db, and a cell1 db | |
| 15:38:26 | spotz | dansmith: Ok so we're definitely missing steps, I do get this for list_cells though cell1 | 143df967-eb22-4d2c-baff-bfc2538816fd | | |
| 15:38:45 | dansmith | yeah, but that doesn't mean anything if you don't have a db to back it up | |
| 15:38:53 | dansmith | spotz: have you looked through our devref docs mriedem pointed to? | |
| 15:39:44 | spotz | dansmith: yeah that's where the steps came from | |
| 15:40:31 | dansmith | spotz: okay I see, you're doing both of those mappings with the same config, which means they probably point to the same place? | |
| 15:40:47 | dansmith | spotz: an unfortunate side-effect of the config file method for getting to this point :/ | |
| 15:41:00 | macsz | sfinucan: lol what about no :D | |
| 15:41:03 | dansmith | spotz: can you pastebin your cel_mappings table? | |
| 15:41:20 | spotz | dansmith: sure hang on | |
| 15:41:22 | macsz | sfinucan: and also from nose's docs: "nose itself supports python 3, but many 3rd-party plugins do not!" | |
| 15:41:43 | sfinucan | macsz: Who says we need plugins, eh? ;) | |
| 15:42:48 | spotz | dansmith: https://gist.github.com/spotz/42b9d15f016073458742268735644578 | |
| 15:43:12 | dansmith | spotz: oh, no you do have a cell1 db | |
| 15:43:24 | dansmith | spotz: in that db (nova) do you have services and compute_nodes? | |
| 15:45:03 | macsz | sfinucan: I have had enough of beers last week, so i can invest my friday's evenings to convert tox to nose, if you want it :P | |
| 15:45:29 | sfinucan | macsz: *testrepository to nose | |
| 15:45:33 | sfinucan | tox would stay | |
| 15:45:49 | spotz | dansmith: added as a comment on the gist | |
| 15:45:58 | macsz | sfinucan: ah, sorry, was focused on typing "beer" | |
| 15:46:18 | sfinucan | Ima take a look and see if it would even be possible. The testrepository library annoys me in many ways | |
| 15:46:24 | sfinucan | macsz: :D | |
| 15:46:38 | dansmith | spotz: cool, so can you show me the exact result of "nova-manage cell_v2 discover_hosts -v" ? | |
| 15:46:59 | spotz | dansmith: it didn't like -v as a switch when I tried earlier | |
| 15:47:09 | spotz | --debug gave no results to screen | |
| 15:47:39 | spotz | moved it but no help | |
| 15:47:41 | spotz | root@controller:~# nova-manage -v cell_v2 discover_hosts | |
| 15:47:42 | spotz | Option "verbose" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. | |
| 15:48:16 | dansmith | spotz: -v goes after discover_hosts | |
| 15:48:47 | spotz | dansmith: that's no joy nova-manage: error: unrecognized arguments: -v | |
| 15:49:08 | dansmith | spotz: sorry it's --verbose | |
| 15:49:36 | spotz | that worked adding as a comment | |
| 15:50:26 | dansmith | spotz: right so I think it worked before when you did --debug | |
| 15:50:30 | macsz | sfinucan: yeah... and I hate my change with "rm" as well, but I am so fed up with doing it manually, and I could not see the use for having this file around either way... | |
| 15:50:36 | dansmith | spotz: it just didn't say anything because you didn't have --verbose | |
| 15:50:42 | dansmith | spotz: that "checking" means the mapping is there now | |
| 15:50:57 | dansmith | spotz: so you should have that host_mapping in the api db and maybe even be able to create instances | |
| 15:51:29 | spotz | dansmith: 2017-02-28 15:31:57 | NULL | 1 | 2 | compute | | |
| 15:51:54 | dansmith | spotz: I need more context, can you paste the full thing in the gist? | |
| 15:51:55 | spotz | dansmith: I wonder why it didn't work before though, I've run the command before | |
| 15:52:19 | dansmith | spotz: it needs to run after the host shows up in the cell db, so if you run it before you start the service it won't find it, of course | |
| 15:52:58 | spotz | dansmith: It was up but I'll make sure we make a note for that | |
| 15:54:36 | spotz | dansmith: no joy on the instance, checking logs to see if it's something new | |
| 15:55:25 | spotz | dansmith: still no valid hosts in the nova-conductor.log though I do think we made progress:) | |
| 15:56:02 | dansmith | spotz: well paste some logs | |
| 15:59:35 | spotz | dansmith: api, scheduler and conductor logs up | |
| 16:00:01 | mriedem | spotz: where? | |
| 16:00:10 | spotz | mriedem: https://gist.github.com/spotz/42b9d15f016073458742268735644578 | |
| 16:01:00 | spotz | looks like placement-api not found from compute side, gonna peek at apache | |
| 16:01:16 | dansmith | spotz: would be a lot easier to read those as a new gist in fixed-width font, fwiw | |
| 16:01:24 | dansmith | damn docs people and their variable-width nonsense :P | |
| 16:01:44 | spotz | dansmith: Ha I'm not really a docs person:) | |
| 16:01:58 | dansmith | heh | |
| 16:02:05 | mriedem | is the nova-compute service reporting as up? | |
| 16:02:12 | mriedem | nova service-list | |
| 16:02:36 | spotz | mriedem: Yep reporting as up | |
| 16:02:39 | mriedem | hmm, but yeah | |
| 16:02:39 | mriedem | 2017-02-28 09:58:07.852 1621 WARNING nova.scheduler.client.report [req-62fa3550-e8f7-4111-a297-6b24b20a7e1f - - - - -] The placement API endpoint not found. Placement is optional in Newton, but required in Ocata. Please enable the placement service before upgrading. | |
| 16:02:59 | mriedem | is the placement endpoint in "openstack endpoint list"? | |
| 16:03:34 | spotz | mriedem: Brian noted an apache issue last night but he's doing rhel install. Public, internal and admin present | |
| 16:03:39 | mriedem | my guess is, the compute node isn't reporting into placement, so when the scheduler asks placement for somewhere to build an instance, it says it doesn't know about anything and we get NoValidHost | |
| 16:04:01 | mriedem | are the credentials correct in the [placement] section of nova.conf on the compute node? | |
| 16:04:41 | spotz | mriedem: Yep | |
| 16:04:48 | spotz | I can paste them if you want | |
| 16:07:20 | mriedem | did you configure the compute before or after placement was running? | |
| 16:07:28 | mriedem | i.e. do you need to restart the nova-compute service? | |
| 16:07:38 | spotz | I did just notice os_region was there twice, once as the default openstack and the one I set to RegionOne | |
| 16:08:06 | spotz | mriedem: I've restarted compute several times since config. New error Quota exceeded for instances: Requested 1, but already used 10 of 10 instances (HTTP 403) (Request-ID: req-6fedaf70-89b9-4bc4-a978-9ab04090767f) | |
| 16:08:46 | mriedem | so you've already got 10 dead instances in ERROR state right? | |
| 16:08:53 | mriedem | nova list would show 10 ERROR instances | |
| 16:09:00 | mriedem | delete those | |
| 16:09:34 | spotz | mriedem: I've been clearing them out | |
| 16:09:50 | akijak | hi, I've found a bug in nova concerning rebuilding and 'hw_disk_bus' image property (https://bugs.launchpad.net/nova/+bug/1667667), I'm looking for someone who could give me some directions how to fix that :) can you help? | |
| 16:09:50 | openstack | Launchpad bug 1667667 in OpenStack Compute (nova) "Rebuilding instance ignores image's property hw_disk_bus" [Low,Confirmed] | |
| 16:09:51 | spotz | yeah just 2 in error currently | |
| 16:11:19 | efried | Howdy folks, is there a Pike equivalent of https://etherpad.openstack.org/p/ocata-nova-priorities-tracking ? I.e. somewhere I can put up-to-date status & review priorities for the PowerVM driver "subteam"? | |
| 16:11:24 | lyarwood | akijak: hey, have you looked at https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L2679 and how we are passing image_meta around? | |
| 16:11:31 | mriedem | efried: i can create one | |
| 16:11:48 | efried | mriedem Thanks. And wipe the agenda for Thursday's meeting? | |
| 16:11:59 | mriedem | efried: it's only tuesday! | |
| 16:12:09 | mriedem | i get to the thurs meeting agenda by about wed night | |
| 16:12:19 | efried | Heh. Didn't know how much lead time we were supposed to have ;-) | |
| 16:12:23 | mriedem | efried: if you need to change something in the meeting agenda, you can do that now | |
| 16:12:29 | efried | rgr | |
| 16:14:27 | spotz | mriedem: 0 instances in Error, same result | |
| 16:15:15 | dansmith | mriedem: so I noticed some weird quota behavior when I was trying to get my multicell patch working | |
| 16:15:42 | dansmith | spotz: what is the [database]/connection set to in the nova conf that you're using for your apis? | |
| 16:15:49 | dansmith | and conductors | |
| 16:16:01 | dansmith | maybe we're reserving in cell0 | |
| 16:17:00 | mriedem | i assume it's mysql+pymysql://nova:linuxacademy123@controller/nova | |