| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-02-06 | |||
| 19:13:26 | mriedem | which seems less than ideal | |
| 19:16:24 | odyssey4me | nova-conductor: No host-to-cell mapping found for selected host ocata-nova1. Setup is incomplete. | |
| 19:16:42 | odyssey4me | Failed to compute_task_build_instances: Host 'ocata-nova1' is not mapped to any cell | |
| 19:17:48 | hrw | https://marcin.juszkiewicz.com.pl/2018/02/06/graphical-console-in-openstack-aarch64/ - please read ;) | |
| 19:24:15 | tssurya | odyssey4me: can you check the value of the "mapped" column inside compute_nodes table of the nova db (cell1 db), ? if you have this host there, then there should be a record | |
| 19:26:05 | odyssey4me | hmm, select mapped from compute_nodes; gives me an unknown column error | |
| 19:26:50 | odyssey4me | yup, none of those are working | |
| 19:27:02 | odyssey4me | none of the db's have that column | |
| 19:29:49 | tssurya | there should be a table called compute_nodes only in your nova db | |
| 19:29:52 | tssurya | not in the api | |
| 19:30:41 | tssurya | odyssey4me: that is in the cell1's db | |
| 19:31:02 | odyssey4me | yup, the nova db has that table - but the table has no 'mapped' column | |
| 19:32:26 | odyssey4me | tssurya here're the columns present: https://pastebin.com/grmGWEQe | |
| 19:33:11 | odyssey4me | tssurya in case you missed it, this is a newton build (with no cells) being upgraded to an ocata build (with cells being setup) | |
| 19:33:20 | odyssey4me | cells v2 to be clear | |
| 19:35:20 | tssurya | odyssey4me: oh okay, ocata.. | |
| 19:35:49 | odyssey4me | tssurya yep :) ye olde crusty stable code ;) | |
| 19:37:02 | odyssey4me | mriedem any further thoughts or ideas? if not I'll work on fixing up the log capturing so that I can point you at a proper set of logs to peruse | |
| 19:37:54 | mriedem | odyssey4me: sorry was eating lunch, | |
| 19:38:01 | mriedem | i think the compute_nodes.mapped column was added in ocata | |
| 19:38:31 | mriedem | the host 'ocata-nova1' is a problem i think, since an earlier paste showed it was discovered in 2 different cell mappings | |
| 19:38:47 | mriedem | https://pastebin.com/mSwpZybQ | |
| 19:39:13 | mriedem | at this point i'd probably need the logs from an untouched env | |
| 19:39:38 | odyssey4me | alright, thanks much for your time and input so far | |
| 19:39:41 | mriedem | yw | |
| 19:40:21 | odyssey4me | I'll work on getting the log collection fixed up so that we can debug better. | |
| 19:44:30 | efried | mriedem: I'm going to put up three pairs of patches to dig a bit deeper, see which part of ServiceTokenAuthWrapper is actually busted. | |
| 19:44:36 | efried | mriedem: Unless you have some other plan. | |
| 19:44:42 | mriedem | so 6 patches? | |
| 19:45:04 | mriedem | i don't, no :) | |
| 19:45:21 | mriedem | we might have to put this in the release notes as a known issue | |
| 19:45:46 | efried | Yeah, going to pass through each of the service_auth-y bits of ServiceTokenAuthWrapper - which I have to do in ksa - and then a blank Nova patch for each that Depends-On its ksa buddy. | |
| 19:45:52 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Create volume attachment during boot from volume in compute https://review.openstack.org/541420 | |
| 19:45:53 | mriedem | ildikov: ^ cleans up that bfv legacy attach stuff | |
| 19:47:10 | ildikov | mriedem: tnx, looking | |
| 19:47:36 | openstackgerrit | Matt Riedemann proposed openstack/nova master: DNM: debug nova-api service version check during boot from volume https://review.openstack.org/540992 | |
| 19:51:04 | mriedem | holy f 679 check queue length | |
| 19:52:21 | melwitt | I saw there was a status message about zuul from 02:30 having problems | |
| 19:52:51 | melwitt | "[02:30:17] -openstackstatus-NOTICE: Our Zuul infrastructure is currently experiencing some problems and processing jobs very slowly, we're investigating. Please do not approve or recheck changes for now." | |
| 19:53:00 | mriedem | yeah i knew about that | |
| 19:53:22 | mriedem | melwitt: you might want to weigh in on this https://review.openstack.org/#/c/532361/ | |
| 19:53:37 | mriedem | given you'll be the ptl of the project that ruined removing mox in rocky for all projects | |
| 19:53:48 | melwitt | -_- | |
| 19:54:13 | mriedem | as the lame duck ptl, i can only fire shots across the bow | |
| 19:56:02 | mriedem | bauzas: don't forget https://review.openstack.org/#/c/526095/ | |
| 19:56:03 | cdent | melwitt, mriedem : I recommend either lying ("yeah, sure we can will do it") or just do it | |
| 19:56:24 | artom | Strictly speaking, *removing* mox is easy | |
| 19:56:25 | mriedem | cdent: i assumed we'd do what we have done in previous releases, | |
| 19:56:33 | artom | *Replacing* it with mock, OTOH... | |
| 19:56:34 | mriedem | which is we can work on it as a low priority thing | |
| 19:56:48 | mriedem | so it would be what doug said, which is forward progress | |
| 19:57:56 | cdent | mriedem: why not, since it is a goal™, make it higher priority and use the fact that it is a goal as an excuse to de-prioritize other stuff? | |
| 19:58:43 | mriedem | i don't consider it high priority | |
| 19:59:11 | cdent | isn't that what "openstack wide goals" means? | |
| 19:59:13 | melwitt | yeah. while I agree it's a fine thing to do, it's not trivial to review and takes away from reviewing important things like bug fixes and placement work that our users want and need | |
| 19:59:45 | cdent | (to be clear, I'm not too wed to the mox thing, but I do think that the goals process should mean _something_) | |
| 19:59:48 | melwitt | and I personally don't get the argument of "maintaining mox3" because, what is there to maintain? | |
| 20:00:34 | mriedem | right, i don't think 'retiring mox3' is a high priority | |
| 20:00:37 | openstackgerrit | Eric Fried proposed openstack/nova master: DNM: Debug nova-next failure: user headers https://review.openstack.org/541427 | |
| 20:00:48 | cdent | melwitt: that seems a far more valid response to the goal process | |
| 20:00:54 | mriedem | ffs, sqlalchemy-migrate was forked over for openstack usage, and we're still using that gd thing forever | |
| 20:01:07 | cdent | much to the chagrin of zzzeek | |
| 20:01:10 | mriedem | this would be like a goal to migrate every project from sqla-migrate to that other thing | |
| 20:01:17 | mriedem | alembic | |
| 20:01:29 | mriedem | which is arguably a more useful goal than removing mox tests | |
| 20:02:17 | cdent | perhaps one of you should respond to zane with the real reasons you don't want to do it? | |
| 20:02:33 | dansmith | didn't mriedem already do that? | |
| 20:03:03 | cdent | dansmith: I'm not entirely sure. I just read his most recent response and that doesn't seem to. | |
| 20:03:16 | mriedem | i did, | |
| 20:03:18 | dansmith | wasn't his first resposne "because we can't possibly get it done" ? | |
| 20:03:24 | openstackgerrit | Eric Fried proposed openstack/nova master: DNM: Debug nova-next failure: invalidate https://review.openstack.org/541430 | |
| 20:03:26 | cdent | But, dansmith, I was just trying to say "instead of talking about it here, maybe talk about it on the review" | |
| 20:03:32 | mriedem | i left comments on the review | |
| 20:03:44 | cdent | right, dansmith, that's not the same as "because we don't think it is worth doing because why not support mox3 forever" | |
| 20:03:47 | melwitt | I will be commenting on the review. I'm just talking here for now since we were talking about it | |
| 20:04:01 | mriedem | i won't -1 it because i don't want to further the notion that nova is the project that blocks progress on everything in openstack | |
| 20:04:27 | cdent | yeah, I'm not saying we shouldn't talk about it here too, just hoping it gets reflected there too (it's not like I'd ever want to stop talking about stuff) | |
| 20:04:42 | dansmith | cdent: okay I don't see the difference between what he said (and the implications on mox3) and what we're saying here, but... that's fine | |
| 20:04:43 | cdent | mriedem: but isn't that exactly what zane has implied? | |
| 20:05:12 | dansmith | cdent: that nova is being a blocker on purpose? | |
| 20:05:14 | openstackgerrit | Eric Fried proposed openstack/nova master: DNM: Debug nova-next failure: connection params https://review.openstack.org/541432 | |
| 20:05:15 | mriedem | cdent: yes it is, but i don't think that means that other projects which actually can get it done and find value in doing it, can/should do it | |
| 20:05:17 | dansmith | I don't think he said that | |
| 20:05:36 | cdent | dansmith: I know it is stupid that for some reason the details of this stuff become such a thing, but they do. The conversations that various members of the TC have had about the goals process have been _involved_ | |
| 20:05:37 | mriedem | *shouldn't do it | |
| 20:05:49 | cdent | dansmith: not on purpose, but sort of de facto | |
| 20:07:18 | cdent | but yeah, whatever, I personally don't really care about the mox goal (or any of them really) | |
| 20:07:30 | bauzas | mriedem: +Wd, thanks | |
| 20:07:36 | efried | mriedem: It occurs to me that disabling bits of the service auth token will have impact outside of glance. | |
| 20:07:37 | bauzas | oh man, mox | |
| 20:07:59 | cdent | mriedem: thanks for that final explicit comment | |
| 20:09:14 | bauzas | wow, https://review.openstack.org/#/c/534605/ too | |
| 20:09:43 | mriedem | bauzas: that one is much more focused and easy | |
| 20:09:52 | mriedem | all it is is make the debug option mutable | |
| 20:09:58 | mriedem | with a sighup | |
| 20:10:05 | mriedem | it's not "make all options mutable" | |
| 20:10:11 | cdent | it was initially | |
| 20:10:12 | bauzas | I'm still chilling | |
| 20:10:20 | cdent | but I nipped it | |
| 20:10:30 | bauzas | anyway, time will see | |