| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-02-08 | |||
| 15:05:54 | EmilienM | mriedem: it works fine today when I deploy OpenStack from trunk except nova which is pin to ocata-3 | |
| 15:05:56 | dansmith | cdent: just call me captain obvious | |
| 15:06:17 | EmilienM | mriedem: I'm having a CI test patch that deploy nova from trunk and it fails | |
| 15:06:30 | EmilienM | mriedem: but like johnthetubaguy and dansmith said, it was probably silently failing? | |
| 15:06:58 | EmilienM | mriedem: if you want I can show you how it works with ocata-3 and you can tell if nova placement/ novacompute are actually working together | |
| 15:06:59 | owalsh | EmilienM: don't we have POST requests in the ocata-3 placement-api log? | |
| 15:07:21 | EmilienM | owalsh: let me check a sec | |
| 15:07:28 | mriedem | EmilienM: so...last week tripleo ci was testing from nova trunk and things worked, but now it's nova ocata-3 (which is older than rc1) and things aren't working? | |
| 15:07:58 | owalsh | mriedem: ocata-3 works, trunk does not | |
| 15:08:19 | mriedem | owalsh: ok | |
| 15:08:22 | EmilienM | cdent, owalsh: that is an ocata-3 CI job: http://logs.openstack.org/50/330050/17/check/gate-tripleo-ci-centos-7-nonha-multinode/95f4ed9/logs/subnode-2/var/log/nova/nova-placement-api.txt.gz#_2017-02-08_13_22_00_143 | |
| 15:08:42 | EmilienM | mriedem: no, we don't test nova-trunk in tripleo, we manually pin to ocata-3 because trunk doesn't work for us | |
| 15:08:47 | EmilienM | mriedem: we do that since days | |
| 15:08:52 | EmilienM | since ocata-3 actually | |
| 15:09:07 | EmilienM | cdent, owalsh : is the POST means we're good? | |
| 15:09:40 | cdent | EmilienM: yes, that POST means "a resource provider was created" | |
| 15:10:01 | cdent | and in the context of this stuff, the only resource provider that ought to be create is a compute node | |
| 15:10:08 | EmilienM | cdent: cool so it works fine with ocata-3 | |
| 15:10:19 | EmilienM | and btw, credentials and config are same | |
| 15:10:29 | EmilienM | the only thing that change is an rpm that contains nova code | |
| 15:10:31 | cdent | EmilienM: is this using eventlet on nova-api? | |
| 15:10:38 | EmilienM | cdent: let me check | |
| 15:10:53 | EmilienM | cdent: http://logs.openstack.org/50/330050/17/check/gate-tripleo-ci-centos-7-nonha-multinode/95f4ed9/logs/subnode-2/var/log/httpd/ | |
| 15:10:55 | EmilienM | eventlet | |
| 15:11:06 | EmilienM | otherwise we would have httpd logs in ^ | |
| 15:12:56 | siva_krishnan | mdbooth: can you please review this spec whenever you find time https://review.openstack.org/#/c/429913/ | |
| 15:13:01 | EmilienM | cdent: and now, if you look at our logs, we don't have the POST anymore | |
| 15:13:16 | EmilienM | could it be related to cells v2? | |
| 15:14:13 | cdent | Is ocata-3 the stable/ocata branch or is there some other hash I can use directly? | |
| 15:14:30 | dansmith | where are the package build logs so we can see the nova hash being tested? | |
| 15:15:09 | mriedem | EmilienM: if the tripleo jobs pin nova at certain points, how do the tripleo jobs in nova's experimental queue actually test the nova change? | |
| 15:15:21 | EmilienM | cdent: rc1 is stable/ocata for nova. In tripleo we don't have stable/ocata yet | |
| 15:15:39 | EmilienM | mriedem: they build it, the pin doesn't work | |
| 15:15:45 | EmilienM | mriedem: so we build nova from master in that case | |
| 15:16:15 | dansmith | EmilienM: so a change against tripleo tests against the pin and a change against nova tests against master+change ? | |
| 15:16:31 | EmilienM | dansmith: exactly | |
| 15:16:46 | EmilienM | we use "delorean" to build rpms from git commits | |
| 15:16:55 | EmilienM | please admit the name is cool | |
| 15:17:11 | mriedem | didn't that guy rip everyone off and go bankrupt? | |
| 15:17:19 | dansmith | yes | |
| 15:17:33 | EmilienM | lol | |
| 15:18:13 | owalsh | ...and get arrested with a suitcase full of coke IIRC | |
| 15:18:22 | mdbooth | siva_krishnan: Something lyarwood was trying to do with rescue was simply add an alternate boot image and set it as the boot volume. For unrescue we do the reverse operation. IIRC this also makes live-migration pretty simple to fix. | |
| 15:18:55 | mdbooth | And it makes rescue better, because the rescue environment has all ephemeral disks and cinder volumes. | |
| 15:19:00 | dansmith | cdent: looking at things that have changed since o3, I see your generation patch | |
| 15:19:02 | mdbooth | As well as the original root disk. | |
| 15:19:06 | dansmith | and the scheduler calling placement | |
| 15:19:16 | mdbooth | siva_krishnan: Could you resurrect that instead? | |
| 15:19:21 | dansmith | cdent: but not much else | |
| 15:19:48 | siva_krishnan | mdbooth: will look into that :) | |
| 15:20:05 | mdbooth | siva_krishnan: lyarwood had come at some point. Hassle him about it :) | |
| 15:20:22 | mriedem | EmilienM: shouldn't you be testing at the highest rc1? | |
| 15:20:26 | mriedem | rather than master? | |
| 15:20:59 | mriedem | https://github.com/openstack/nova/compare/15.0.0.0b3...15.0.0.0rc1 | |
| 15:21:08 | mdbooth | siva_krishnan: Also, does this need a spec? The spec-level stuff would seem to me to be: 'all drivers should be able to live migrate rescued instances if possible'. | |
| 15:21:16 | mdbooth | +1 agreed :) | |
| 15:21:21 | dansmith | mriedem: oh right, I've been thinking of rc1 not o3 | |
| 15:21:24 | mriedem | mdbooth: i think there is already a spec for that... | |
| 15:21:46 | mdbooth | mriedem: Yeah, I was just wondering if the implementation detail of that was suitable for a spec. | |
| 15:22:01 | EmilienM | openstack-nova-common-15.0.0-0.20170208055035.fefac70.el7.centos.noarch | |
| 15:22:02 | mriedem | mdbooth: https://review.openstack.org/#/c/347161/ | |
| 15:22:05 | EmilienM | that's our version in CI | |
| 15:22:16 | EmilienM | where fefac70 is the last commit in Nova | |
| 15:22:17 | mriedem | mdbooth: that was accepted in newton, didn't make it, then when reviewing the spec again for ocata i raised some issues | |
| 15:22:26 | mriedem | which weren't resolved and now paul-carlton is gone i think | |
| 15:22:30 | dansmith | EmilienM: that's not a nova commit afaict | |
| 15:22:32 | mdbooth | mriedem: siva_krishnan has a new one | |
| 15:22:37 | EmilienM | which is https://review.openstack.org/#/c/429008/ | |
| 15:22:40 | EmilienM | (based on master) | |
| 15:22:44 | mdbooth | https://review.openstack.org/#/c/429913/ | |
| 15:22:45 | dansmith | ah | |
| 15:22:46 | EmilienM | dansmith: it is ^ | |
| 15:23:51 | EmilienM | what is weird is that https://review.openstack.org/#/c/429008/ is passing tripleo multinode job | |
| 15:24:27 | EmilienM | even gate-tripleo-ci-centos-7-ovb-nonha ... | |
| 15:24:28 | mdbooth | Ah, the new one is actually the old one resurrected | |
| 15:24:31 | raj_singh | mdbooth: mriedem: It is the same spec. We didn't have enough time in Ocata to get it merge and not proposing for Pike | |
| 15:24:43 | EmilienM | so here's my plan. I'm going to unpin Nova in RDO, build images and retry TripleO Ci | |
| 15:24:57 | raj_singh | Hopefully we can resolve the conflict and push it forward | |
| 15:24:58 | EmilienM | maybe there is something the packaging building tool is missing | |
| 15:25:02 | EmilienM | (a dependency or something) | |
| 15:25:18 | dansmith | EmilienM: isn't that what your 429008 patch is doing? | |
| 15:25:43 | EmilienM | dansmith: what surprises me is that 429008 is green and it test master | |
| 15:25:48 | mdbooth | raj_singh: Yeah, that's cool. I still wonder if this needs a spec, though. Doesn't seem very 'architectural', just a bunch of implementation detail. That's not what mriedem is saying, though, so I suggest you don't listen to me :) | |
| 15:25:58 | EmilienM | dansmith: but not 430571 | |
| 15:26:05 | EmilienM | dansmith: and 430571 depends-on 429008 | |
| 15:26:07 | dansmith | oh right I see, I was confusing the two | |
| 15:26:08 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Consider startup scenario in _get_compute_nodes_in_db https://review.openstack.org/430994 | |
| 15:26:10 | mriedem | johnthetubaguy: this is for you (and me) ^ | |
| 15:26:22 | EmilienM | dansmith: I wanted to see more Ci jobs | |
| 15:26:25 | mdbooth | raj_singh: However, I really would prefer to see lyarwood's approach to rescue instead. It's both simpler and more functional. | |
| 15:26:37 | dansmith | EmilienM: sounds like a likely making for a setup race or something | |
| 15:26:56 | raj_singh | mdbooth: I think its good to have discussion on the spec and sort out details as this does not seems to be trivial. | |
| 15:27:11 | raj_singh | mdbooth: sure will ping him as well | |
| 15:27:14 | mdbooth | raj_singh: True. | |
| 15:27:23 | EmilienM | dansmith: ok I'm doing recheck and in 40m if I still see multinode failing, I'll unpin Nova in RDO and see how it goes | |
| 15:28:08 | raj_singh | mdbooth: siva_krishnan pinged you because you were thinking to sort out how to persist domain xml and we thought your input will be valuable. | |
| 15:28:25 | mdbooth | raj_singh siva_krishnan: I'm glad you did, thanks! | |
| 15:28:35 | cdent | EmilienM: I'm around pretty much all evening, so check in with me a things proceed. At the moment I'm pretty much bewildered and don't really understand how the pieces are fitting together ... :( | |
| 15:29:42 | mriedem | mdbooth: raj_singh: and i've -1ed https://review.openstack.org/#/c/429913/1 | |
| 15:29:58 | mriedem | i'd prefer to continue the review on the original change for ocata, | |