| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-01-06 | |||
| 10:01:27 | sean-k-mooney | but we dont test that | |
| 10:01:40 | alexe9191 | ok, and rolling upgrade being moving everything at once from one release to the other, both the APi & the nova-compute agents, K, L, N, ... | |
| 10:01:46 | sean-k-mooney | there is typically 5-6 release betwwen major rpc version bumps | |
| 10:02:32 | alexe9191 | That's very helpful to know! | |
| 10:02:50 | sean-k-mooney | alexe9191: rolling upgrades are where you move the contolplane in one go to n+1, then move the computes in a "rolling" or incremental fashion 1 by 1 | |
| 10:03:31 | sean-k-mooney | rolling upgrades allow you to maintain managmeablity of the computes with mixed verions | |
| 10:03:41 | sean-k-mooney | but we only test a delta of 1 verions | |
| 10:04:11 | sean-k-mooney | we write the code such that it could be a larger delta but the test matix is too large for use to offialy support that | |
| 10:04:12 | alexe9191 | Right, so if upgrading K to L. I could basically roll the upgrade to the compute nodes one rack at a time from K to L | |
| 10:04:28 | sean-k-mooney | yes | |
| 10:04:47 | alexe9191 | So in theory one could do N+2 but that's not tested. | |
| 10:05:28 | sean-k-mooney | yes in princiapl n->n+2 should work provided there is not an rpc major version bump in between | |
| 10:05:47 | sean-k-mooney | but really you would be the first to test that | |
| 10:06:10 | alexe9191 | I'd rather stick to what you guys are doing upstream in that case. | |
| 10:06:13 | sean-k-mooney | we have been on RPC version 5.X for about 5 releases now | |
| 10:06:44 | sean-k-mooney | alexe9191: ya if this is a prod deployment that is likely your safest option | |
| 10:06:49 | alexe9191 | I am guessing there are major changes between K and R in RPC versions, is that documented in the release note? | |
| 10:07:05 | alexe9191 | Yes, which sounds that doing the rolling-upgrades is the safest supported solution. | |
| 10:08:08 | sean-k-mooney | i think it would be in the release notes but i can quickly check the code | |
| 10:08:38 | alexe9191 | Fantastic, if I could also know which file you are checking in the source tree that would be great as well. | |
| 10:09:07 | sean-k-mooney | we have each verion bump documented with a comment https://github.com/openstack/nova/blame/master/nova/compute/rpcapi.py#L76 | |
| 10:09:37 | sean-k-mooney | https://github.com/openstack/nova/blame/master/nova/compute/rpcapi.py#L385 | |
| 10:09:42 | alexe9191 | Awesome! | |
| 10:09:45 | sean-k-mooney | the aliase are the quickest way to check | |
| 10:10:23 | alexe9191 | Interesting, so K to P is on the same Major RPC release. | |
| 10:10:32 | sean-k-mooney | so kilo was 4.0 | |
| 10:10:39 | sean-k-mooney | rocky was 5.0 | |
| 10:10:57 | sean-k-mooney | technically 5.0 is the ame as 4.22 | |
| 10:11:46 | sean-k-mooney | so if rocky still has the 4.22 compat code then you could pin the rpc version to 4.0 in principal | |
| 10:11:56 | sean-k-mooney | but it proably will have bugs | |
| 10:12:36 | alexe9191 | This is really helpful! | |
| 10:13:35 | sean-k-mooney | honestly i doubt we actully manged to keep perfect rpc compate and i suspect rocky has had the 4.x compat code removed | |
| 10:14:31 | sean-k-mooney | but looking quickly it looks like the compat code was maybe removed in stine | |
| 10:14:58 | sean-k-mooney | without testign it first however i would not bet on it it fully working | |
| 10:15:07 | alexe9191 | If I am not mistaken, this could mean that in theory I could run Newton API on kilo nova compute. Since the RPC API is on the same major version. | |
| 10:15:39 | sean-k-mooney | in theory yes | |
| 10:16:02 | sean-k-mooney | in practice you would likely be the first to try that and you would have to let us know if it works :) | |
| 10:16:27 | alexe9191 | I will most definitely do that :) | |
| 10:16:33 | sean-k-mooney | you would pin the RPC version of the newton contolplane to 4.0 | |
| 10:16:49 | alexe9191 | This should be done in nova.conf I could imagine ? | |
| 10:16:50 | sean-k-mooney | you can do that via the upgrade levels in the nova.conf | |
| 10:19:02 | alexe9191 | I am gonna let you guys know once I test that! Thanks a lot for all of the helpful information. | |
| 10:19:44 | bauzas | sean-k-mooney: alexe9191: well, Kilo is very prehistoric | |
| 10:20:09 | bauzas | and from the RPC PoV, we only started to introduce massively nova objects by this time | |
| 10:20:15 | sean-k-mooney | it is but so is newton :P | |
| 10:20:33 | bauzas | at newton, we mostly stopped to use dicts on RPC calls | |
| 10:20:48 | bauzas | while during kilo, most of the values were those | |
| 10:20:50 | sean-k-mooney | mostly but we still have a few even on master | |
| 10:21:21 | bauzas | speaking of RPC compat, I'd say a nova boot *could* work | |
| 10:21:22 | sean-k-mooney | bauzas: having random dicts of stings migth actully help in this case | |
| 10:21:33 | sean-k-mooney | provide the required data is there | |
| 10:21:51 | bauzas | maybe | |
| 10:22:48 | bauzas | anyway, all of this is very experimental | |
| 10:23:25 | bauzas | but I sincerely hope alexe9191 is not expecting live-migration to work with this env :) | |
| 10:23:26 | sean-k-mooney | oh yes it is. | |
| 10:26:15 | alexe9191 | bauzas no and this is not a production environment where I will be testing. Our of curiosity i am going to run an experiment but for production I am probably gonna do the rolling upgrade. | |
| 10:28:31 | bauzas | alexe9191: anyway, I'm very curious about your experiment results :) | |
| 10:29:11 | bauzas | I'm personnally not opposed to have operators breaking the N+1 rule by running very old computes if they wish | |
| 10:29:44 | bauzas | provided they understand the maintenance price | |
| 10:31:15 | bauzas | but... Kilo/Rocky is like a gymnast front oversplit to me :) | |
| 10:31:40 | bauzas | front side* even | |
| 10:31:42 | sean-k-mooney | one of the cost is really you can only safely use the max api version of the oldest compute | |
| 10:31:50 | sean-k-mooney | as in microversion | |
| 10:32:08 | sean-k-mooney | you might be able to use later microverions but if they needed an rpc change it wont work | |
| 10:32:28 | sean-k-mooney | we will backlevel the request but you wont get teh behavior the new microverion implies | |
| 10:33:10 | gibi | stephenfin: I finished reviewing the migration compacting series, nice work! | |
| 10:33:18 | stephenfin | gibi: Thanks :) | |
| 10:33:42 | gibi | I have couple of questions in the middle but overall I think we are good | |
| 10:33:44 | stephenfin | If you want me to review anything, just let me know. I'm almost done on the os-hypervisors fixup (many rabbit holes) | |
| 10:33:58 | stephenfin | Sweet. Will take a look shortly | |
| 10:35:09 | sean-k-mooney | bauzas: by the way i have not got around to doing the full code review for the cyborg shevle patches but did my respoce make sense | |
| 10:35:20 | sean-k-mooney | regarding the conductor chagnes | |
| 10:37:24 | bauzas | sean-k-mooney: I'll look at them later today, I was on a meeting just before and I want to rebase my own routed networks series | |
| 10:37:37 | sean-k-mooney | no worries | |
| 10:37:38 | bauzas | in other words, I lag :D | |
| 10:38:46 | gibi | stephenfin: there is a trivial review https://review.opendev.org/c/openstack/nova/+/769467 you can lok at | |
| 10:39:06 | sean-k-mooney | hehe well i was indening to do a fully review yesterady and havent started other then to respond ot your comments so your still ahead of me | |
| 10:39:15 | stephenfin | gibi: have it open already :) | |
| 10:40:34 | gibi | besides that I have nothing :) | |
| 10:40:45 | gibi | I need to do a bunch of rebases | |
| 10:49:14 | sean-k-mooney | speaking of rebases i wil be rebasing my vdpa spec in a day or so but if people have time it would be nice if ye could review that spec https://review.opendev.org/c/openstack/nova-specs/+/764999 and also the numa port polices spec https://review.opendev.org/c/openstack/nova-specs/+/765901 | |
| 10:50:03 | sean-k-mooney | i intend to start pushing some standalone patches for those this week namely the xml generation for vdpa and some constants for the newtuon extensions | |
| 10:51:25 | sean-k-mooney | the larger change will still be a few weeks out but i would like to start submitting an merging some of the smaller patches which would be dead code but easilly reviewable while i work on the more complex part but the spec need to be approved first before that can happen | |
| 11:07:50 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Create a fixture around fake_notifier https://review.opendev.org/c/openstack/nova/+/758446 | |
| 11:07:50 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Use the non polling notification waiter in func test https://review.opendev.org/c/openstack/nova/+/758445 | |
| 11:07:51 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Use NotificationFixture for legacy notifications too https://review.opendev.org/c/openstack/nova/+/758448 | |
| 11:09:12 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Test the NotificationFixture https://review.opendev.org/c/openstack/nova/+/758450 | |
| 11:09:13 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Move fake_notifier impl under NotificationFixture https://review.opendev.org/c/openstack/nova/+/758451 | |
| 11:16:16 | openstackgerrit | Stephen Finucane proposed openstack/nova master: api: Drop statistics-style fields from os-hypervisors https://review.opendev.org/c/openstack/nova/+/764040 | |
| 11:16:17 | openstackgerrit | Stephen Finucane proposed openstack/nova master: api: Normalize exception handling for os-hypervisors https://review.opendev.org/c/openstack/nova/+/769520 | |
| 11:16:17 | openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Remove 'test_extended_hypervisors' https://review.opendev.org/c/openstack/nova/+/769519 | |
| 11:17:13 | stephenfin | gibi: Done ^ Sorry for the delay | |
| 11:17:22 | gibi | on it | |
| 11:29:10 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: WIP: Add a routed networks scheduler pre-filter https://review.opendev.org/c/openstack/nova/+/749068 | |
| 11:29:10 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: Add requested_networks field to RequestSpec object https://review.opendev.org/c/openstack/nova/+/749977 | |
| 11:43:00 | sean-k-mooney | does anywone know if shareing resouce providers actully work by the way | |
| 11:43:36 | sean-k-mooney | its in relateion to http://lists.openstack.org/pipermail/openstack-discuss/2021-January/019647.html | |
| 11:43:48 | sean-k-mooney | specifically for the RBD image backend | |
| 11:44:28 | sean-k-mooney | if we used provider.yaml to set the local disk_gb inventory to 0 and created a shareing resocue provider with a disk_gb inventory for the ceph pool | |
| 11:44:55 | lyarwood | I don't think anyone was working on that | |