Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-10-21
15:00:52 clarkb seems like it should? I remember when we first set up nova live migration testing one of the things I tried was defining a custom cpu since we have heterogenous resources in the clouds we use (and have no way to request consistent cpus)
15:01:02 clarkb but that didn't work at all whcih is how we ened up using he qemu64 model
15:01:07 kashyap Yeah, I know :-(
15:01:39 clarkb I want to say at the time we identified bugs in nova and friends and they were getting fixed so it is possible this is no longer an issue today but I think that if centos is saying that you need this minimum cpu now it is reasonable for there to be that cpu predefined in qemu
15:01:46 kashyap clarkb: Sadly, QEMU does not provide a model that will work on (a) TCG, i.e. plain emulation, *and* KVM; and (b) that works on Intel and AMD
15:01:52 kashyap clarkb: But!
15:02:01 kashyap There is an option I just tested. And this works:
15:02:15 kashyap clarkb: Using "Nehalem" satisfies both the above conditions.
15:02:21 kashyap And it works on both Intel and AMD.
15:02:34 kashyap Nehalem is the oldest comatible model that works with the x86_64-v2
15:03:00 clarkb got it so there is a predefined model that addresses things. That is great news. I guess we should consider defaulting devstack to that then?
15:03:06 kashyap Yes! Indeed
15:03:17 clarkb I can try pushing a devstack change later today that does that
15:03:21 kashyap clarkb: I was going to write an email to openstack-discuss list with this recommendation
15:03:27 clarkb kashyap: that would be great, thank you
15:03:42 kashyap clarkb: I'm drafting as we speak. So your timing couldn't be perfect :-)
15:04:05 kashyap s/be perfect/be more perfect/
15:04:34 kashyap clarkb: The only assumption, which I think you'll agree is reasonable is:
15:04:57 kashyap clarkb: ... I imagine any hardware older than Nehalem is not capable of running OpenStack.
15:05:03 kashyap Say "yes", please :D
15:05:05 opendevreview Balazs Gibizer proposed openstack/nova master: Prevent leaked greenlets to interact with later tests https://review.opendev.org/c/openstack/nova/+/815017
15:05:56 clarkb kashyap: well thats the next thing we need to figure out because we get virtual resources with their own cpu models being managed. Currently we can't boot fedora-34 in half of our clouds and are wondering if this is the same issue
15:06:18 clarkb kashyap: it isn't just real hardware to consider but also virtual hardware
15:06:20 kashyap I don't know of F34 switched to the -v2, lemme check
15:07:41 kashyap clarkb: Wait, pretty sure F34 is switched to it too
15:08:01 kashyap How do I know? By inference :D RHEL9 is based on F33/F34, so I presume it did too
15:08:04 kashyap I just need to double-check
15:08:44 clarkb kashyap: ya and half our clouds can't boot it
15:09:01 clarkb so there is possibility that we can't actually switch to Nehalem, but we can make a change and test it
15:10:15 opendevreview Balazs Gibizer proposed openstack/nova master: Prevent leaked greenlets to interact with later tests https://review.opendev.org/c/openstack/nova/+/815017
15:19:08 kashyap clarkb: Yeah, I'm actually quickly pushing a test-only patch to DevStack to see where it fails. If you haven't already done so
15:19:45 clarkb kashyap: remote: https://review.opendev.org/c/openstack/devstack/+/815020 Use Nehalem CPU model by default I just pushed that to have the CI system check things. But thats because I'm in meetings and don't have a current devstack install anywhere to check with
15:20:09 kashyap Ah-ha, thank you
15:20:18 kashyap clarkb: I don't have it either right now. But:
15:20:42 kashyap clarkb: I have tested today outside of DevStack: Nehalem model works with virt_type=qemu *and* with Intel and AMD
15:21:24 clarkb kashyap: ok, ya I suspect the question becomes whether or not the CPUs we get in the host VMs are new enough to support Nehalem
15:22:01 clarkb since they too get virtual capabilities to enable live mgiration in clouds and if they were too conservative we'll have problems. The good news if there is any is that they won't be able to boot centos/rhel 9 either so clouds seem likely to update them
15:22:19 clarkb we can recheck 815020 if it generally works to get it to run across a bunch of clouds and double check that
15:24:38 kashyap clarkb: Nehalem is nearly 13 years old ... I hope there are no such hosts :
15:26:00 clarkb kashyap: the issue is the clouds provide virtual CPUs with custom models too
15:26:13 clarkb because they want to do live mgiration. And if they were too conservative you have this problem. I don't think the actual CPUs are that old
15:26:23 clarkb but the CPUs we get in our instances may be
15:27:34 kashyap Right; I see what you mean. Even if they are conservative w/ the virtual CPUs, I'd be really surprised (wouldn't be the first time) if they're more conservative than Nehalem
15:29:50 kashyap clarkb: Bad me. I was _wrong_ earlier on my stupid "inference" about Fedora: Fedora *did not* switch the baseline ABI to x86-64-v2
15:29:54 kashyap So we can rule that out
15:30:56 clarkb ok good, our problems for booting are differen then :)
15:32:25 kashyap Yeah; grr, now to find out the actual cause
15:32:31 kashyap Do you have the boot console log?
15:33:36 clarkb kashyap: not right now. It is something I can probably dig into tomorrow morning if you want to dive into it (have ptg stuff now then real world school meeting stuff after and by the time I'm done you should be enjoying your evening)
15:34:06 clarkb kashyap: also ianw is interested in the fedora 34 issue and his timezone overlap might be better? I guess it depends on how much of a morning person you are :)
15:34:24 kashyap No problem at all
15:34:52 kashyap Go handle what you need to. This can wait. And yes, will check with ianw. I'm in CEST; he's in Australia...so there should be some overlap :)
15:35:03 clarkb sounds good, thanks again!
15:56:00 gibi stephenfin, artom: another way to test is to move heat from novaclient to sdk ;)
15:56:15 artom gibi, you're an evil, evil man
15:56:30 stephenfin gibi: or tempest. That uses novaclient under the hood, I assume rather than subprocessing to the shell?
15:56:49 artom stephenfin, no, tempest reimplements all the client stuff from scratch in Python in-tree
15:56:57 artom By design
15:56:57 stephenfin oh, so it does
15:58:11 artom All I was saying is - we need to make https://opendev.org/openstack/openstacksdk/src/branch/master/.zuul.yaml#L116 work with 2 nodes do we can test how sdk does the client-side stuff for migrations and such
15:58:13 sean-k-mooney by the way when i was mentioning tempst i ment recreeate teh senario test using sdk in the sdk func test
15:58:17 gibi artom: neutron just realized that they have to do the same switch in heat
15:58:17 tosky but in general for non testing stuff gibi's proposed solution is correct (assuming the sdk implementation of orchestration is complete enough)
15:59:18 sean-k-mooney gibi: well heat need to do it rather then neutron but im sure it will be the same people in either case
15:59:50 gibi sean-k-mooney: except if neutron people want heat support for new neutron features where the only client support is in sdk :)
16:00:10 gibi anyhow starting the transition does not seem scarry https://review.opendev.org/c/openstack/heat/+/813425
16:00:22 sean-k-mooney gibi: but do they want that support :)
16:00:28 stephenfin gibi: artom: I hate to be that guy but we also need to move our internal use of cinderclient, glanceclient and neutronclient to SDK at some point :)
16:00:36 stephenfin I think ironicclient was done a few cycles ago
16:00:41 sean-k-mooney gibi: you are assuming neutron care about heat
16:01:04 sean-k-mooney they proably do but maybe not
16:01:10 stephenfin wait, no, there's still ironicclient. It's just not a mandatory import
16:01:13 stephenfin *requirements
16:01:28 artom stephenfin, yeah...
16:01:55 artom Like, I'm taking part in this debat, but who am I kidding, I won't be the one doing the work - I tried to start, then just ran out of steam
16:02:29 gibi stephenfin: ouch, you are right :)
16:02:47 artom OTOH, what's kind of annoying is that the problem that I'm fixing in https://review.opendev.org/c/openstack/openstacksdk/+/741688 is still a problem.
16:02:57 artom image is *still* optional, when it isn't in the PAI
16:02:59 artom *API
16:03:50 artom And it's *still* missing the kwargs things for all the other params that are in the API
16:04:02 stephenfin artom: I should have looked at that. My bad :( If you got time to rebase it onto the feature/r1 branch (which will be merged into master soon enough) I'll review it in the AM
16:04:19 artom stephenfin, I need to fix the unit test below it and rebase
16:04:31 artom stephenfin, it's just such a slog, and I'm lazy and easily distracted
16:04:42 stephenfin you should try cocaine
16:04:50 artom Speaking from experience?
16:04:58 stephenfin all the bankers I know swear by it
16:05:29 artom Explains the state of the financial system
16:07:53 gibi happy hours already?
16:31:24 sean-k-mooney am are we goint to call it a day or do we want to do the pain points discussion
16:31:29 sean-k-mooney since we got time back
16:31:54 sean-k-mooney bauzas: ^
16:31:56 bauzas sean-k-mooney: I prefer to leave early
16:32:18 sean-k-mooney ok
16:32:38 bauzas sean-k-mooney: I feel we can make all the agenda by tomorrow
16:33:04 bauzas we have 7 topics left
16:33:11 sean-k-mooney well we might but there is also tc sesssions tomorow
16:33:25 sean-k-mooney so we might not have quorm for the full day
16:33:39 sean-k-mooney but yes we likely can finish tomorow
16:34:11 bauzas sean-k-mooney: you're right, we're constrainted by the big TC RBAC thing
16:34:43 bauzas sean-k-mooney: but I feel we can postpone a few topics if we really need

Earlier   Later