| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-02-03 | |||
| 15:02:30 | dansmith | efried: knowing that latest version of that matches your expectation and what should be done is important | |
| 15:02:33 | dansmith | thanks | |
| 15:03:21 | dansmith | sean-k-mooney: last I heard you had deployed the cyborg set in devstack and had identified some real issue, but I don't know what that was, and it's been a while.. have you done it recently? | |
| 15:03:39 | sean-k-mooney | efried: it is alredy on pypi also its weired i have blockdiag in my local requiremetns but i dont see it on master. ill talk to the release folks and make sure we are good | |
| 15:03:59 | sean-k-mooney | dansmith: no but i can do it again tomorrow | |
| 15:04:06 | ignaziocassano1 | lyarwood: I am sure the device is not used by multiple instances. I tried substituing the command "multipath -f" with "mutipath -r" in /usr/lib/python2.7/site-packages/os_brick/initiator/linuxscsi.py and it works . But flush and reload are different operations :-( | |
| 15:04:22 | sean-k-mooney | dansmith: the issue was the patches needed to be rebased for root required and cyboge had a python 3 issue | |
| 15:04:34 | dansmith | sean-k-mooney: ah okay, good to know, thanks | |
| 15:04:38 | sean-k-mooney | dansmith: sundar has rebased the cybog patch chain so it should work | |
| 15:04:48 | dansmith | gotcha, I remember now | |
| 15:04:57 | openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Remove custom comparison methods https://review.opendev.org/472285 | |
| 15:05:49 | sean-k-mooney | ill kick off devstack stacking in the backgound and if it stack ill let you know what i find. | |
| 15:05:58 | dansmith | thanks | |
| 15:06:08 | ignaziocassano1 | lyarwood: keep in minf all volumes I attach have lvm inside them. Is it possible lvm on kvm node lock them ? | |
| 15:06:50 | Sundar | sean-k-mooney: Yes. I have also updated the Nova patch series, with all expected functionality, including the RPCAPI version bump to pass ARQ UUIDs from the conductor to manager. | |
| 15:06:52 | sean-k-mooney | dansmith: you have ipv6 connectivity right? when its stacked i can proably just add your public key to the vm if you want to take alook later | |
| 15:07:07 | sean-k-mooney | Sundar: cool | |
| 15:07:07 | dansmith | I do | |
| 15:09:07 | sean-k-mooney | cool i can always port forward but i have my router handing out public ipv6 adress to my openstack vms by default so if i tweek the security group rule it shoudl work | |
| 15:13:53 | lyarwood | ignaziocassano1: if you're host is also using LVM and has a default filter then yes it could be picking them all up | |
| 15:14:00 | lyarwood | your* | |
| 15:16:15 | ignaziocassano1 | lyarwood: I added the filter filter = [ "a|^/dev/sda|", "r|.*|" ] because I use lvm on kvm node only on /dev/sda but it does not resolve | |
| 15:17:50 | ignaziocassano1 | lyarwood: has the above filter a correct syntax ? | |
| 15:21:47 | lyarwood | ignaziocassano1: sorry I don't have the bandwidth to debug this with you at the moment | |
| 15:22:54 | ignaziocassano1 | lyarwood: I wrote all information on the bug I opened. I hope someone can help. | |
| 15:34:24 | sean-k-mooney | dansmith: Sundar efried just as an fyi this is the local.conf im using to deploy the cyborg env. the only thing you need to do manually is install the correct version of the cyborg client | |
| 15:34:31 | sean-k-mooney | http://paste.openstack.org/show/789067/ | |
| 15:34:44 | artom | stephenfin, reply about the 3/2 split here: https://review.opendev.org/#/c/672595/64 | |
| 15:34:55 | artom | Let's get this sorted so we can both move on ;) | |
| 15:35:36 | dansmith | sean-k-mooney: why is that? shouldn't it come in via reqs? | |
| 15:36:10 | sean-k-mooney | the cyborg plugin does not install it | |
| 15:36:21 | sean-k-mooney | devstack plugin that is | |
| 15:36:31 | sean-k-mooney | its not needed for nova its just for osc | |
| 15:36:41 | sean-k-mooney | so i dont have to use curl | |
| 15:37:01 | dansmith | oh you mean you have to install the cyborg osc plugin? | |
| 15:37:08 | sean-k-mooney | ya | |
| 15:37:16 | dansmith | okay gotcha | |
| 15:37:31 | stephenfin | artom: Could just do a negative test? | |
| 15:37:41 | stephenfin | actually, nvm | |
| 15:37:49 | sean-k-mooney | thats why im also checking out the openstack sdk from a review branch | |
| 15:37:57 | sean-k-mooney | that isnt neededed by nova | |
| 15:38:09 | sean-k-mooney | its just for osc | |
| 15:38:18 | stephenfin | artom: I'm dumb and don't see the problem you're pointing out with that comment. Can you restate it for me? :) | |
| 15:38:45 | artom | stephenfin, image the ascii diagram on the first "line", but with VM1 on N2 | |
| 15:38:58 | mmidolesov | Does anayone know why functional-py36 returns TIMED_OUT in nova when rechecking patch? This happens for quite some time now. | |
| 15:39:30 | artom | If we then live migrate it, and we end up with the ascii diagram on the second "line", we have no way of knowing whether that's because NUMA live migration worked and updated the XML, or just because that's the old behaviour anyways | |
| 15:39:42 | sean-k-mooney | mmidolesov: in the gate? | |
| 15:39:59 | sean-k-mooney | i have not seen that on any patches i have reviewd lately | |
| 15:40:13 | stephenfin | Oh, so you're saying if it ended up landing on node 2, we wouldn't really be testing anything? Fair point. Could you just force the instances onto separate hosts though? | |
| 15:40:22 | stephenfin | Using the 'host' parameter from...2.53? | |
| 15:40:32 | sean-k-mooney | mmidolesov: if you are seing it on patche you pushed it might mean the mocking is not done correctly | |
| 15:41:07 | sean-k-mooney | stephenfin: the anti affintiy filer might be better | |
| 15:41:18 | stephenfin | sean-k-mooney: any reason? | |
| 15:41:28 | sean-k-mooney | im not sure tempest suppofrt the host parmater currently | |
| 15:41:32 | artom | stephenfin, we can force on separate hosts, but... can we force on specific NUMA nodes? Or, can we count that they'll be assigned to instances deterministically? | |
| 15:41:40 | sean-k-mooney | and we dont know what the host would be called generically | |
| 15:41:58 | sean-k-mooney | artom: no we cant force to a specif numa node | |
| 15:42:21 | artom | stephenfin, IOW, I want a situation where we *know for sure* that without the NUMA live migration feature, we'd have ended up with overlapping instances on the destination | |
| 15:42:41 | sean-k-mooney | we should not rely on the ordering | |
| 15:42:53 | sean-k-mooney | it tecnically is determisitic but only if we run serially | |
| 15:42:56 | artom | So 1. is that a fair thing to want? 2. can we achieve it with a less artificial NUMA topology | |
| 15:43:03 | sean-k-mooney | so we shoudl froce it with the pinsets | |
| 15:43:23 | sean-k-mooney | artom: no we do not want to exposee the ability to select a host numa node | |
| 15:43:41 | stephenfin | sean-k-mooney: we can't do different pinsets on different hosts | |
| 15:43:43 | artom | sean-k-mooney, except we can't have different pinsets per "host" in func tests | |
| 15:43:43 | sean-k-mooney | telcos want that but that is too invaisive to be a thing in the flavor | |
| 15:43:47 | stephenfin | not easily | |
| 15:44:03 | sean-k-mooney | stephenfin: in tempest we can are you talking about func tests? | |
| 15:44:08 | sean-k-mooney | i was assuming whitebox | |
| 15:44:22 | stephenfin | nah, artom's giant live migration functional test | |
| 15:44:30 | dansmith | Sundar: this can be abandoned, right? it still looks as if it's in the stack, despite being very outdated: https://review.opendev.org/#/c/673939/1 | |
| 15:45:10 | Sundar | sean-k-mooney: The local.conf seems fine to me. You are pulling in one more patch than what you need. | |
| 15:45:14 | Sundar | dansmith: Yes | |
| 15:45:30 | sean-k-mooney | Sundar: the uuid one | |
| 15:45:37 | sean-k-mooney | ya i know that log was annoying me | |
| 15:45:40 | Sundar | dansmith: I'll ping the developer | |
| 15:45:45 | dansmith | Sundar: I can do it | |
| 15:46:17 | Sundar | sean-k-mooney: The more egregious one about the bad UUID is already fixed: https://review.opendev.org/#/c/702785/ | |
| 15:46:24 | Sundar | dansmith: OK, thanks | |
| 15:46:39 | dansmith | Sundar: what about the two WIP ones underneath? | |
| 15:46:47 | dansmith | I think those are all effectively orphaned | |
| 15:47:30 | stephenfin | artom: create three instances | |
| 15:47:35 | Sundar | dansmith: Yes. The to-be-abandoned patch is based on very old versions of my patches. | |
| 15:47:49 | dansmith | Sundar: ack, will get those too | |
| 15:48:04 | dansmith | oh nm, I see, | |
| 15:48:13 | dansmith | those are when the current patches were marked as WIP, nevermind | |
| 15:48:26 | artom | stephenfin, go on... | |
| 15:48:28 | Sundar | But only 73939 nees to be abandoned | |
| 15:48:29 | stephenfin | artom: if you've two nodes on N1 and one on N2, move one of the former | |
| 15:48:47 | Sundar | dansmith: Others are part of the Nova patch series | |
| 15:48:53 | stephenfin | *two instances on N1 | |
| 15:49:04 | Sundar | dansmith: I meant 673939 | |
| 15:49:15 | stephenfin | and if you've one instance on N1 and two on N2, move one of the latter | |
| 15:49:20 | dansmith | Sundar: I abandoned two and it looks cleaner nwo | |
| 15:49:24 | stephenfin | guaranteed XML regeneration | |
| 15:49:24 | artom | stephenfin, that... seems like it's equally complex :/ | |
| 15:49:51 | artom | As in, we need to figure out the instance on the host with 2 of them that's pinned to the "equivalent" NUMA node as the other instance on the host with 1 instance | |
| 15:50:14 | Sundar | dansmith: https://review.opendev.org/#/c/631244/26 is part of the current patch series | |
| 15:50:35 | dansmith | Sundar: yes I see now, which is why I said this: [07:48:13] <dansmith>those are when the current patches were marked as WIP, nevermind | |
| 15:51:18 | artom | stephenfin, my "counter-offer" would be keep things as is, but with a nice ASCII diagram explaining the thing | |