| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-11-21 | |||
| 13:16:54 | bauzas | efried: sure, it's an interop issue | |
| 13:16:55 | bauzas | but | |
| 13:17:09 | johnthetubaguy | efried: I am more thinking, the SDK and CLI should be able to know when its missing, like with all our other APIs | |
| 13:17:16 | bauzas | if you use nova-network since 2.58, you are a bit having problems | |
| 13:17:32 | johnthetubaguy | to be clear, I think we should remove it, and it should return 210 | |
| 13:17:32 | efried | Either way, I think that ship has sailed, cause we've already ripped a bunch of stuff out. Pretty sure the point of no return was in train, too. | |
| 13:17:37 | bauzas | also, if you use OSC, you won't see the APIs be deleted | |
| 13:17:54 | johnthetubaguy | I just think we should have a microversion that tells a user, if that is here, you know those APIs are gone | |
| 13:18:12 | johnthetubaguy | we can add it as the last patch in the series, and I think that kinda works, its just a handy hit | |
| 13:18:34 | johnthetubaguy | it makes the docs easier though, if microversion X is available, you know this API will always return HTTP gone | |
| 13:19:05 | zigo | bauzas: Hi there! I'm trying to start an instance with a GPU, and I get this: | |
| 13:19:05 | zigo | $ openstack server create --image bionic-server-cloudimg-amd64_20190726_GPU --nic net-id=bdb-blue-int01 --key-name yubikey-zigo --flavor cpu4-ram12-disk20-gpu-nvidia-p1000 --availability-zone=AZ2 zigo-gpu | |
| 13:19:05 | zigo | PCI alias nvidia-p1000 is not defined (HTTP 400) (Request-ID: req-b0514752-9e50-4e9b-a085-6ef9169b1d59) | |
| 13:19:23 | zigo | bauzas: Though I do have this in nova.conf: | |
| 13:19:24 | zigo | alias={"vendor_id":"10de","product_id":"1cb1","name":"nvidia-p1000","device_type":"type-PCI"} | |
| 13:19:27 | zigo | What am I missing? | |
| 13:19:27 | openstackgerrit | Elod Illes proposed openstack/nova stable/pike: Explicitly fail if trying to attach SR-IOV port https://review.opendev.org/695408 | |
| 13:19:54 | bauzas | johnthetubaguy: well, we could indeed | |
| 13:20:17 | bauzas | once all the APIs are giving HTTP410 | |
| 13:20:22 | johnthetubaguy | bauzas: it is a nice to have, really its the docs that worried me | |
| 13:20:22 | bauzas | stephenfin: ^ | |
| 13:20:44 | johnthetubaguy | zigo: is that in the API and Compute nova.conf? | |
| 13:20:52 | zigo | johnthetubaguy: Yeah... | |
| 13:21:12 | zigo | johnthetubaguy: It's ok to have multiple times: | |
| 13:21:12 | zigo | [pci] | |
| 13:21:12 | zigo | alias=<something> | |
| 13:21:13 | zigo | right? | |
| 13:21:18 | johnthetubaguy | hmm, I think that is how I screwed it once | |
| 13:21:23 | zigo | Cause I have 2 different boards in this cloud ... | |
| 13:21:41 | zigo | johnthetubaguy: You mean it should be only in the compute? | |
| 13:22:06 | johnthetubaguy | no, sorry, I was meaning it needs to be everyone, almost | |
| 13:22:15 | gmann | johnthetubaguy: bauzas efried : and that microversion is just to notify the users that nova-net APIs (including url) are gone and not maintaining those API for older microversion right ? | |
| 13:22:28 | johnthetubaguy | gmann: +1 | |
| 13:22:38 | bauzas | gmann: yup, just a signal | |
| 13:22:44 | gmann | ok. that make sense. | |
| 13:23:08 | gibi | works for me | |
| 13:23:14 | bauzas | zigo: not sure I understand | |
| 13:23:28 | bauzas | zigo: you need to set the alias value *once* | |
| 13:24:01 | zigo | bauzas: Yeah, but I have one Nvidia p1000 and one t4 (on 2 different compute nodes). | |
| 13:24:27 | efried | johnthetubaguy: I think the patch to make those 410s already flew... | |
| 13:24:46 | efried | johnthetubaguy: okay, patches plural. And yes, they merged in train. | |
| 13:24:47 | zigo | Therefore, I have: | |
| 13:24:47 | zigo | [pci] | |
| 13:24:47 | zigo | alias={"vendor_id":"10de","product_id":"1cb1","name":"nvidia-p1000","device_type":"type-PCI"} | |
| 13:24:47 | zigo | alias={"vendor_id":"10de","product_id":"1eb8","name":"nvidia-t4","device_type":"type-PCI"} | |
| 13:24:50 | zigo | Is this correct? | |
| 13:25:06 | efried | johnthetubaguy: so really we could do that microversion any time? | |
| 13:25:15 | efried | and the rest is just cleanup? | |
| 13:25:40 | johnthetubaguy | efried: yes, expect for the docs needing to be updated to reflect how you find out when its gone | |
| 13:26:27 | efried | okay. Pretty sure the docs were the first thing stephenfin hit -- but yeah, they wouldn't mention a microversion cutover if we didn't do that. | |
| 13:26:46 | johnthetubaguy | efried: we can go back an add that for sure | |
| 13:26:47 | efried | So you're looking for a patch that creates a new "signal microversion" and updates the docs accordingly. | |
| 13:26:56 | efried | that makes sense ++ | |
| 13:27:07 | johnthetubaguy | yeah, thanks, that clears it up in my head too | |
| 13:27:08 | johnthetubaguy | phew | |
| 13:27:37 | efried | you can still have the interop snafu I mentioned earlier. But at least in that scenario your "signal" is the 410. | |
| 13:28:32 | sean-k-mooney | zigo: the nviad-t4 apparently supprot sriov so you have to set teh device_type to type-PF | |
| 13:28:40 | johnthetubaguy | efried: yeah, its that you could have known better | |
| 13:28:53 | sean-k-mooney | stephenfin: ^ that is the thin you were writing the docs patch for right | |
| 13:29:07 | sean-k-mooney | https://review.opendev.org/#/c/694522/ | |
| 13:29:09 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Specify what RPs _ensure_resource_provider collects https://review.opendev.org/695429 | |
| 13:29:12 | zigo | sean-k-mooney: The issue is with my p1000, the t4 looks like working ... | |
| 13:29:18 | zigo | But thanks, I'll try. | |
| 13:29:32 | gibi | efried: fixed up the comment in _ensure_resource_provider_collects https://review.opendev.org/695429 | |
| 13:29:46 | sean-k-mooney | zigo: what is the issue you are having specificly | |
| 13:30:00 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Force config drive in nova-next multinode job https://review.opendev.org/695431 | |
| 13:30:00 | zigo | sean-k-mooney: PCI alias nvidia-p1000 is not defined (HTTP 400) | |
| 13:30:07 | zigo | (when trying to spawn my instance) | |
| 13:30:21 | gmann | efried: but from nova ussuri onwards, any microversion will be no-nova-net-api for all cloud so interop things needs to be taken care by discoverability of new microversion we will introduce instead of code handle that. | |
| 13:30:27 | sean-k-mooney | have you set the alias on both the compute nodes and the controler nodes | |
| 13:30:35 | zigo | sean-k-mooney: Yeah, I did that ... | |
| 13:30:47 | zigo | I did with puppet, so normally, it will have restart all nova services. | |
| 13:31:41 | sean-k-mooney | and you have it set in the [pci] section not default | |
| 13:32:06 | gmann | efried: RE: provider as 'openstack:' these are log it return from 60_nova/resources.sh:_get_inventory_value for VCPU etc: https://zuul.opendev.org/t/openstack/build/323afb9d5fd94f62b0bba4bac6004442/log/logs/grenade.sh.txt.gz#28851 | |
| 13:32:36 | gmann | and that is heppning when octavia grenade job is moving to py3. i ma not sure how it has to do with py3 htings. | |
| 13:32:38 | gmann | thins | |
| 13:32:54 | zigo | sean-k-mooney: That's all I did yes... | |
| 13:32:54 | gmann | https://review.opendev.org/#/c/693486/ | |
| 13:33:49 | gmann | mriedem: have you seen this before in any grenade job - https://zuul.opendev.org/t/openstack/build/323afb9d5fd94f62b0bba4bac6004442/log/logs/grenade.sh.txt.gz#28854 | |
| 13:33:59 | zigo | Oh, another thing which is very annoying, I keep having in my logs: | |
| 13:33:59 | zigo | Instance 2aab6469-4292-4d04-80de-2ae2a7174b3a has been moved to another host clint1-compute-2.infomaniak.ch(clint1-compute-2.infomaniak.ch). There are allocations remaining against the source host that might need to be removed: {'resources': {'DISK_GB': 80, 'MEMORY_MB': 24576, 'VCPU': 8}}. | |
| 13:34:05 | zigo | Many of this ... | |
| 13:34:29 | zigo | Is this a known issue with Rocky? | |
| 13:34:33 | sean-k-mooney | that usually means you have not compuleted migrations using resize-verify | |
| 13:34:45 | zigo | sean-k-mooney: It's mostly all live migrations. | |
| 13:34:58 | efried | gmann: those CI results are still not loading up for me :( | |
| 13:35:13 | gmann | oh | |
| 13:35:24 | zigo | sean-k-mooney: IMO, the only thing that remains is the placement record... | |
| 13:35:29 | efried | gmann: if you have them open, maybe you could pastebin the relevant chunk? | |
| 13:35:34 | zigo | I could easily write a clean-up script I suppose. | |
| 13:35:34 | sean-k-mooney | efried: https://zuul.opendev.org/t/openstack/build/323afb9d5fd94f62b0bba4bac6004442/log/logs/grenade.sh.txt.gz#28854 loaded for me fine | |
| 13:36:16 | sean-k-mooney | zigo: it should be updated automatically | |
| 13:36:37 | efried | gmann: ah, it's working now. | |
| 13:36:53 | mriedem | gmann: the broken pipe? no | |
| 13:38:34 | gmann | it return 'openstack:' as provider name from 60_nova/resources.sh:_get_inventory_value | |
| 13:39:18 | mriedem | there is a broken pipe right before that | |
| 13:39:34 | mriedem | 2019-11-20 17:52:23.572 | +++ /opt/stack/new/grenade/projects/60_nova/resources.sh:_get_inventory_value:57 : head -n1 2019-11-20 17:52:23.573 | +++ /opt/stack/new/grenade/projects/60_nova/resources.sh:_get_inventory_value:57 : openstack resource provider list -f value 2019-11-20 17:52:23.573 | +++ /opt/stack/new/grenade/projects/60_nova/resources.sh:_get_inventory_value:57 : cut -d ' ' -f 1 2019-11-20 17:52:24.613 | Exc | |
| 13:39:34 | mriedem | on raised: [Errno 32] Broken pipe | |
| 13:39:40 | mriedem | so parsing the output is failing | |
| 13:40:37 | mriedem | openstack resource provider list -f value | |