| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-11-22 | |||
| 16:53:25 | openstackgerrit | Matt Riedemann proposed openstack/nova master: nova-net: Remove 'nova-network' binary https://review.opendev.org/686816 | |
| 16:53:26 | melwitt | badum csh! | |
| 16:53:34 | openstackgerrit | Matt Riedemann proposed openstack/nova master: docs: Blast most references to nova-network https://review.opendev.org/686817 | |
| 16:53:47 | melwitt | man, I really can't type anything today | |
| 16:54:10 | mriedem | have a kit kat | |
| 16:54:39 | melwitt | hm, I could. still have some halloween candy | |
| 16:55:34 | efried | stephenfin: did you see the conversation yesterday with johnthetubaguy about adding a "signal microversion" so clients can discover that nova-net is gone? | |
| 16:55:58 | stephenfin | efried: I did. mriedem is probably the person you want to discuss that with, tbh | |
| 16:56:23 | mriedem | -1 | |
| 16:56:32 | efried | oh? | |
| 16:56:50 | mriedem | oh. | |
| 16:56:51 | mriedem | yeah. | |
| 16:57:10 | stephenfin | I realise why he suggested it but I'm not sure about the idea of using a microversion when we can't actually provide the previous behavior on old microversions | |
| 16:57:15 | mriedem | the deprecation microversions were the signals | |
| 16:57:30 | stephenfin | we didn't do it for things like os-cells, os-fping, etc. | |
| 16:57:36 | mriedem | if you want to add a microversion for each api we've removed now, you're talking about all of these https://docs.openstack.org/api-ref/compute/#obsolete-apis | |
| 16:57:37 | mriedem | so -1 | |
| 16:58:13 | efried | johnthetubaguy: ^ | |
| 16:58:14 | efried | you have five minutes to respond | |
| 16:58:23 | mriedem | i mean, i'm sure we can deal with someone that shows up and is like, "WTF HAPPENED TO CLOUDPIPE?!" | |
| 16:59:39 | melwitt | the good old days | |
| 17:00:06 | mriedem | https://docs.openstack.org/nova/latest/contributor/api.html#removing-deprecated-apis | |
| 17:00:16 | mriedem | i think i wrote that after we did the first 410s | |
| 17:01:20 | efried | dig | |
| 17:02:05 | openstackgerrit | Merged openstack/nova stable/train: Block deleting compute services with in-progress migrations https://review.opendev.org/694546 | |
| 17:02:11 | openstackgerrit | Merged openstack/nova stable/train: Improve metadata server performance with large security groups https://review.opendev.org/694409 | |
| 17:05:28 | efried | mriedem: I got confused where it was suggested that GET /os-hypervisors*?service_host=CONF.host would ever return more than one entry. | |
| 17:05:40 | efried | It would still return a list, but it would have (at most) one entry, right? | |
| 17:05:46 | johnthetubaguy | mriedem: OK, I buy that. If you see the deprecation micro version, expect to be disappointed with HTTP Gone | |
| 17:05:48 | mriedem | WRONG | |
| 17:05:48 | efried | even for ironic | |
| 17:06:06 | mriedem | efried: compute service : compute node (hypervisor) is 1:M with ironic | |
| 17:06:21 | mriedem | please to be enjoying https://bugs.launchpad.net/nova/+bug/1852446 | |
| 17:06:21 | openstack | Launchpad bug 1852446 in OpenStack Compute (nova) "Hypervisors in nova - no subpage details for ironic" [Undecided,New] | |
| 17:06:21 | gregwork | is it expected that when an instance is booted into a non nova az, that volumes created by cinder in the nova AZ (successfully as part of the stack create) will fail to attach to the instance? | |
| 17:06:28 | gregwork | also cinder doesnt seem to know about the az's i created in nova | |
| 17:06:42 | gregwork | trying to pass availability_zone: mynovaAZ returns an unknown az error | |
| 17:07:05 | mriedem | gregwork: see cross_az_attach in https://docs.openstack.org/nova/latest/admin/availability-zones.html | |
| 17:07:15 | efried | mriedem: ack | |
| 17:07:37 | gregwork | oh interesting | |
| 17:08:06 | mriedem | gregwork: also https://www.openstack.org/videos/summits/vancouver-2018/curse-your-bones-availability-zones-1 | |
| 17:08:27 | mriedem | before you go too deep down the AZ rabbit hole you might want to watch that | |
| 17:08:33 | gregwork | i was at that summit, i wish i attended that session | |
| 17:08:39 | gregwork | ill check it out | |
| 17:09:08 | gregwork | wait cross_az_attach defaults to true | |
| 17:09:16 | gregwork | at least thats how it looks? | |
| 17:09:47 | gregwork | "By default there is no availability zone restruction on volume attach." | |
| 17:09:51 | mriedem | correct | |
| 17:10:33 | gregwork | i see in cinder that the volumes i wanted to attach were created, but they just didnt attach | |
| 17:10:37 | gregwork | also heat didnt throw an error | |
| 17:10:39 | mriedem | read https://docs.openstack.org/nova/latest/admin/availability-zones.html#resource-affinity | |
| 17:12:34 | gregwork | mriedem: i read that but since my cross_az_attach is unset (defaulted to true) that doesnt explain why the volume silently didnt attach. it seems to list the caveats with setting it to false | |
| 17:17:10 | mriedem | are you talking about attaching volumes to an existing server or boot from volume where nova creates the volumes and attaches them to the server being created? | |
| 17:17:34 | openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Resolve (most) flake8 3.x issues https://review.opendev.org/695732 | |
| 17:17:34 | openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP: Switch to flake8 3.x https://review.opendev.org/695733 | |
| 17:19:22 | mriedem | if cross_az_attach=True (default) then nova won't specify any AZ when creating volumes during boot from volume, | |
| 17:19:36 | mriedem | if the attach is failing, it's likely due to something with the setup/storage backend etc | |
| 17:19:42 | mriedem | so you'd have to dig into that | |
| 17:21:34 | stephenfin | have a good weekend | |
| 17:24:05 | openstackgerrit | Merged openstack/nova stable/train: Added openssh-client into bindep https://review.opendev.org/691808 | |
| 17:51:03 | openstackgerrit | Merged openstack/nova stable/stein: Use admin neutron client to query ports for binding https://review.opendev.org/694665 | |
| 17:51:10 | openstackgerrit | Merged openstack/nova master: Abort live-migration during instance_init https://review.opendev.org/678016 | |
| 17:59:07 | gregwork | mriedem: so you were right, it was something failing with cinder, however (and this is probably a bug) the volume attachment would fail because nova running "multipathd show status" failed (multipathd was not started on the compute node). the issue is that neither heat, nor nova, nor manually doing it in horizon caught the exception. | |
| 17:59:18 | gregwork | it was only when i looked at nova-compute.log on the compute node that i saw the traceback | |
| 17:59:31 | gregwork | im using queens so maybe this is fixed upstream | |
| 18:00:00 | gregwork | just took the request and happily said it was done | |
| 18:13:57 | sean-k-mooney | well on one hand managing the lifecycle of multipathd is out of scope of nova | |
| 18:15:07 | sean-k-mooney | im not sure if os-brick would have detechted the fact that multipathd was not running or if nova would to be able to raise it and im not sure if we would want to expose that to a non admin if it did | |
| 18:15:27 | sean-k-mooney | heat cant know this unless cinder/nova told it as heat has no acess to the compute node | |
| 18:15:59 | gregwork | sean-k-mooney: well from the end user experience take horizon for example.. doing volumes -> manage attachments and picking my instance .. that should throw an error | |
| 18:16:18 | gregwork | but it doesnt, it accepts the request and nothing happens | |
| 18:16:28 | gregwork | a user should be informed what they asked for failed at least | |
| 18:16:33 | gregwork | maybe not the precise details | |
| 18:16:49 | sean-k-mooney | well it shoudl go to an error state or not be marked as attached unless it completes | |
| 18:17:08 | gregwork | but it doesnt which is why im mentioning it here | |
| 18:17:37 | sean-k-mooney | so nova could do two things at that point | |
| 18:17:46 | sean-k-mooney | it could put the instance in an error state | |
| 18:17:53 | sean-k-mooney | or it could role back the attachment | |
| 18:17:55 | gregwork | (to be clear it doesnt mark it as attached, it just returns an unmodified horizon screen) | |
| 18:18:15 | sean-k-mooney | ok well that is different | |
| 18:18:23 | sean-k-mooney | what is the volume status | |
| 18:18:33 | gregwork | available | |
| 18:18:42 | gregwork | it doesnt go into an error state | |
| 18:18:44 | sean-k-mooney | then it sound like its doing the right thing | |
| 18:18:52 | gregwork | its like you didnt do anything at all | |
| 18:19:06 | sean-k-mooney | yes because it tried to attach and failed | |
| 18:19:07 | melwitt | did you check the instance actions api? :P | |
| 18:19:17 | sean-k-mooney | i was just going to mention that | |
| 18:19:28 | melwitt | I won! | |
| 18:19:33 | melwitt | jk | |
| 18:19:37 | sean-k-mooney | that is where the error if it was going to be reported would likely end up | |
| 18:19:47 | gregwork | in cinder or nova ? | |
| 18:19:51 | melwitt | nova | |
| 18:20:39 | melwitt | I don't know how it looks on horizon, but it's like 'openstack server event list' on the openstackclient | |
| 18:21:40 | gregwork | so i cant openstack server event list <that instance> because we fixed it just now (configuring multipathd) and redeployed the stack | |
| 18:21:40 | sean-k-mooney | in horizon there is the action log for the instace | |
| 18:21:47 | gregwork | where would it be in the infra | |
| 18:21:51 | sean-k-mooney | when you click into it | |
| 18:21:52 | gregwork | compute or control plane | |
| 18:22:04 | gregwork | the logs you are looking for | |