| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-02-08 | |||
| 17:03:53 | mriedem | unless you're still running essex somewhere | |
| 17:04:24 | hrw | essex for me is some part of uk or sth like that ;D | |
| 17:04:55 | openstackgerrit | Eric Fried proposed openstack/nova master: Ensure resource classes correctly https://review.openstack.org/539738 | |
| 17:04:55 | openstackgerrit | Eric Fried proposed openstack/nova master: New-style _set_inventory_for_provider https://review.openstack.org/537648 | |
| 17:04:56 | openstackgerrit | Eric Fried proposed openstack/nova master: SchedulerReportClient.update_from_provider_tree https://review.openstack.org/533821 | |
| 17:04:57 | openstackgerrit | Eric Fried proposed openstack/nova master: Use update_provider_tree from resource tracker https://review.openstack.org/520246 | |
| 17:04:57 | openstackgerrit | Eric Fried proposed openstack/nova master: Fix nits in update_provider_tree series https://review.openstack.org/531260 | |
| 17:04:57 | openstackgerrit | Eric Fried proposed openstack/nova master: Move refresh time from report client to prov tree https://review.openstack.org/535517 | |
| 17:04:58 | openstackgerrit | Eric Fried proposed openstack/nova master: Make generation optional in ProviderTree https://review.openstack.org/539324 | |
| 17:04:58 | openstackgerrit | Eric Fried proposed openstack/nova master: WIP: Add nested resources to server moving tests https://review.openstack.org/527728 | |
| 17:06:07 | stvnoyes | mriedem: fyi - I just updated that bug on multiattach + multi-create. The next issue is stickier than the first one. https://bugs.launchpad.net/nova/+bug/1747985 | |
| 17:06:08 | openstack | Launchpad bug 1747985 in OpenStack Compute (nova) "Create Multiple Servers fails when a volume is specified even with a multiattach volume" [Low,Confirmed] | |
| 17:06:26 | bauzas | mriedem: diablo, not essex | |
| 17:06:34 | bauzas | mriedem: cinder was a thing by essex | |
| 17:06:45 | mriedem | nova-volume still existed in essex | |
| 17:06:54 | smcginnis | Split out in folsom. | |
| 17:07:05 | mriedem | check and mate | |
| 17:07:09 | efried | jaypipes: gory manual rebase on ==> Ensure resource classes correctly https://review.openstack.org/539738 No functional change, ready for re-review (ditto remainder of series) | |
| 17:07:13 | bauzas | wait | |
| 17:07:25 | bauzas | nova-volume was the thing in Essex ? | |
| 17:07:34 | bauzas | I thought it was diablo | |
| 17:07:37 | smcginnis | Trivial Persuit - OpenStack Edition :) | |
| 17:07:47 | bauzas | the only change I made when upgrading to Folsom was to deploy Quantum | |
| 17:08:01 | bauzas | I don't recall needing to spin cinder | |
| 17:08:07 | mriedem | https://github.com/openstack/nova/blob/essex-eol/bin/nova-api-os-volume | |
| 17:08:41 | bauzas | mmm, I should check the governance files directly and see if that's just an incubated/integrated question | |
| 17:08:46 | mriedem | https://github.com/openstack/cinder/tree/essex-eol | |
| 17:08:57 | hrw | the first version of openstack I touched code was Liberty. Mitaka/Newton were ones I packaged. Ocata+ were ones I contributed | |
| 17:09:34 | openstackgerrit | Merged openstack/osc-placement master: Update doc link in README.rst https://review.openstack.org/542002 | |
| 17:09:50 | mriedem | stvnoyes: re comment 2, yes that's what i was saying when i thought this would fail, but with a multiattach volume we should be able to create >1 attachment to that same volume | |
| 17:10:09 | mriedem | stvnoyes: or is the issue that we create the first attachment, but we can't create the 2nd attachment until the 1st attachment is 'completed'? | |
| 17:10:44 | stvnoyes | currently cinder won't allow you to attach to a reserved volume. has to be in-use or available | |
| 17:11:08 | stvnoyes | so yes, the attach has to be completed | |
| 17:11:19 | bauzas | snap, I dislike to be wrong https://wiki.openstack.org/wiki/ReleaseNotes/Folsom#OpenStack_Block_Storage_.28Cinder.29 | |
| 17:11:48 | mriedem | stvnoyes: we could easily recreate by adding a min_count=2 to the server create call in this tempest test https://github.com/openstack/tempest/blob/70ebe69cc2e4fa8a06071d48d19f61d2f01a33f6/tempest/api/compute/volumes/test_attach_volume.py#L393 | |
| 17:12:02 | mriedem | i can push a patch quick | |
| 17:12:16 | bauzas | probably because migrating from nova-volumes to cinder was seamless - compared to deploying Quantum *cough* *cough* | |
| 17:12:18 | stvnoyes | yeah I have a temptest test already to do this. I can put it up for rv if you like | |
| 17:12:53 | smcginnis | bauzas: Hah | |
| 17:13:17 | ameeda | mriedem: Thank you, please check the channel , they answered me | |
| 17:14:28 | stephenfin | dansmith: Alright, so I think I've figured this all out, if you're curious | |
| 17:14:41 | stephenfin | this being the RPC versioning for those functions | |
| 17:15:03 | stvnoyes | mriedem: let me know if you want me to post a review for the multi-create + multi-attach tempest test I have. (Once you go > 1 vms you'll need to set validated=False on server_create.) | |
| 17:15:34 | dansmith | stephenfin: shoot | |
| 17:15:55 | mriedem | stvnoyes: easier than this? https://review.openstack.org/542322 | |
| 17:16:34 | stephenfin | dansmith: So the issue isn't with the HostState object any more - it's with the HostState.numa_topology attribute | |
| 17:16:34 | ameeda | mriedem: FYI: ameeda: in that case "test" is just an arbitrary profile name we've settled on to indicate that it's not a runtime dependency just a test dependency | |
| 17:16:51 | stvnoyes | mriedem: that's pretty good. | |
| 17:17:34 | mriedem | ameeda: that's what i figured, so yeah just use that, we only need this dep for building docs | |
| 17:17:48 | stephenfin | dansmith: Anything that plays with HostState objects needs to call functions like this https://github.com/openstack/nova/blob/master/nova/scheduler/filters/numa_topology_filter.py#L32-L33 | |
| 17:18:08 | stvnoyes | Though I was using the volume as a data disk, not a boot disk. just to avoid the corruption on a m-a boot disk. | |
| 17:18:21 | stephenfin | That's because we set HostState.numa_topology to a value that we're retrieving from ComputeNode objects (ComputeNode.numa_topology) | |
| 17:18:21 | ameeda | mriedem: cool :) | |
| 17:18:39 | stephenfin | and, for legacy reasons, the ComputeNode.numa_topology attribute is a JSON blob | |
| 17:19:22 | stephenfin | dansmith: I've traced the flow here http://paste.openstack.org/show/666746/ | |
| 17:20:39 | stephenfin | dansmith: So, basically, if we want to _really_ get rid of all that crap, we need to bump the version of ComputeNode to 2.0 and store objects, or add some wrapper that will auto-convert the JSON to an object | |
| 17:20:55 | dansmith | stephenfin: that's going to be a lot of work | |
| 17:21:00 | stephenfin | Yup :( | |
| 17:21:11 | dansmith | definitely not happening as part of this effort before GA | |
| 17:21:41 | stephenfin | Anything that deals with converting the HostState objects from strings or non-object'y stuff _can_ go and the comments need to be updated as such | |
| 17:21:57 | stephenfin | Nope, definitely not. Far too much intertwined stuff here | |
| 17:23:59 | openstackgerrit | melanie witt proposed openstack/nova stable/ocata: Add release note for Aggregate[Core|Ram|Disk]Filter change https://review.openstack.org/542323 | |
| 17:25:00 | hrw | hm. whom to believe... "nova volume-attach dd9 uu-i-d" told "/dev/vdb" and on second try says "volume already attached" but horizon/instance/overview says "no volumes attached" | |
| 17:26:25 | hrw | multitail++ | |
| 17:27:37 | hrw | ok. logs show traceback. will keep unknown then | |
| 17:30:02 | bauzas | stephenfin: dansmith: I don't have the context, but yeah, it requires a major version bump to get rid of the past JSON string | |
| 17:31:15 | openstackgerrit | Ameed Ashour proposed openstack/nova master: Bindep does not catch missing libpcre3-dev on Ubuntu https://review.openstack.org/542340 | |
| 17:32:16 | openstackgerrit | Eric Fried proposed openstack/nova master: Avoid inventory DELETE API (no conflict detection) https://review.openstack.org/539712 | |
| 17:32:29 | efried | jaypipes: ^ another gory rebase | |
| 17:41:45 | sean-k-mooney | stephenfin: glad to hear. | |
| 17:42:14 | sean-k-mooney | stephenfin: did you modify our ovs-dpdk compilateion code to explcitly enable numa support? | |
| 17:44:54 | openstackgerrit | Marcin Juszkiewicz proposed openstack/nova master: Provide basic data for AArch64 support matrix/functionality https://review.openstack.org/541728 | |
| 17:45:21 | hrw | still several unknown states but should be closer to sanity | |
| 17:45:27 | hrw | mriedem: upgrade note added | |
| 17:46:16 | sean-k-mooney | stephenfin: try this before you get too far in grep CONFIG_RTE_LIBRTE_VHOST_NUMA /opt/stack/DPDK-*/build/.config. if it is set to n which i belive it will be the numa support in ovs/dpdk is disabled but that should allow you to get the worst case setup | |
| 17:48:06 | stephenfin | sean-k-mooney: I did not. Damn it | |
| 17:48:19 | sean-k-mooney | stephenfin: actullly wost case is nic on numa node 0, vcpu_pin_set allows only numa node 0 cpus and ovs_pmd_mask set pmd on numa node 1 | |
| 17:49:02 | sean-k-mooney | stephenfin: well if you want to mesure the maxium performance hit then leave it disabled and set up the topology above. | |
| 17:49:13 | mriedem | so apparently this is a regression unique to queens https://review.openstack.org/#/c/522161/ | |
| 17:49:20 | mriedem | recreated it here https://review.openstack.org/#/c/542232/ | |
| 17:49:26 | mriedem | doesn't fail in pike, ocata or newton | |
| 17:50:31 | sean-k-mooney | stephenfin: if you want to fix it just add another sed line here https://github.com/openstack/networking-ovs-dpdk/blob/master/devstack/libs/ovs-dpdk#L392-L399 | |
| 17:54:02 | openstackgerrit | Dan Smith proposed openstack/nova master: Bump compute RPC API to version 5.0 https://review.openstack.org/541005 | |
| 17:54:03 | openstackgerrit | Dan Smith proposed openstack/nova master: Compute RPC client bump to 5.0 https://review.openstack.org/541035 | |
| 17:57:44 | dansmith | melwitt: I was about to say this dep change proves your patch works, but.. I didn't think you could specify the url as the depends-on.. did I miss them adding that feature? https://review.openstack.org/#/c/540087/ | |
| 17:58:09 | melwitt | dansmith: yeah. you missed the controversy /me digs up link | |
| 17:58:18 | dansmith | ugh seriously? | |
| 17:59:10 | melwitt | here's the original post http://lists.openstack.org/pipermail/openstack-dev/2018-January/126535.html | |
| 17:59:16 | dansmith | yeah found it | |
| 17:59:19 | dansmith | me no likey | |
| 17:59:24 | melwitt | same | |
| 18:03:28 | dansmith | melwitt: mriedem: so on that change, we passed all but one in the last round, which was a volume timeout | |
| 18:03:34 | dansmith | same for the dep patch, | |
| 18:03:40 | dansmith | but on this recheck we failed a ton of volume things | |
| 18:04:03 | dansmith | I would expect these are unrelated, but... know of any known-broken volume stuff at the moment? | |
| 18:04:18 | openstackgerrit | Ameed Ashour proposed openstack/nova master: change instance_system_metadata column type https://review.openstack.org/526900 | |
| 18:04:29 | mriedem | this job http://logs.openstack.org/87/540087/2/check/tempest-full-py3/0d74482/ ? | |
| 18:04:45 | dansmith | and tempest-full | |
| 18:06:24 | mriedem | looks like cinder-scheduler shit the bed http://logs.openstack.org/42/541442/3/check/tempest-full/3f33082/controller/logs/screen-c-sch.txt.gz?level=TRACE | |
| 18:06:27 | mriedem | which isn't our fault | |