| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-02-14 | |||
| 18:09:52 | openstack | bug 1839560 in OpenStack Compute (nova) stein "ironic: moving node to maintenance makes it unusable afterwards" [High,Fix committed] https://launchpad.net/bugs/1839560 - Assigned to Matt Riedemann (mriedem) | |
| 18:09:52 | openstackgerrit | Artom Lifshitz proposed openstack/nova stable/queens: Add functional regression recreate test for bug 1839560 https://review.opendev.org/707886 | |
| 18:10:04 | artom | And while that runs, I need lunch | |
| 20:14:02 | gmann | efried: artom it was working till morning with latest fix of stachviz. | |
| 20:15:17 | gmann | this stable/queens got in today- https://review.opendev.org/#/c/699718/ | |
| 20:16:26 | spatel | sean-k-mooney: question if you around | |
| 20:17:23 | spatel | when we do cpu pinning in that case does openstack make sure it properly map with sibling? | |
| 20:17:41 | artom | gmann, thanks, I'll try with and without that change | |
| 20:18:47 | gmann | artom: but i would not be surprise if there is new one :) by seeing current frequency of 1-2 issue per day on stable branches. | |
| 20:19:00 | artom | gmann, ack | |
| 20:21:50 | artom | gmann, yeah, no, still happening without that last change | |
| 20:22:35 | artom | gmann, however, it seems it's only happening on my machine... | |
| 20:24:01 | gmann | artom: until you have updated stable constraint locally or something. may be recreate tox can help ? | |
| 20:24:20 | artom | gmann, I've been doing *nothing but* recreates :) | |
| 20:24:29 | gmann | :) | |
| 20:43:45 | spatel | Does anyone know what would be the advantage of hw:cpu_thread_policy=require | |
| 20:44:08 | spatel | currently i have default which is "prefer" | |
| 20:44:25 | spatel | I am seeing performance issue so trying to understand what would be the advantage | |
| 20:48:11 | efried | spatel: let me know if this doesn't help: https://docs.openstack.org/nova/latest/admin/cpu-topologies.html#customizing-instance-cpu-thread-pinning-policies | |
| 20:48:43 | spatel | efried: i am on same page but i am trying to understand language | |
| 20:48:50 | efried | ah | |
| 20:49:14 | efried | this is not my area of expertise. Normally I would call on stephenfin or sean-k-mooney, but I think they're out at the moment. artom, how's your understanding here? | |
| 20:50:25 | spatel | currently i am seeing on vm siblings are not correctly align with my physical cores... | |
| 20:50:48 | spatel | look like cpu_thread_policy=require may fix that but wanted to understand first before i just go and play | |
| 20:51:42 | artom | spatel, it's getting into the nitty gritty of low level CPU arch, which is not my strong point | |
| 20:52:13 | artom | But IIUC putting workloads on thread siblings (policy=require) means the content for the hardware that's shared between the two threads | |
| 20:52:35 | artom | I *think* there would be 2 decoding units, but a single execution unit | |
| 20:53:06 | artom | So if your workload is CPU heavy, I would advice against putting it on thread sibligns, and use policy=isolate (at the cost of unusable CPU threads) | |
| 20:53:29 | spatel | we are running Erlang application which is CPU and memory bound | |
| 20:53:30 | artom | *means there's contention | |
| 20:54:07 | artom | spatel, OK, so try `isolate`, see if there's an improvement | |
| 20:54:10 | spatel | and i want to create single VM on single compute (want to assign all resources to vm) | |
| 20:54:52 | spatel | artom: i am reading all other document and they are saying for performance use policy=require it will make sure your process stay with sibling.. | |
| 20:55:04 | spatel | trying to understand this language | |
| 20:55:40 | artom | spatel, "The presence of an SMT implementation like Intel Hyper-Threading can boost performance by up to 30% for some workloads. However, thread siblings share a number of components and contention on these components can diminish performance for other workloads." | |
| 20:55:51 | artom | spatel, there's no magic answer, it depends on the workload | |
| 20:56:19 | spatel | hmm! | |
| 20:56:48 | spatel | Just FYI - i am running "qemu-kvm-ev-2.12.0-33.1.el7_7.4.x86_64" hope this version is fully stable | |
| 20:57:33 | artom | Presumably it is, if you got from your OS's regular repos/sources | |
| 20:57:58 | artom | But I'm not a qemu developer, so I don't know off the top of my head (and qemu developers themselves probably don't) | |
| 21:32:13 | abaindur | hello, seeing an issue with SRIOV. When we delete a VM, it's VF is released back, but then we see the VF running DHCP. This causes the VF to get assigned an IP and change default route out of the host, screwing things up | |
| 21:32:52 | abaindur | Dont know if this is openstack related, or NIC driver related, or some Linux setting | |
| 21:34:04 | abaindur | Jan 30 16:36:44 compute-02 systemd: Starting DHCP interface eth40... | |
| 21:35:30 | abaindur | NetworkManager is disabled... and the physical NIC for the VFs has NM_CONTROLLED=no and BOOTPROTO=none | |
| #openstack-nova - 2020-02-15 | |||
| 10:13:30 | gibi | efried, bauzas: NUMA spec is now approved | |
| 10:13:34 | gibi | nice work! | |
| 10:25:13 | openstackgerrit | Merged openstack/nova-specs master: Proposes NUMA topology with RPs https://review.opendev.org/552924 | |
| 10:33:10 | openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Support re-configure deleted_on_termination in server https://review.opendev.org/580336 | |
| 10:34:26 | openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Support re-configure deleted_on_termination in server https://review.opendev.org/580336 | |
| #openstack-nova - 2020-02-16 | |||
| 20:16:09 | openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP libvirt: Support attaching RBD volumes as host block devices https://review.opendev.org/708029 | |
| 20:17:19 | openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP workarounds: Add option to disable native LUKSv1 decryption by QEMU https://review.opendev.org/708030 | |
| 20:32:24 | openstackgerrit | Lee Yarwood proposed openstack/nova master: DNM: Test RBD block device support https://review.opendev.org/708033 | |
| 20:53:18 | openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP workarounds: Add option to disable native LUKSv1 decryption by QEMU https://review.opendev.org/708030 | |
| 20:53:19 | openstackgerrit | Lee Yarwood proposed openstack/nova master: DNM: Test RBD block devices with dm-crypt encryptor workaround https://review.opendev.org/708036 | |
| 20:54:35 | openstackgerrit | Lee Yarwood proposed openstack/nova master: DNM: Test RBD block device support https://review.opendev.org/708033 | |
| #openstack-nova - 2020-02-17 | |||
| 05:41:44 | openstackgerrit | Abhishek Kekane proposed openstack/nova-specs master: Fixed minor suggestions https://review.opendev.org/708058 | |
| 07:37:21 | openstackgerrit | Kobi Samoray proposed openstack/nova master: Support large network queries towards neutron https://review.opendev.org/704520 | |
| 07:39:04 | openstackgerrit | Kobi Samoray proposed openstack/nova master: Support large network queries towards neutron https://review.opendev.org/704520 | |
| 08:26:10 | bauzas | gibi: thanks for +Wing https://review.opendev.org/#/c/552924/ I'll create a FUP change | |
| 09:23:08 | gibi | bauzas: I was happy to push it through | |
| 09:23:54 | bauzas | cool thanks | |
| 10:23:49 | openstackgerrit | Brin Zhang proposed openstack/nova master: Add new default roles in os-instance-actions policies https://review.opendev.org/706470 | |
| 10:30:39 | openstackgerrit | Brin Zhang proposed openstack/nova master: Add new default roles in os-instance-actions policies https://review.opendev.org/706470 | |
| 10:40:26 | openstackgerrit | Sylvain Bauza proposed openstack/nova-specs master: FUP for NUMA Topology spec https://review.opendev.org/708088 | |
| 10:40:46 | bauzas | gibi: ^ | |
| 10:40:53 | gibi | bauzas: ack | |
| 10:42:13 | gibi | bauzas: thanks, +2 | |
| 10:42:20 | bauzas | kk thanks | |
| 11:32:29 | openstackgerrit | Kevin Zhao proposed openstack/nova master: [WIP] Add arm64 CI https://review.opendev.org/708102 | |
| 11:36:46 | openstackgerrit | Kevin Zhao proposed openstack/nova master: [WIP] Add arm64 CI https://review.opendev.org/708102 | |
| 11:42:14 | openstackgerrit | Kevin Zhao proposed openstack/nova master: [WIP] Add arm64 CI https://review.opendev.org/708102 | |
| 12:11:12 | openstackgerrit | Luyao Zhong proposed openstack/nova master: bug-fix: Reject live migration with vpmem https://review.opendev.org/708110 | |
| 12:14:07 | openstackgerrit | Kevin Zhao proposed openstack/nova master: [WIP] Add arm64 CI https://review.opendev.org/708102 | |
| 12:25:09 | openstackgerrit | Kevin Zhao proposed openstack/nova master: [WIP] Add arm64 CI https://review.opendev.org/708102 | |
| 12:35:37 | openstackgerrit | Kevin Zhao proposed openstack/nova master: [WIP] Add arm64 CI https://review.opendev.org/708102 | |
| 12:47:00 | openstackgerrit | Kevin Zhao proposed openstack/nova master: [WIP] Add arm64 CI https://review.opendev.org/708102 | |
| 12:49:40 | openstackgerrit | Kevin Zhao proposed openstack/nova master: [WIP] Add arm64 CI https://review.opendev.org/708102 | |
| 12:55:35 | sean-k-mooney | bauzas: gibi by the way rather then adapt the exitisting filter to work on allocation candiates we might actully want to a seperate filtering laywer that works on allocatgion candiates before the existing filters | |
| 12:56:10 | gibi | sean-k-mooney: that could be an alternative | |
| 12:56:30 | sean-k-mooney | that would avoid needing to pass an addtional paramater to all the filters so we wont break out of tree filters by changing the interface | |
| 12:56:37 | gibi | sean-k-mooney: it has a trade off to change the order of the filters as then NTF would run before any other filter | |
| 12:57:10 | gibi | but the such order just affects scheduling performance not scheduling result | |
| 12:57:32 | sean-k-mooney | im suggesting we leave the NTF unchanged and add a new one but yes the oder does affect performance | |
| 12:57:38 | sean-k-mooney | *order | |
| 12:58:16 | sean-k-mooney | you could put the new filter after the exstiing onece i guess | |
| 12:58:37 | sean-k-mooney | the NTF normally should go at the end of the existing list anyway | |
| 12:58:45 | sean-k-mooney | so maybe that would make more sense | |
| 12:59:42 | sean-k-mooney | so prefilters->placemnt->currnent filters->allocation candiate filters->weighers? | |
| 13:01:49 | gibi | sean-k-mooney: yeah, NTF tend to be at the end of the list so having the ac filter at the end would be closer to todays behavior | |
| 13:02:31 | sean-k-mooney | from an information gain point of view the host based fileter have the potentail to eliminate multipel allocation candiates by removing the host | |
| 13:02:40 | sean-k-mooney | so i think it makes sesne for them to come after too | |
| 13:02:52 | bauzas | sean-k-mooney: well, let's discuss this during the implementation | |
| 13:03:00 | sean-k-mooney | ya we can | |
| 13:03:17 | sean-k-mooney | i was just wondering if we could avoid breaking out of tree filters | |
| 13:03:27 | sean-k-mooney | which i think this would allow | |
| 13:07:18 | openstackgerrit | Brian Rosmaita proposed openstack/nova master: Reject boot request for unsupported images https://review.opendev.org/708126 | |
| 13:20:09 | gibi | stephenfin: do you have some ideas why jsonutils fails while json does not in https://review.opendev.org/#/c/703133 ? | |
| 13:20:55 | stephenfin | have you got a traceback? | |
| 13:21:28 | gibi | stephenfin: sure, give me a sec | |
| 13:21:51 | gibi | stephenfin: http://paste.openstack.org/show/789642/ | |