Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-03-13
22:50:51 fungi we couldn't actually show the effect of the change without a depends-on chain to an alternate universe base job
22:51:06 fungi at least not until the actual base job change merged
22:52:07 efried ack
22:52:07 efried and with that, I'm outta here o/
23:25:13 openstackgerrit Merged openstack/nova master: Update instance.availability_zone during live migration https://review.openstack.org/567701
23:25:24 openstackgerrit Merged openstack/nova master: Skip the ceph based live migration testing https://review.openstack.org/643122
23:40:27 gmann fungi: efried we have cross-dependency to unblock the gate. 643129 cannot merge which need 642940 and vice versa.
23:41:17 fungi possible we'll need to squash those
23:41:26 gmann fungi: can we get this forcefully in - 642940? or i can make temp patch to unblock either one
23:43:13 fungi they'll be properly tested if we squash the two commits together, but would need a nova-core on hand to approve the resulting revision of whichever change we squash into
23:43:54 gmann ok, let me squash first then i can catch someone alex_xu might be coming online soon
23:47:18 fungi i'm just veeery hesitant to bypass ci for a change where it's not strictly necessary
23:47:36 fungi pretty much every time i do, i break something a job would have caught
23:48:50 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Disable the tls-proxy in nova-next & fix nova-tox-functional-py35 parent https://review.openstack.org/643129
23:48:55 gmann done ^^
23:49:33 gmann efried: melwitt dansmith alex_xu bauzas stephenfin mriedem_afk ^^ if you are around.
23:50:02 openstackgerrit Merged openstack/nova master: Trivial: fix typo in reno https://review.openstack.org/643137
23:54:39 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Migrate legacy jobs to Ubuntu Bionic https://review.openstack.org/639017
#openstack-nova - 2019-03-14
00:03:58 melwitt gmann: I'll get it. thank you for all your work on this
00:04:49 gmann thanks.
00:13:18 fungi thanks melwitt!
00:19:51 openstackgerrit melanie witt proposed openstack/nova master: Re-enable Ceph in live migration testing https://review.openstack.org/643150
01:12:46 openstackgerrit Matt Riedemann proposed openstack/nova master: Add functional recreate test for bug 1819963 https://review.openstack.org/643151
01:12:46 openstack bug 1819963 in OpenStack Compute (nova) "Reverting a resize does not update the instance.availability_zone value to the source az" [Medium,In progress] https://launchpad.net/bugs/1819963 - Assigned to Matt Riedemann (mriedem)
01:12:46 openstackgerrit Matt Riedemann proposed openstack/nova master: Update instance.availability_zone on revertResize https://review.openstack.org/643155
02:14:20 openstackgerrit Takashi NATSUME proposed openstack/nova stable/rocky: Fix resetting non-persistent fields when saving obj https://review.openstack.org/643215
02:31:54 openstackgerrit huanhongda proposed openstack/nova stable/queens: Fix resource tracker updates during instance evacuation https://review.openstack.org/643219
02:44:00 openstackgerrit Boxiang Zhu proposed openstack/nova master: Raise error when force_raw_images and images_type are incompatible https://review.openstack.org/640271
04:06:33 openstackgerrit Merged openstack/nova master: Disable the tls-proxy in nova-next & fix nova-tox-functional-py35 parent https://review.openstack.org/643129
04:14:40 openstackgerrit Takashi NATSUME proposed openstack/nova master: Override the 'get' method in DriverBlockDevice class https://review.openstack.org/638821
05:20:32 openstackgerrit Luyao Zhong proposed openstack/nova master: object: Add pmem_namespaces field to the NUMACell obj https://review.openstack.org/634547
05:20:33 openstackgerrit Luyao Zhong proposed openstack/nova master: object: Add virtual_pmems fields to the InstanceNUMACell obj https://review.openstack.org/634548
05:20:33 openstackgerrit Luyao Zhong proposed openstack/nova master: libvirt: Enable driver configures PMEM namespace when initiating libvirt driver https://review.openstack.org/634549
05:20:34 openstackgerrit Luyao Zhong proposed openstack/nova master: libvirt: Update PMEM namespaces info and usage https://review.openstack.org/634550
05:20:34 openstackgerrit Luyao Zhong proposed openstack/nova master: libvirt: report pmem namespaces resources by provider tree https://review.openstack.org/634551
05:20:35 openstackgerrit Luyao Zhong proposed openstack/nova master: API: parse pmem related flavor extra spec https://review.openstack.org/634552
05:20:35 openstackgerrit Luyao Zhong proposed openstack/nova master: scheduler: translate virtual pmems request to placement request group https://review.openstack.org/634553
05:20:36 openstackgerrit Luyao Zhong proposed openstack/nova master: update _numa_fit_instance_cell for the support of virtual_pmems https://review.openstack.org/634554
05:20:36 openstackgerrit Luyao Zhong proposed openstack/nova master: libvirt: boot VM with vpmems and cleanup vpmems when destroying https://review.openstack.org/634555
05:20:37 openstackgerrit Luyao Zhong proposed openstack/nova master: support VM resizing with vpmem data migration https://review.openstack.org/634556
07:35:49 openstackgerrit zhaixiaojun proposed openstack/nova master: Fix bug: non-admin user can not filter instance by instance-uuid https://review.openstack.org/642403
07:47:49 openstackgerrit Boxiang Zhu proposed openstack/nova master: Raise error when force_raw_images and images_type are incompatible https://review.openstack.org/640271
08:34:56 openstackgerrit Yongli He proposed openstack/nova master: Clean up orphan instances https://review.openstack.org/627765
08:44:01 openstackgerrit Merged openstack/nova master: Update compute rpc version alias for stein https://review.openstack.org/642599
09:04:57 openstackgerrit Merged openstack/nova master: Migrate legacy jobs to Ubuntu Bionic https://review.openstack.org/639017
09:05:56 openstackgerrit Boxiang Zhu proposed openstack/nova master: Remove unsafe convert of lvm imagebackend https://review.openstack.org/642667
09:08:44 kashyap gibi: Morning, mind having a quick gander at this (simple change, but brings non-trivial perf benefit): https://review.openstack.org/#/c/640781/
09:12:04 openstackgerrit zhaixiaojun proposed openstack/nova master: Fix bug: non-admin user can not filter instance by instance-uuid https://review.openstack.org/642403
10:49:41 jangutter kashyap: 15x --- there's that story that nearly every programming task reduces to either a caching or string manipulation problem.
10:49:57 kashyap jangutter: Yeah...
10:50:26 kashyap It was a "one-worder" change, but I guess you read the full commit message
10:50:37 kashyap It took 2 days to dig into QEMU and talk to Block Layer folks
10:51:08 jangutter and the persistent nagging feeling that it can't be this simple.
11:22:18 kashyap jangutter: Yeah, need to consider impact on multiple use cases
11:23:20 kashyap mdbooth: Morning, if you get a moment: https://review.openstack.org/#/c/641981/ and https://review.openstack.org/#/c/640781/
11:23:26 kashyap (You were already +1 on those before.)
11:24:08 mdbooth "But that's not correct, because of our misunderstanding of how cache modes work."
11:24:23 mdbooth I'll say again: we never understood how cache modes work, only what they're called ;)
11:24:34 mdbooth ...misunderstood...
11:24:47 mdbooth Don't change that, though.
11:25:07 kashyap Hehe, noted.
11:25:38 kashyap Now I think not in terms of just named cache modes, but the three booleans each one impacts
11:27:22 kashyap mdbooth: FWIW, the little blob of text written in driver.py -- I got it reviewed by Kevin (of QEMU) over a few emails. So it's not me just riffing on it :-)
11:37:16 mdbooth ack
11:52:11 openstackgerrit Fan Zhang proposed openstack/nova master: Retry after hitting libvirt error VIR_ERR_OPERATION_INVALID in live migration. https://review.openstack.org/612272
12:10:17 awalende I'am trying to use hugepages with nova and qemu/libvirt. Hugepages is enabled and I have reserved a bunch of hugepages in standard 2048kb size. I added flavor properties with "flavor set hugePagesTest --property hw:mem_page_size=2Mb". However starting instances fails with "Exceeded maximum number of retries". Nova-compute log does not show any exceptions. Any Idea how to debug this?
12:34:57 efried awalende: Have you looked at the scheduler log?
12:36:04 awalende Yes, I could narrow it down. It has something to do with NUMA-Topology:
12:36:05 awalende Insufficient compute resources: Requested instance NUMA topology cannot fit the given host NUMA topology.
12:36:50 awalende Which is weird, I have setup hw:numa_nodes=4 in the flavor, since my Intel Xeon has 4 Nodes.
12:37:32 awalende beforeheand there is a bunch of repeating logs like: Attempting to fit instance cell InstanceNUMACell(cpu_pinning_raw=None,cpu_policy=None,cpu_thread_policy=None,cpu_topology=<?>,cpuset=set([0,1,2,3,4,5,6,7,8,9,10,11,12,13]),cpuset_reserved=None,id=0,memory=512,pagesize=256) on host_cell NUMACell(cpu_usage=0,cpuset=set([3,51,35,7,15,11,47,43,19,23,39,27,55,31]),id=3,memory=193532,memory_usage=0,mempages=[NUMAP
12:37:32 awalende agesTopology,NUMAPagesTopology,NUMAPagesTopology],pinned_cpus=set([]),siblings=[])
12:43:38 efried mdbooth: are you a NUMA guy?
12:43:51 efried awalende: (looking for someone who can spell NUMA)
12:43:58 awalende :D
12:46:50 openstackgerrit Jens Harbott (frickler) proposed openstack/nova master: Re-enable Ceph in live migration testing https://review.openstack.org/643150
12:55:49 openstackgerrit Elod Illes proposed openstack/nova stable/ocata: WIP: Use ubuntu-xenial for sphinx jobs https://review.openstack.org/643320
12:57:33 mdbooth efried: I am not, unfortunately.
12:58:01 efried kashyap: what about you? :)
12:58:33 mdbooth efried: I can spell it, though. Our 2 best numa guys are travelling today.
12:58:46 efried okay.
12:58:58 efried eventually here I'm gonna need to learn something about it...
12:59:10 kashyap efried: Hi, I have a NUMA-capable machine, and occasionally do testing of NUMA+SR-IOV stuff; besides that I am not an expert
12:59:39 kashyap efried: What's the question, it might make curious enough to learn the answer :-)
13:00:33 efried kashyap: awalende is having trouble scheduling an instance requesting 4 numa nodes and hugepages, see logs starting around :10
13:00:57 efried I'm guessing it's a simple thing - for somebody with a clue how NUMA-related scheduling works
13:01:19 efried http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2019-03-14.log.html#t2019-03-14T12:10:17
13:04:51 kashyap Hmm; the error "Requested instance NUMA topology cannot fit the given host NUMA topology" is indeed weird *if* you have told the scheduler to use more than one NUMA node
13:04:54 kashyap ... which you did.
13:06:59 mdbooth kashyap: Did you rebase that writeback patch for some reason?
13:08:18 mdbooth Incidentally, ^^^ is a super-obvious change in light of the informative comment kashyap has added to disk_cachemode()
13:08:37 mdbooth Erm... https://review.openstack.org/#/c/641981/
13:09:07 kashyap mdbooth: Had to rebase because "add the compress flag" thingie to 'qemu-img' got merged (you'll recall it, as you reviewed it)
13:09:17 mdbooth kashyap: Ah, ok.
13:12:02 kashyap (It caused a conflict, so I was forced to fix it and rebase :-))
13:15:42 kashyap mdbooth: What about also doing: s/writethrough/writeback/ for SMBFS and virtuozzo?
13:15:50 kashyap See: nova/virt/libvirt/volume/smbfs.py and nova/virt/libvirt/volume/vzstorage.py.
13:16:04 mdbooth kashyap: We should do that.
13:16:14 mdbooth kashyap: Separate patches though, please.
13:16:18 kashyap Yes!

Earlier   Later