| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-03-14 | |||
| 07:47:49 | openstackgerrit | Boxiang Zhu proposed openstack/nova master: Raise error when force_raw_images and images_type are incompatible https://review.openstack.org/640271 | |
| 08:34:56 | openstackgerrit | Yongli He proposed openstack/nova master: Clean up orphan instances https://review.openstack.org/627765 | |
| 08:44:01 | openstackgerrit | Merged openstack/nova master: Update compute rpc version alias for stein https://review.openstack.org/642599 | |
| 09:04:57 | openstackgerrit | Merged openstack/nova master: Migrate legacy jobs to Ubuntu Bionic https://review.openstack.org/639017 | |
| 09:05:56 | openstackgerrit | Boxiang Zhu proposed openstack/nova master: Remove unsafe convert of lvm imagebackend https://review.openstack.org/642667 | |
| 09:08:44 | kashyap | gibi: Morning, mind having a quick gander at this (simple change, but brings non-trivial perf benefit): https://review.openstack.org/#/c/640781/ | |
| 09:12:04 | openstackgerrit | zhaixiaojun proposed openstack/nova master: Fix bug: non-admin user can not filter instance by instance-uuid https://review.openstack.org/642403 | |
| 10:49:41 | jangutter | kashyap: 15x --- there's that story that nearly every programming task reduces to either a caching or string manipulation problem. | |
| 10:49:57 | kashyap | jangutter: Yeah... | |
| 10:50:26 | kashyap | It was a "one-worder" change, but I guess you read the full commit message | |
| 10:50:37 | kashyap | It took 2 days to dig into QEMU and talk to Block Layer folks | |
| 10:51:08 | jangutter | and the persistent nagging feeling that it can't be this simple. | |
| 11:22:18 | kashyap | jangutter: Yeah, need to consider impact on multiple use cases | |
| 11:23:20 | kashyap | mdbooth: Morning, if you get a moment: https://review.openstack.org/#/c/641981/ and https://review.openstack.org/#/c/640781/ | |
| 11:23:26 | kashyap | (You were already +1 on those before.) | |
| 11:24:08 | mdbooth | "But that's not correct, because of our misunderstanding of how cache modes work." | |
| 11:24:23 | mdbooth | I'll say again: we never understood how cache modes work, only what they're called ;) | |
| 11:24:34 | mdbooth | ...misunderstood... | |
| 11:24:47 | mdbooth | Don't change that, though. | |
| 11:25:07 | kashyap | Hehe, noted. | |
| 11:25:38 | kashyap | Now I think not in terms of just named cache modes, but the three booleans each one impacts | |
| 11:27:22 | kashyap | mdbooth: FWIW, the little blob of text written in driver.py -- I got it reviewed by Kevin (of QEMU) over a few emails. So it's not me just riffing on it :-) | |
| 11:37:16 | mdbooth | ack | |
| 11:52:11 | openstackgerrit | Fan Zhang proposed openstack/nova master: Retry after hitting libvirt error VIR_ERR_OPERATION_INVALID in live migration. https://review.openstack.org/612272 | |
| 12:10:17 | awalende | I'am trying to use hugepages with nova and qemu/libvirt. Hugepages is enabled and I have reserved a bunch of hugepages in standard 2048kb size. I added flavor properties with "flavor set hugePagesTest --property hw:mem_page_size=2Mb". However starting instances fails with "Exceeded maximum number of retries". Nova-compute log does not show any exceptions. Any Idea how to debug this? | |
| 12:34:57 | efried | awalende: Have you looked at the scheduler log? | |
| 12:36:04 | awalende | Yes, I could narrow it down. It has something to do with NUMA-Topology: | |
| 12:36:05 | awalende | Insufficient compute resources: Requested instance NUMA topology cannot fit the given host NUMA topology. | |
| 12:36:50 | awalende | Which is weird, I have setup hw:numa_nodes=4 in the flavor, since my Intel Xeon has 4 Nodes. | |
| 12:37:32 | awalende | agesTopology,NUMAPagesTopology,NUMAPagesTopology],pinned_cpus=set([]),siblings=[]) | |
| 12:37:32 | awalende | beforeheand there is a bunch of repeating logs like: Attempting to fit instance cell InstanceNUMACell(cpu_pinning_raw=None,cpu_policy=None,cpu_thread_policy=None,cpu_topology=<?>,cpuset=set([0,1,2,3,4,5,6,7,8,9,10,11,12,13]),cpuset_reserved=None,id=0,memory=512,pagesize=256) on host_cell NUMACell(cpu_usage=0,cpuset=set([3,51,35,7,15,11,47,43,19,23,39,27,55,31]),id=3,memory=193532,memory_usage=0,mempages=[NUMAP | |
| 12:43:38 | efried | mdbooth: are you a NUMA guy? | |
| 12:43:51 | efried | awalende: (looking for someone who can spell NUMA) | |
| 12:43:58 | awalende | :D | |
| 12:46:50 | openstackgerrit | Jens Harbott (frickler) proposed openstack/nova master: Re-enable Ceph in live migration testing https://review.openstack.org/643150 | |
| 12:55:49 | openstackgerrit | Elod Illes proposed openstack/nova stable/ocata: WIP: Use ubuntu-xenial for sphinx jobs https://review.openstack.org/643320 | |
| 12:57:33 | mdbooth | efried: I am not, unfortunately. | |
| 12:58:01 | efried | kashyap: what about you? :) | |
| 12:58:33 | mdbooth | efried: I can spell it, though. Our 2 best numa guys are travelling today. | |
| 12:58:46 | efried | okay. | |
| 12:58:58 | efried | eventually here I'm gonna need to learn something about it... | |
| 12:59:10 | kashyap | efried: Hi, I have a NUMA-capable machine, and occasionally do testing of NUMA+SR-IOV stuff; besides that I am not an expert | |
| 12:59:39 | kashyap | efried: What's the question, it might make curious enough to learn the answer :-) | |
| 13:00:33 | efried | kashyap: awalende is having trouble scheduling an instance requesting 4 numa nodes and hugepages, see logs starting around :10 | |
| 13:00:57 | efried | I'm guessing it's a simple thing - for somebody with a clue how NUMA-related scheduling works | |
| 13:01:19 | efried | http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2019-03-14.log.html#t2019-03-14T12:10:17 | |
| 13:04:51 | kashyap | Hmm; the error "Requested instance NUMA topology cannot fit the given host NUMA topology" is indeed weird *if* you have told the scheduler to use more than one NUMA node | |
| 13:04:54 | kashyap | ... which you did. | |
| 13:06:59 | mdbooth | kashyap: Did you rebase that writeback patch for some reason? | |
| 13:08:18 | mdbooth | Incidentally, ^^^ is a super-obvious change in light of the informative comment kashyap has added to disk_cachemode() | |
| 13:08:37 | mdbooth | Erm... https://review.openstack.org/#/c/641981/ | |
| 13:09:07 | kashyap | mdbooth: Had to rebase because "add the compress flag" thingie to 'qemu-img' got merged (you'll recall it, as you reviewed it) | |
| 13:09:17 | mdbooth | kashyap: Ah, ok. | |
| 13:12:02 | kashyap | (It caused a conflict, so I was forced to fix it and rebase :-)) | |
| 13:15:42 | kashyap | mdbooth: What about also doing: s/writethrough/writeback/ for SMBFS and virtuozzo? | |
| 13:15:50 | kashyap | See: nova/virt/libvirt/volume/smbfs.py and nova/virt/libvirt/volume/vzstorage.py. | |
| 13:16:04 | mdbooth | kashyap: We should do that. | |
| 13:16:14 | mdbooth | kashyap: Separate patches though, please. | |
| 13:16:18 | kashyap | Yes! | |
| 13:16:22 | kashyap | That's what I noted in the TODO | |
| 13:16:27 | kashyap | (In the commit) | |
| 13:16:27 | mdbooth | Also super-low priority, because likely nobody cares. | |
| 13:16:36 | kashyap | [quote] | |
| 13:16:37 | kashyap | TODO: Investigate if this is okay for Cinder volume drivers such as SMBFS and Virtuozzo. If so, then change them to use 'writeback' as well (probably in separate patches). | |
| 13:16:40 | kashyap | [/quote] | |
| 13:16:52 | kashyap | mdbooth: Yes, nobody cares, except when it comes to bite you. | |
| 13:17:04 | kashyap | Might as well do it when I remember it; or else it goes into /dev/null of my brain | |
| 13:17:06 | mdbooth | kashyap: It would only bite you if you used those things. | |
| 13:17:24 | kashyap | (Yep.) | |
| 13:28:17 | openstackgerrit | Boxiang Zhu proposed openstack/nova master: Raise error when force_raw_images and images_type are incompatible https://review.openstack.org/640271 | |
| 13:34:45 | openstackgerrit | Boxiang Zhu proposed openstack/nova master: Remove unsafe convert of lvm imagebackend https://review.openstack.org/642667 | |
| 13:37:13 | zhubx007 | mdbooth: looking forward to your review on two patches ^ thanks :) | |
| 13:45:03 | mriedem | dansmith: can i get your +2 back on https://review.openstack.org/#/c/643151/ - i had to fix a test otherwise it's the same | |
| 13:45:21 | dansmith | mriedem: well, the test failed in a weird way in functional, | |
| 13:45:30 | dansmith | so I was waiting to see how the recheck turned out | |
| 13:45:38 | mriedem | that's the infra changes | |
| 13:45:39 | mriedem | but ok | |
| 13:45:51 | mriedem | this https://review.openstack.org/#/c/643129/ | |
| 13:45:52 | dansmith | I looked in the log and couldn't see anything actually broken | |
| 13:46:00 | mriedem | it was this https://review.openstack.org/#/c/643129/4/.zuul.yaml | |
| 13:46:04 | mriedem | broke everything over night | |
| 13:46:18 | dansmith | okay... and the result is it just doesn't even run? | |
| 13:46:36 | mriedem | the old func py35 job couldn't find the py35 interpreter | |
| 13:46:47 | mriedem | with the new job parent it should be setup for that | |
| 13:46:54 | mriedem | uses a xenial rather than bionic node | |
| 13:47:08 | dansmith | okay I didn't see anything about that in the log, but that's fine, just sayin' I was looking at it already this morning and that's why I didn't just slap back on there | |
| 13:47:13 | dansmith | but if it's known | |
| 13:47:14 | dansmith | .. | |
| 13:47:37 | mriedem | thanks | |
| 13:52:21 | mriedem | frickler: looks like you fixed that ceph thing https://review.openstack.org/#/c/643150/ | |
| 13:53:14 | openstackgerrit | Eric Fried proposed openstack/nova master: WIP/PoC: Bypass ironicclient for node.get https://review.openstack.org/642899 | |
| 14:10:56 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Re-enable Ceph in live migration testing https://review.openstack.org/643150 | |
| 14:10:57 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Customize irrelevant-files for nova-live-migration job https://review.openstack.org/643346 | |
| 14:18:51 | openstackgerrit | Elod Illes proposed openstack/nova stable/ocata: Use ubuntu-xenial for sphinx jobs https://review.openstack.org/643320 | |
| 14:40:57 | dansmith | is jaypipes just never around anymore? | |
| 14:46:40 | cdent | dansmith: I _think_ he's travelling this week. Not sure if it was business or pleasure. | |
| 14:46:48 | dansmith | oh okay | |
| 14:47:09 | cdent | he was at least on a plane on monday | |
| 15:41:22 | kashyap | aspiers: When you're about, refer to comment#11 here: https://review.openstack.org/#/c/633855/11 | |
| 15:47:42 | openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: qemu: Make disk image conversion dramatically faster https://review.openstack.org/640781 | |