| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-12-20 | |||
| 09:55:16 | danpawlik | Hi, I have small question: I try to run unit tests on newton, and it raises me an error described here: https://bugs.launchpad.net/nova/+bug/1630420 . My question is: should I cherry-pick patch: https://review.openstack.org/#/c/383524/ to newton or add genisoimage to bindeps? | |
| 09:55:18 | openstack | Launchpad bug 1630420 in OpenStack Compute (nova) "config_drive unit tests (libvirt driver) aren't mocking genisoimage" [Low,Fix released] - Assigned to Diana Clarke (diana-clarke) | |
| 09:58:09 | openstackgerrit | Daniel Pawlik proposed openstack/nova stable/newton: Patch mkisofs calls https://review.openstack.org/529284 | |
| 10:01:48 | ebbex | mdbooth: I've created an instance with swap, where both root and swap are on rbd, yet there's a big swap file under nova/instances/_base on the compute-node, and according to the nova-compute logs it get's touched about once every minute (which would prevent _remove_old_enough_file from acting upon it). Is any of this intended behaviour? | |
| 10:02:48 | mdbooth | It's intended behaviour that it exists, although it could be done much better. | |
| 10:03:13 | mdbooth | We initialise a swap device on the compute host, then copy it to rbd iirc | |
| 10:03:32 | mdbooth | It's in _base because that's the cache | |
| 10:03:48 | mdbooth | Caching swap and ephemeral disks is a bug imho | |
| 10:04:15 | mdbooth | Not sure why it's being touched every minute, though | |
| 10:04:19 | mdbooth | Any idea what's touching it? | |
| 10:07:23 | ebbex | 2017-12-20T10:06:22.143079+00:00 oscompute01 sudo: nova : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/opt/openstack/nova/venv/bin/nova-rootwrap /opt/openstack/nova/etc/nova/rootwrap.conf touch -c /opt/openstack/nova/instances/_base/swap_16384 | |
| 10:09:30 | mdbooth | ebbex: Ah... IIRC there's a periodic task which touches cache files of active instances to prevent them from being deleted. | |
| 10:09:37 | mdbooth | But of course that's irrelevant for rbd. | |
| 10:09:46 | mdbooth | Another reason that rbd should have its own cache... | |
| 10:10:15 | mdbooth | Also LVM. | |
| 10:10:19 | mdbooth | And flat | |
| 10:10:33 | mdbooth | Everything except qcow2, in fact | |
| 10:12:52 | ebbex | Super, cause the compute-node has about 120gb storage, while ceph has a couple TBs. And I can see this clog up the disk on compute-node after 5-6 created vms. | |
| 10:13:39 | ebbex | Any idea for a quick fix? Disable the periodic touching perhaps? | |
| 10:14:08 | mdbooth | ebbex: If you found the relevant periodic task you could disable it if images_type != qcow2 | |
| 10:14:23 | mdbooth | As long as you accompanied it with a sufficiently explanatory comment | |
| 10:15:41 | mdbooth | We should have refactored imagebackend first time we implemented a backend other than qcow2 | |
| 10:19:18 | mdbooth | Unfortunately getting sufficient review attention for large refactors is hard, which discourages doing them. | |
| 10:24:05 | ebbex | Yeah, I have yet to understand how everything hangs together, so can't really contribute much there yet. | |
| 10:32:17 | openstackgerrit | Ameed Ashour proposed openstack/nova master: change instance_system_metadata column type https://review.openstack.org/526900 | |
| 10:53:46 | openstackgerrit | Ameed Ashour proposed openstack/nova master: change instance_system_metadata column type https://review.openstack.org/526900 | |
| 11:00:18 | openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: doc: update supported drivers for cpu topology https://review.openstack.org/529294 | |
| 11:06:26 | lyarwood | mdbooth: FYI I'm dropping the code to handle rbd volumes from the native LUKS change | |
| 11:07:11 | lyarwood | mdbooth: turns out that c-vol will always attempt to attach the os-brick encryptors and thus fail for rbd when creating a volume from an image. | |
| 11:11:12 | openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: doc: update supported drivers for cpu topology https://review.openstack.org/529294 | |
| 12:15:30 | jianghuaw_ | bauzas, jaypipes: Hi you may be interested on the new *test-vgpu* CI job added in "Citrix XenServer CI": https://review.openstack.org/#/c/516899/ | |
| 12:16:54 | jianghuaw_ | basically it's monitoring changes on some files in openstack/nova and openstack/os-xenapi and run vGPU test in devstack env. | |
| 12:18:57 | jianghuaw_ | bauzas, do you have time to review the above patch sometime today? Once it's merged, we can make the above test-vgpu CI job to be a formal test for XenAPI. | |
| 12:22:54 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Refactor encryptor attach and detach calls https://review.openstack.org/460243 | |
| 12:22:54 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Introduce disk encryption config classes https://review.openstack.org/464008 | |
| 12:22:55 | openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP libvirt: QEMU native LUKS decryption for volumes https://review.openstack.org/523958 | |
| 12:39:45 | jaypipes | jianghuaw_: excellent. :) | |
| 12:43:15 | cdent | jaypipes: can you confirm that gerrit is being a big sow in the wallow down back the holler? | |
| 12:47:18 | jaypipes | cdent: confirmed. | |
| 12:47:30 | jaypipes | cdent: it doesn't just hate you. | |
| 12:48:46 | cdent | thanks | |
| 13:09:30 | mnaser | and i think someone just emailed the ml about gerrit issues ^ | |
| 13:20:29 | jaypipes | gibi: https://review.openstack.org/#/c/528591/ is ready to go, IMHO. | |
| 13:23:35 | jaypipes | jianghuaw_: I'm not seeing in the test-vgpu logs where a flavor is being modified to include a request for resources:VGPU=1? | |
| 13:24:03 | jaypipes | jianghuaw_: erp, never mind. found it :) | |
| 13:24:14 | jaypipes | | flavor:extra_specs | {"resources:VGPU": "1"} | |
| 13:25:58 | jianghuaw_ | jaypipes, yeah. it's set at here "+ ./test_vgpu.sh:main:175 : nova flavor-key 1 set resources:VGPU=1" | |
| 13:26:23 | jaypipes | jianghuaw_: yep, found it eventually :) | |
| 13:26:33 | jianghuaw_ | :-) | |
| 13:27:13 | jaypipes | jianghuaw_: I'm wondering... is there any way to functionally test that the VM (once started) has the vGPU resources properly allocated from the hypervisor? | |
| 13:28:29 | jianghuaw_ | the test script will query xapi to verify if the VGPU is created and attached to the VM. | |
| 13:30:00 | jianghuaw_ | jaypipes, + ./test_vgpu.sh:main:214 : result='other-config (MRW): nova-instance-uuid: 64d20778-055a-4b58-a615-87a467caf7a2' | |
| 13:30:41 | jaypipes | jianghuaw_: excellent! | |
| 13:30:55 | jaypipes | jianghuaw_: nice work on this. :) | |
| 13:31:11 | jianghuaw_ | jaypipes, :-) thanks. | |
| 13:33:42 | jianghuaw_ | jaypipes, what we need is to get another core review on the patch. Hope bauzas will have time to review it. I really don't want to leave it to the new year:-( | |
| 13:34:04 | jianghuaw_ | I mean this patch: https://review.openstack.org/#/c/516899/ | |
| 13:34:34 | jaypipes | gibi: ^^ | |
| 13:35:07 | jianghuaw_ | thanks. | |
| 13:35:43 | gibi | jaypipes: looking... | |
| 13:36:03 | gibi | jaypipes: I was about to +2 https://review.openstack.org/#/c/528591/ when gerrit disappeared | |
| 13:37:13 | openstackgerrit | Jay Pipes proposed openstack/nova master: Remove server group sched filter support caching https://review.openstack.org/529200 | |
| 13:37:13 | openstackgerrit | Jay Pipes proposed openstack/nova master: WIP Support aggregate affinity filters https://review.openstack.org/529201 | |
| 13:37:59 | maciejjozefczyk | Guys, maybe you know if there is similiar thing implemented like https://blueprints.launchpad.net/nova/+spec/boot-order-for-instance | |
| 13:38:09 | maciejjozefczyk | Boot order for instance | |
| 13:38:29 | maciejjozefczyk | Or this idea died in Kilo? | |
| 13:38:33 | jianghuaw_ | gibi, Thanks. Gerrit service was restarted to resolve the slowness issue. it's recovered now. | |
| 13:43:07 | gibi | bauzas: is there a chance that you can check https://review.openstack.org/#/c/516899/ today as you already familiar with that patch? If not then I can dig | |
| 13:53:26 | mriedem1 | bauzas: can you take a look at https://bugs.launchpad.net/nova/+bug/1739323 ? | |
| 13:53:27 | openstack | Launchpad bug 1739323 in OpenStack Compute (nova) pike "KeyError in host_manager for _get_host_states" [High,Confirmed] | |
| 13:53:48 | mriedem1 | i can't remember why we changed from an iterator there in ocata, but you must have had a reason | |
| 13:58:48 | jaypipes | bauzas, kashyap, mriedem: regarding https://review.openstack.org/#/c/529294/ ... do I remember some patch recently that disabled either CPU pinning or NUMA topology (or both?) for any hypervisor other than KVM? I have this strange feeling that we recently restricted NUMA/CPU pinning to just KVM... | |
| 13:59:13 | gibi | mriedem: I left some suggestion in https://review.openstack.org/#/c/529194/ | |
| 13:59:26 | gibi | mriedem: if you want to discuss then i'm still here for couple of hours today | |
| 13:59:39 | kashyap | jaypipes: Hey, just going to get a piece of bread for lunch | |
| 13:59:52 | mriedem | jaypipes: yes i can dig up the link | |
| 14:00:01 | jaypipes | kashyap: I think you can probably afford more than just a piece of bread for lunch. | |
| 14:00:02 | kashyap | Should be back soon | |
| 14:00:06 | mriedem | jaypipes: https://review.openstack.org/#/c/465160/ | |
| 14:00:17 | jaypipes | kashyap: clearly, Holland has become too expensive. | |
| 14:00:19 | kashyap | jaypipes: Haha, I have the spreads at home; it's the Italian Ciabatta bread :P | |
| 14:00:26 | jaypipes | :) | |
| 14:00:31 | mriedem | "the spreads" | |
| 14:00:34 | mriedem | sounds like "the runs" | |
| 14:00:35 | jaypipes | ah, mriedem that's the ticket. thanks! | |
| 14:00:36 | kashyap | jaypipes: Gentleman, it's called Gent -- http://wikitravel.org/en/Gent | |
| 14:00:43 | kashyap | s/men/man/ | |
| 14:00:47 | jaypipes | heh | |
| 14:00:56 | kashyap | (It's in Flanders, Belgium :P) | |
| 14:01:58 | kashyap | It's late lunch; was debugging the Gate migration bug from hell | |
| 14:02:02 | kashyap | While I'm here; I'll update lyarwood too | |
| 14:02:10 | kashyap | lyarwood: On that migration bug we were talking about | |
| 14:02:19 | kashyap | I spent past 2 hours creating a reproducer -- | |
| 14:02:49 | kashyap | Migrate a guest with a `dd if=/dev/zero bs=1M | md5sum` workload in it, AND throttle the bandwitdh to 1MiB/s | |
| 14:02:59 | stephenfin | jaypipes: Why do you ask? I was waiting on sean-k-mooney to wade in on that again before doing anything, personally | |
| 14:02:59 | kashyap | And also set the clock back 20 minutes on the source | |
| 14:03:07 | kashyap | To see if those keepalives will kick in | |
| 14:03:19 | kashyap | s/keepalives/keep-alive timeout/ | |
| 14:03:54 | jaypipes | stephenfin: mostly I was just looking for a link to that other patch :) I couldn't remember what the commit summary was and couldn't find it on gerrit. | |
| 14:03:56 | kashyap | I couldn't reproduce it so far; the guest migrated slowly after a long while. So, the investigation to be conitnued (assuming it's still a problem for the stable gate) | |