| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-09-17 | |||
| 17:16:18 | efried | which kind of forked off of the test_walk_versions thing. | |
| 17:17:02 | sean-k-mooney | the mysql timeouts | |
| 17:17:28 | sean-k-mooney | so i was wondering if it could be related to eventlet in genreal | |
| 17:18:28 | sean-k-mooney | connection pooling might not play nicely with evnetly monkey patching when don in a wsgi envionment | |
| 17:18:49 | sean-k-mooney | or rather a mod_wsgi or uwsgi env | |
| 17:19:36 | sean-k-mooney | that said if it was an eventlet issue i dont think it would affect the test_walk_versions stuff | |
| 17:21:09 | efried | sean-k-mooney: it may be entirely unrelated to the gate timeout in those tests. But I'm seeing it in the mysqld logs for those tests (whether they pass or fail). So, can't hurt to get rid of them, nah? | |
| 17:23:56 | sean-k-mooney | well i thik fixing mysql issue in general is a good thing | |
| 17:24:12 | sean-k-mooney | just not sure as you said fi they are releated or not | |
| 17:25:29 | openstackgerrit | François Palin proposed openstack/nova master: Add retry when attachment_delete or terminate_connection fails https://review.opendev.org/669674 | |
| 17:51:23 | openstackgerrit | sean mooney proposed openstack/nova master: make config drives sticky bug 1835822 https://review.opendev.org/669738 | |
| 17:51:24 | openstack | bug 1835822 in OpenStack Compute (nova) "vms loose acess to config drive with CONF.force_config_drive=True after hard reboot" [Medium,In progress] https://launchpad.net/bugs/1835822 - Assigned to sean mooney (sean-k-mooney) | |
| 17:52:39 | mriedem_afk | sean-k-mooney: i tagged the bug so we won't forget about it https://bugs.launchpad.net/nova/+bugs?field.tag=train-rc-potential | |
| 17:52:54 | donnyd | I fixed my mysql issues by removing the ulimits in systemd for the service | |
| 17:52:56 | sean-k-mooney | mriedem_afk: i added it to the etherpad | |
| 17:53:12 | donnyd | probably a better answer out there... but i haven't ran into it again | |
| 17:53:17 | sean-k-mooney | mriedem_afk: i think i have done the things you asked for | |
| 17:54:46 | sean-k-mooney | ah i see you are using launchpad to track the rc potentail bugs ya that is better | |
| 17:55:12 | sean-k-mooney | im going to grab food but ill asress your comments in the devname patch when i get back | |
| 18:05:13 | mriedem | sean-k-mooney: https://review.opendev.org/#/c/669738/5 | |
| 18:37:47 | efried | donnyd: Were they the same mysql issues we've been discussing? | |
| 18:38:25 | efried | I sort of doubt it would be a problem to remove ulimits on CI nodes? | |
| 18:47:06 | donnyd | yea it was pretty close to the same issue | |
| 18:47:34 | donnyd | my load is different... but I just didn't want to hear any guff from mariadb | |
| 18:48:22 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: lxc: make use of filter python3 compatible https://review.opendev.org/676498 | |
| 18:48:31 | donnyd | woot woot LXC | |
| 18:56:16 | sean-k-mooney | donnyd: you will like this then https://review.opendev.org/#/c/667976/ | |
| 18:56:32 | donnyd | YESSSS sean-k-mooney | |
| 18:57:13 | sean-k-mooney | i think i said this becore but we need to fix the console and resizeing of the root file system bit it more or less works | |
| 18:57:14 | donnyd | I am unreasonably pumped for LXC support... I think it works great and can't wait | |
| 18:57:35 | sean-k-mooney | well we supported lxc years ago | |
| 18:57:42 | sean-k-mooney | its just has not been maintianed | |
| 18:57:51 | sean-k-mooney | but i would like to see it working again | |
| 18:58:08 | sean-k-mooney | *fully working again | |
| 18:58:45 | donnyd | I know... and it worked great... LXC can support a large number of workloads and its heavily underused by those who just don't know... I use it every day | |
| 18:59:22 | sean-k-mooney | my main issue with lxc was the lack of a good way to build images for it | |
| 18:59:37 | donnyd | i think distrobuilder is growing | |
| 18:59:40 | sean-k-mooney | i first start to use docker when it use the lxc backend | |
| 18:59:51 | donnyd | there a quite a few image types available in there now | |
| 18:59:54 | sean-k-mooney | and that was perfoect form me | |
| 19:19:05 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/queens: Drop source node allocations if finish_resize fails https://review.opendev.org/682722 | |
| 19:23:31 | donnyd | I see nothing wrong with LXC | |
| 19:25:07 | donnyd | and additionally there are solid examples of people using it for prod workloads...It also makes me happy to see some work going into lxc to make it work again... Many thanks | |
| 19:31:42 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Add functional recreate test for regression bug 1825537 https://review.opendev.org/682725 | |
| 19:31:43 | openstack | bug 1825537 in OpenStack Compute (nova) queens "finish_resize failures incorrectly revert allocations" [Medium,In progress] https://launchpad.net/bugs/1825537 - Assigned to Matt Riedemann (mriedem) | |
| 19:46:49 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Changing scheduler sync event from INFO to DEBUG https://review.opendev.org/669127 | |
| 19:49:39 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Refactor pre-live-migration work out of _do_live_migration https://review.opendev.org/641453 | |
| 20:20:31 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Refactor volume connection cleanup out of _post_live_migration https://review.opendev.org/682741 | |
| 20:36:36 | gyee | if the given vm is using pci passthrough, is resizing to the same host allowed or disallowed? | |
| 20:52:56 | artom | mriedem, left a question in https://review.opendev.org/#/c/682741/1 | |
| 20:53:15 | artom | Figured it's more polite to ask on IRC than -1 the thing (even if that's what I initially did) | |
| 21:02:11 | openstackgerrit | Akihiro Motoki proposed openstack/nova master: doc: Improve PDF document structure https://review.opendev.org/682746 | |
| 21:06:53 | mriedem | replied | |
| 21:07:46 | artom | Cheers | |
| 21:13:44 | mriedem | gyee: resize on the same host with pci passthrough should work but there is no integration testing for it, if there are limitations sean-k-mooney might be aware of them | |
| 21:14:32 | mriedem | i could see the code not being so smart and realizing that if pci devices in the flavor have not changed and we're resizing on the same host that we don't need to re-claim them | |
| 21:15:24 | gyee | mriedem, if the host has one PCI card, I am guessing we can't resize and it has already claimed | |
| 21:15:54 | gyee | but I am not too sure, have had time to dive into the code yet, was hoping for a quick yes or no answer :-) | |
| 21:16:18 | mriedem | i don't have that code memorized so i can't say off the top of my head | |
| 21:17:08 | mordred | mriedem: I'm very disappointed that you don't have it memorized | |
| 21:17:17 | mriedem | i'm not | |
| 21:17:18 | gyee | nstance_uuid=<?>,requests=[InstancePCIRequest])) host_passes /usr/lib/python2.7/site-packages/nova/scheduler/filters/pci_passthrough_filter.py:54" | |
| 21:17:18 | gyee | I see this in the logs "nova-scheduler.log:2019-09-17 19:41:34.269 31312 DEBUG nova.scheduler.filters.pci_passthrough_filter [req-9165ec70-88b2-4ce1-804b-ee28d7c1992c c6d51320a59041588a26477c64ec0973 d35583e194d845b79aaa2a6de2bef94b - default default] (d14-58-d0-52-4c-28, d14-58-d0-52-4c-28.i1.cloud.suse.de) ram: 114519MB disk: 1468416MB io_ops: 0 instances: 2 doesn't have the required PCI devices (InstancePCIRequests(i | |
| 21:17:25 | mriedem | gyee: the claims code in the resource tracker would be here for resize https://github.com/openstack/nova/blob/0984ebb9f1c3cc3c8ff352dd2a615a58da79833c/nova/compute/resource_tracker.py#L280 | |
| 21:17:52 | gyee | so I am guessing the device has already be claimed and therefore we can't share the same device, even on resizing | |
| 21:19:06 | mriedem | gyee: the PciPassthroughFilter might not be smart enough for resize to the same host yeah | |
| 21:19:20 | mriedem | in general the scheduler and its filters don't know about move operations | |
| 21:19:44 | mriedem | mordred: also, what a weird time for you to show up and heckle me :) | |
| 21:21:27 | mriedem | gyee: i wouldn't be surprised if you look at the starlingx nova repo that they have something for pci passthrough same host resize | |
| 21:22:00 | mriedem | cfriesen: ^ | |
| 21:23:02 | mordred | mriedem: it's 11:22pm where I am right now - that seems like perfect heckling time | |
| 21:23:04 | gyee | mriedem, thanks for the info. Time to read code I guess. :-) | |
| 21:24:23 | mriedem | mordred: so you're bored and lonely eh | |
| 21:24:52 | mordred | mriedem: well, there's no aurora borealis tonight | |
| 21:26:28 | cfriesen | mriedem: gyee: I think we're now aligned with upstream nova now, and that doesn't ring any bells for me. Checking with someone. | |
| 21:28:12 | mriedem | i would think being able to resize a pci passthrough instance on a single node edge site would be important for starlingx | |
| 21:28:37 | gyee | cfriesen, thanks. Our instance is running stable/rocky right now, if that makes any difference. | |
| 21:41:50 | cfriesen | gyee: mriedem: just checked and can't find anything like that. Since Stein at least Starlingx has been aligned with upstream nova. Sounds like a useful change though. | |
| 21:45:23 | gyee | cfriesen, thanks for the info! Yeah, I agree, resizing to the same host is a useful feature | |
| 22:27:07 | cfriesen | gyee: I did some more digging, turns out that we did do some special-casing for sriov and resize/cold migration for Pike, because cold migration with PCI passthrough was broken. For Stein, I think the only customization is numa-aware live migration. | |
| 22:45:12 | openstackgerrit | Merged openstack/nova stable/stein: Fix 'has_calls' method calls in unit tests https://review.opendev.org/677329 | |
| 23:22:00 | stephenfin | alex_xu: This failing PMEM patch is killing me. If I rebase the PCPU series and push it up, can you fast re-approve them. I see only one merge conflict with master | |
| 23:22:14 | alex_xu | stephenfin: yes, I can | |
| 23:22:57 | stephenfin | Thanks. Incoming | |
| 23:23:16 | alex_xu | \o/ | |
| 23:23:24 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Start reporting PCPU inventory to placement https://review.opendev.org/671793 | |
| 23:23:25 | openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Add 'InstanceNUMATopology.cpu_pinning' property https://review.opendev.org/680106 | |
| 23:23:25 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: '_get_(v|p)cpu_total' to '_get_(v|p)cpu_available' https://review.opendev.org/672693 | |
| 23:23:26 | openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Add 'NUMACell.pcpuset' field https://review.opendev.org/680108 | |
| 23:23:26 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Validate CPU config options against running instances https://review.opendev.org/680107 | |
| 23:23:27 | openstackgerrit | Stephen Finucane proposed openstack/nova master: hardware: Differentiate between shared and dedicated CPUs https://review.opendev.org/671800 | |
| 23:23:28 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Include both VCPU and PCPU in core quota count https://review.opendev.org/681374 | |
| 23:23:28 | openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Additional functional tests for pinned instances https://review.opendev.org/681750 | |
| 23:23:28 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Start reporting 'HW_CPU_HYPERTHREADING' trait https://review.opendev.org/675571 | |
| 23:23:29 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add support for translating CPU policy extra specs, image meta https://review.opendev.org/671801 | |
| 23:23:30 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add reshaper for PCPU https://review.opendev.org/674895 | |
| 23:23:30 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Mock 'libvirt_utils.file_open' properly https://review.opendev.org/681061 | |
| 23:23:30 | openstackgerrit | Stephen Finucane proposed openstack/nova master: fakelibvirt: Make 'Connection.getHostname' unique https://review.opendev.org/681060 | |
| 23:23:31 | openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Use sane indent https://review.opendev.org/680229 | |
| 23:23:43 | stephenfin | alex_xu: FYI the merge conflict was on 'libvirt: Start reporting 'HW_CPU_HYPERTHREADING' trait' and due to commit 36ee9c1913a | |