| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-09-17 | |||
| 16:09:01 | mriedem | let's just do that in V | |
| 16:09:09 | mriedem | V for Volumessssss | |
| 16:09:24 | sean-k-mooney | well it woudl be a policy flag taht coudl default to false so not force them but im not sure how common it is to mix bfv and non bfv on the same host | |
| 16:09:57 | sean-k-mooney | i got the impression people tend to segration the bfv hosts form the rest but i could be wrong | |
| 16:11:10 | shilpasd | mriedem: sean-k-mooney: it will be great help, if you add review comments on patch so that can be addressed appropriately https://review.opendev.org/#/c/612626/ | |
| 16:12:25 | sean-k-mooney | mriedem_afk: oh thanks for reviewig the config drive stuff ill fix that up | |
| 16:14:21 | openstackgerrit | Merged openstack/nova master: Remove SchedulerReportClient from AggregateRequestFiltersTest https://review.opendev.org/682480 | |
| 16:14:29 | openstackgerrit | Merged openstack/nova stable/stein: Remove descriptions of nonexistent hacking rules https://review.opendev.org/679695 | |
| 16:21:02 | sean-k-mooney | efried: stephenfin do either of ye have the rc bug etherpad link | |
| 16:21:17 | efried | https://etherpad.openstack.org/p/nova-train-release-todo ? | |
| 16:21:25 | sean-k-mooney | yes that one | |
| 16:21:59 | sean-k-mooney | efried: https://review.opendev.org/#/c/669738/4 would be a regression in train if we dont land it in the rc | |
| 16:22:55 | efried | sean-k-mooney: ack | |
| 16:23:00 | efried | make sure mriedem_afk is aware | |
| 16:23:09 | sean-k-mooney | efried: he is | |
| 16:23:18 | sean-k-mooney | im going to address his review feedback now | |
| 16:23:33 | sean-k-mooney | we "fixed" https://bugs.launchpad.net/nova/+bug/1827492 | |
| 16:23:34 | openstack | Launchpad bug 1827492 in OpenStack Compute (nova) stein "vms failed to hard reboot and became error after set force_config_drive in compute nodes" [Low,In progress] - Assigned to Lee Yarwood (lyarwood) | |
| 16:23:53 | sean-k-mooney | and intoduced https://bugs.launchpad.net/nova/+bug/1835822 | |
| 16:23:54 | openstack | Launchpad bug 1835822 in OpenStack Compute (nova) "vms loose acess to config drive with CONF.force_config_drive=True after hard reboot" [Medium,In progress] - Assigned to sean mooney (sean-k-mooney) | |
| 16:43:53 | zzzeek | efried: how does that huge amount of change in the .zuul.yaml file relate to the extra config variable being set ? | |
| 16:44:33 | efried | zzzeek: It just restricts the run to only do unit tests, and gather logs from the mysql dir | |
| 16:44:47 | efried | zzzeek: if we end up doing this irl we'll revert that bit and just keep the test setup bit. | |
| 16:45:05 | zzzeek | efried: oh ok | |
| 16:45:16 | zzzeek | efried: well great, see if it fixes those errors | |
| 16:45:26 | efried | The extra zuul job (*collect) is to actually grab the logs so we can make sure this works. The removals is just so we don't waste the gate resources on the other test jobs. | |
| 16:45:41 | efried | zzzeek: thanks for the pointer! | |
| 16:51:37 | sean-k-mooney | efried: i just commented on the mail but could the mysql issues be related too the the rabbit mq issue we have in the api | |
| 16:52:01 | sean-k-mooney | e.g. could it be cause by eventlet and running multiple thread in the wsgi server | |
| 16:52:16 | sean-k-mooney | im not sure if this is happeng in the gate or not | |
| 16:52:21 | sean-k-mooney | if its in the gate then no | |
| 16:52:23 | efried | sean-k-mooney: I have no. freaking. clue. If you can solve this one, I will buy you cookies forever. | |
| 16:52:42 | efried | it's happening in the gate, yeah. | |
| 16:52:54 | sean-k-mooney | ok in the gate we hard code one thread i think | |
| 16:52:57 | sean-k-mooney | but ill check | |
| 16:52:59 | efried | wait | |
| 16:53:04 | efried | we're talking about different issues, sorry. | |
| 16:53:16 | efried | disregard cookie promise | |
| 16:53:25 | sean-k-mooney | is was talking about RE: [oslo][nova] Nova causes MySQL timeouts | |
| 16:53:38 | efried | cookie promise applies to http://bugs.launchpad.net/bugs/1823251 | |
| 16:53:39 | openstack | Launchpad bug 1823251 in OpenStack Compute (nova) "Spike in TestNovaMigrationsMySQL.test_walk_versions/test_innodb_tables failures since April 1 2019 on limestone-regionone" [High,Confirmed] | |
| 16:53:41 | sean-k-mooney | which i assme is what you and zzzeek were talking about? | |
| 16:53:51 | sean-k-mooney | ah ok | |
| 16:53:51 | efried | no, zzzeek and I were talking about ----^ | |
| 17:16:11 | efried | sean-k-mooney: oh, it looks like it may actually be the same thing zzzeek and I were talking about. | |
| 17:16:18 | efried | which kind of forked off of the test_walk_versions thing. | |
| 17:17:02 | sean-k-mooney | the mysql timeouts | |
| 17:17:28 | sean-k-mooney | so i was wondering if it could be related to eventlet in genreal | |
| 17:18:28 | sean-k-mooney | connection pooling might not play nicely with evnetly monkey patching when don in a wsgi envionment | |
| 17:18:49 | sean-k-mooney | or rather a mod_wsgi or uwsgi env | |
| 17:19:36 | sean-k-mooney | that said if it was an eventlet issue i dont think it would affect the test_walk_versions stuff | |
| 17:21:09 | efried | sean-k-mooney: it may be entirely unrelated to the gate timeout in those tests. But I'm seeing it in the mysqld logs for those tests (whether they pass or fail). So, can't hurt to get rid of them, nah? | |
| 17:23:56 | sean-k-mooney | well i thik fixing mysql issue in general is a good thing | |
| 17:24:12 | sean-k-mooney | just not sure as you said fi they are releated or not | |
| 17:25:29 | openstackgerrit | François Palin proposed openstack/nova master: Add retry when attachment_delete or terminate_connection fails https://review.opendev.org/669674 | |
| 17:51:23 | openstackgerrit | sean mooney proposed openstack/nova master: make config drives sticky bug 1835822 https://review.opendev.org/669738 | |
| 17:51:24 | openstack | bug 1835822 in OpenStack Compute (nova) "vms loose acess to config drive with CONF.force_config_drive=True after hard reboot" [Medium,In progress] https://launchpad.net/bugs/1835822 - Assigned to sean mooney (sean-k-mooney) | |
| 17:52:39 | mriedem_afk | sean-k-mooney: i tagged the bug so we won't forget about it https://bugs.launchpad.net/nova/+bugs?field.tag=train-rc-potential | |
| 17:52:54 | donnyd | I fixed my mysql issues by removing the ulimits in systemd for the service | |
| 17:52:56 | sean-k-mooney | mriedem_afk: i added it to the etherpad | |
| 17:53:12 | donnyd | probably a better answer out there... but i haven't ran into it again | |
| 17:53:17 | sean-k-mooney | mriedem_afk: i think i have done the things you asked for | |
| 17:54:46 | sean-k-mooney | ah i see you are using launchpad to track the rc potentail bugs ya that is better | |
| 17:55:12 | sean-k-mooney | im going to grab food but ill asress your comments in the devname patch when i get back | |
| 18:05:13 | mriedem | sean-k-mooney: https://review.opendev.org/#/c/669738/5 | |
| 18:37:47 | efried | donnyd: Were they the same mysql issues we've been discussing? | |
| 18:38:25 | efried | I sort of doubt it would be a problem to remove ulimits on CI nodes? | |
| 18:47:06 | donnyd | yea it was pretty close to the same issue | |
| 18:47:34 | donnyd | my load is different... but I just didn't want to hear any guff from mariadb | |
| 18:48:22 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: lxc: make use of filter python3 compatible https://review.opendev.org/676498 | |
| 18:48:31 | donnyd | woot woot LXC | |
| 18:56:16 | sean-k-mooney | donnyd: you will like this then https://review.opendev.org/#/c/667976/ | |
| 18:56:32 | donnyd | YESSSS sean-k-mooney | |
| 18:57:13 | sean-k-mooney | i think i said this becore but we need to fix the console and resizeing of the root file system bit it more or less works | |
| 18:57:14 | donnyd | I am unreasonably pumped for LXC support... I think it works great and can't wait | |
| 18:57:35 | sean-k-mooney | well we supported lxc years ago | |
| 18:57:42 | sean-k-mooney | its just has not been maintianed | |
| 18:57:51 | sean-k-mooney | but i would like to see it working again | |
| 18:58:08 | sean-k-mooney | *fully working again | |
| 18:58:45 | donnyd | I know... and it worked great... LXC can support a large number of workloads and its heavily underused by those who just don't know... I use it every day | |
| 18:59:22 | sean-k-mooney | my main issue with lxc was the lack of a good way to build images for it | |
| 18:59:37 | donnyd | i think distrobuilder is growing | |
| 18:59:40 | sean-k-mooney | i first start to use docker when it use the lxc backend | |
| 18:59:51 | donnyd | there a quite a few image types available in there now | |
| 18:59:54 | sean-k-mooney | and that was perfoect form me | |
| 19:19:05 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/queens: Drop source node allocations if finish_resize fails https://review.opendev.org/682722 | |
| 19:23:31 | donnyd | I see nothing wrong with LXC | |
| 19:25:07 | donnyd | and additionally there are solid examples of people using it for prod workloads...It also makes me happy to see some work going into lxc to make it work again... Many thanks | |
| 19:31:42 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Add functional recreate test for regression bug 1825537 https://review.opendev.org/682725 | |
| 19:31:43 | openstack | bug 1825537 in OpenStack Compute (nova) queens "finish_resize failures incorrectly revert allocations" [Medium,In progress] https://launchpad.net/bugs/1825537 - Assigned to Matt Riedemann (mriedem) | |
| 19:46:49 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Changing scheduler sync event from INFO to DEBUG https://review.opendev.org/669127 | |
| 19:49:39 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Refactor pre-live-migration work out of _do_live_migration https://review.opendev.org/641453 | |
| 20:20:31 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Refactor volume connection cleanup out of _post_live_migration https://review.opendev.org/682741 | |
| 20:36:36 | gyee | if the given vm is using pci passthrough, is resizing to the same host allowed or disallowed? | |
| 20:52:56 | artom | mriedem, left a question in https://review.opendev.org/#/c/682741/1 | |
| 20:53:15 | artom | Figured it's more polite to ask on IRC than -1 the thing (even if that's what I initially did) | |
| 21:02:11 | openstackgerrit | Akihiro Motoki proposed openstack/nova master: doc: Improve PDF document structure https://review.opendev.org/682746 | |
| 21:06:53 | mriedem | replied | |
| 21:07:46 | artom | Cheers | |
| 21:13:44 | mriedem | gyee: resize on the same host with pci passthrough should work but there is no integration testing for it, if there are limitations sean-k-mooney might be aware of them | |