| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-09-05 | |||
| 14:11:38 | gibi | mriedem: thx | |
| 14:22:09 | openstackgerrit | Brin Zhang proposed openstack/nova master: Add user_id and project_id colume to Migration https://review.opendev.org/673990 | |
| 14:22:09 | openstackgerrit | Brin Zhang proposed openstack/nova master: Add operator user_id/project_id to the migrations https://review.opendev.org/679413 | |
| 14:22:10 | openstackgerrit | Brin Zhang proposed openstack/nova master: Filter migrations by user_id/project_id https://review.opendev.org/674243 | |
| 14:28:31 | brinzhang | mriedem: https://review.opendev.org/#/c/674243/21 has bean resolved the merge conflict. | |
| 14:51:55 | sean-k-mooney | donnyd: it looks like its not completing and going to node_failure fairly consitently. | |
| 14:52:10 | donnyd | hrm | |
| 14:52:23 | donnyd | I will try to give that flavor a spin and make sure it actually works | |
| 14:57:38 | donnyd | Well I didn't have any issues getting it to spin up, so I guess we dig a little deeper | |
| 14:57:54 | donnyd | sean-k-mooney: do you have a log file handy? | |
| 14:58:52 | sean-k-mooney | donnyd: node_failure means that nodepool was not able to provision a node for the zuul job | |
| 14:59:04 | sean-k-mooney | so it failed before starting the job | |
| 14:59:22 | sean-k-mooney | there would be a log entry in nodepool but not in what is uploaded | |
| 14:59:43 | donnyd | http://grafana.openstack.org/d/3Bwpi5SZk/nodepool-fortnebula?orgId=1 | |
| 14:59:57 | donnyd | I only see 3 failures from the last 6 hours | |
| 15:00:47 | sean-k-mooney | if you search for 679656 in https://zuul.opendev.org/t/openstack/status you wil see the latest one | |
| 15:01:16 | sean-k-mooney | this can happen if zuul is jsut at capastity for the provider | |
| 15:01:27 | sean-k-mooney | and it timed out waiting | |
| 15:01:47 | sean-k-mooney | i was seeing some node faiures before you update the lables too | |
| 15:02:08 | sean-k-mooney | so maybe its just worse due to the shared pool | |
| 15:02:58 | sean-k-mooney | it was about 1 in 2 or 1 in 3 that failed previously | |
| 15:03:03 | donnyd | sean-k-mooney: I am not sure where I am supposed to be looking | |
| 15:03:22 | donnyd | that is a pretty high failure rate no matter what | |
| 15:03:27 | donnyd | shouldn't be failing that often | |
| 15:04:06 | donnyd | maybe someone in infra can give us some insights. I just provisioned 10 without a single failure | |
| 15:04:07 | sean-k-mooney | well as i said it was a node failutre meaning noodpool could not provide a node to zuul to run the job on. that can happen due to hitting quota | |
| 15:04:20 | efried | kashyap: can we talk about https://blueprints.launchpad.net/nova/+spec/allow-secure-boot-for-qemu-kvm-guests ? | |
| 15:04:32 | kashyap | efried: Saddled with calls; but do ask away | |
| 15:04:41 | kashyap | efried: Actively working on it... | |
| 15:04:55 | sean-k-mooney | donnyd: FN is proably fine this could be on infras side. | |
| 15:04:57 | donnyd | sean-k-mooney: let me up the quota and see if that is helpful | |
| 15:04:58 | efried | kashyap: do you hope to land it in the next week or should I defer to ussuri? | |
| 15:05:27 | sean-k-mooney | i need to jump on a call but for now ill wait and check the jobs later when the load drops a bit | |
| 15:05:32 | donnyd | ok | |
| 15:05:54 | openstackgerrit | Kobi Samoray proposed openstack/nova master: Avoid fetching metadata when no subnets found https://review.opendev.org/679247 | |
| 15:06:16 | sean-k-mooney | we intend to run this job nightly at 6am with ocational manual triggering so it should not generally be a proablem | |
| 15:10:16 | donnyd | sean-k-mooney: infra said they were hitting quota issues, so i just oversubscribed it for instances by 30% and pulled the rest of the quota entirely | |
| 15:10:58 | donnyd | openstack.exceptions.HttpException: HttpException: 403: Client Error for url: https://openstack.fortnebula.com:13774/v2.1/e8fd161dc34c421a979a9e6421f823e9/servers, Quota exceeded for instances: Requested 1, but already used 70 of 70 instances | |
| 15:11:11 | donnyd | sean-k-mooney: that issue should be solved | |
| 15:11:37 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: migrate: Add bw min service level check of source compute https://review.opendev.org/680395 | |
| 15:11:38 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: resize: Add bw min service level check of source compute https://review.opendev.org/680396 | |
| 15:13:25 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Validate CPU config options against running instances https://review.opendev.org/680107 | |
| 15:13:25 | openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Use sane indent https://review.opendev.org/680229 | |
| 15:13:26 | openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Add 'NUMACell.pcpuset' field https://review.opendev.org/680108 | |
| 15:13:26 | openstackgerrit | Stephen Finucane proposed openstack/nova master: hardware: Differentiate between shared and dedicated CPUs https://review.opendev.org/671800 | |
| 15:13:27 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Start reporting 'HW_CPU_HYPERTHREADING' trait https://review.opendev.org/675571 | |
| 15:13:27 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add support for translating CPU policy extra specs, image meta https://review.opendev.org/671801 | |
| 15:13:28 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add reshaper for PCPU https://review.opendev.org/674895 | |
| 15:14:37 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: prepare func test env for moving servers with bandwidth https://review.opendev.org/655109 | |
| 15:14:53 | artom | dansmith, just want to make sure you saw my updates to the NUMA LM patch last night | |
| 15:15:16 | dansmith | artom: I did, but Imma wait until mriedem has a go through it | |
| 15:15:24 | artom | dansmith, ack | |
| 15:16:39 | efried | dustinc: I'd like to defer provider-config-file to ussuri given it has only had one reviewer (me) looking at it so far. Please let me know if you have reason to believe it can land in the next week. | |
| 15:20:30 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Func test for migrate server with ports having resource request https://review.opendev.org/655113 | |
| 15:20:30 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Make _rever_allocation nested allocation aware https://review.opendev.org/676138 | |
| 15:20:55 | kashyap | efried: Don't defer it yet, please. But I'll keep you posted. | |
| 15:21:04 | efried | kashyap: ack | |
| 15:21:32 | kashyap | efried: I'll also be using that exception from Adam's InvalidMachineType patch; so there's a small dep there | |
| 15:21:56 | efried | kashyap: unclear whether SEV will make it too fyi | |
| 15:22:20 | kashyap | efried: Hmm, understandable; review bandwidth, topic complexity, etc... | |
| 15:23:10 | kashyap | But to state the obvious, I don't want to rush anything, though. | |
| 15:23:46 | efried | stephenfin: how are you feeling about cpu-resources? | |
| 15:24:42 | stephenfin | efried: I'm happy with it, but I need someone to take the hit on https://review.opendev.org/#/c/671800/ I've simplified it as much as I can at this point, I think | |
| 15:24:52 | stephenfin | I've been bugging bauzas for that :) | |
| 15:25:02 | efried | stephenfin: what I mean is: do you reasonably expect it can land by next Thursday? | |
| 15:25:14 | stephenfin | Yeah, I think so | |
| 15:25:20 | efried | okay | |
| 15:25:27 | bauzas | efried: I feel it's a reasonable target if someone else but me does reviews as well | |
| 15:25:33 | bauzas | hint : could be you | |
| 15:25:45 | efried | bauzas: I've been doing what I can, but much of it is beyond my ken. | |
| 15:25:50 | bauzas | cool | |
| 15:26:27 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Support reverting migration / resize with bandwidth https://review.opendev.org/676140 | |
| 15:27:02 | stephenfin | efried: tbh, it's probably beyond most people that aren't called me and sean-k-mooney (and maybe cfriesen) | |
| 15:27:22 | stephenfin | so I think we're going to have to rely on the fact that there are lot of functional tests there, and ask me all the questions if necessary | |
| 15:27:35 | stephenfin | (and I mean, a _lot_ of functional tests) | |
| 15:28:04 | efried | Okay, sean-k-mooney if you've got room, it would be helpful if you could ack that stuff. | |
| 15:28:34 | efried | I think between bauzas, alex_xu, sean-k-mooney, and me, we should be able to scrape together the expertise. It's more a question of bandwidth. | |
| 15:29:24 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Func test for migrate re-schedule with bandwidth https://review.opendev.org/676972 | |
| 15:32:26 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Support migrating SRIOV port with bandwidth https://review.opendev.org/676980 | |
| 15:32:26 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Allow migrating server with port resource request https://review.opendev.org/671497 | |
| 15:35:15 | sean-k-mooney | efried: sorry was on a call | |
| 15:35:18 | sean-k-mooney | what was the context | |
| 15:35:53 | efried | sean-k-mooney: The cpu-resources series | |
| 15:36:05 | bauzas | sean-k-mooney: the point was whether we have reasonable amount of eyes for cpu-resources | |
| 15:36:07 | efried | sean-k-mooney: it would increase confidence if it had your ack | |
| 15:36:26 | efried | dansmith: regarding isolated vs forbidden, this is a Nova UX vs Placement API thing. | |
| 15:36:32 | sean-k-mooney | i can dedicate the next few days to testing it like i was toing with artoms stuff | |
| 15:36:40 | dansmith | efried: yeah I called that out | |
| 15:36:59 | efried | From the Nova perspective, we're isolating aggregates. Internally we're using placement's forbidden aggregates feature. | |
| 15:37:02 | sean-k-mooney | i think the numa migration is more or less in a good place so i can review the cpu serieis and start testing | |
| 15:37:28 | dansmith | efried: did you see my comment on the later patch? | |
| 15:38:00 | sean-k-mooney | efried: while i hate to do this to my own seires we could punt https://review.opendev.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/image-metadata-prefiltering if required to help to U. its ready to go but the sev and cpu series conflict with it | |
| 15:38:20 | sean-k-mooney | so untill they are landed its going to keep gong into merge conflict | |
| 15:38:29 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Do not query allocations twice in finish_revert_resize https://review.opendev.org/678827 | |
| 15:39:22 | efried | sean-k-mooney: thanks, that's helpful | |
| 15:40:28 | sean-k-mooney | efried: stephen created https://etherpad.openstack.org/p/nova-cpu-resources as a set of test he wanted me to try. | |
| 15:40:54 | sean-k-mooney | i can start on them today/tomorrow but if anything else woudl help i could try that. | |
| 15:41:17 | sean-k-mooney | i can if need also try and create a temp ci job | |
| 15:41:24 | efried | dansmith: seeing it now. Sorry, I'm spread in a number of directions at the moment. | |
| 15:41:42 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Allow resizing server with port resource request https://review.opendev.org/679019 | |