| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-08-30 | |||
| 08:55:59 | lyarwood | bauzas: https://review.openstack.org/#/q/topic:bug/1787606 - would you mind sticking that on your review queue if you have time today or tomorrow? | |
| 08:56:12 | dpawlik | Hello, which rules from policy.json are used by placement api? I have an another role for admin and its raising me an error on compute host that "ailed to retrieve resource provider tree from placement API for UUID 573c492d-7387-4a3a-b21c-20ce531eb483. Got 403: {"errors": [{"status": 403, "request_id": "req-50ecb39c-8bea-4f52-85f2-7b92db9ae9cf", "detail": "Access was denied to this resource.\n\n admin required ", "title" | |
| 08:56:13 | dpawlik | : "Forbidden"}]}. | |
| 08:56:18 | lyarwood | after rm -rf'ing all of your emails obviously | |
| 08:56:19 | bauzas | lyarwood: today could be difficult but I can try | |
| 08:56:25 | lyarwood | bauzas: yeah no rush | |
| 08:56:38 | bauzas | lyarwood: rm -rf is one option, the other involves reading | |
| 08:56:43 | gibi | sean-k-mooney, melwitt: fyi, we are planning to show some demo on the PTG about the state of the bandwidth work http://lists.openstack.org/pipermail/openstack-dev/2018-August/134015.html | |
| 08:56:46 | lyarwood | bauzas: both are WIP I just wanted to get some input on the bug and potential fix | |
| 08:56:48 | dpawlik | problem is that I changed in policy.json file that admin_api is role:my_role | |
| 08:56:51 | bauzas | lyarwood: I'm not sure which one is the best | |
| 08:56:54 | dpawlik | but its not working on queens | |
| 08:57:02 | lyarwood | bauzas: ^_^ rm -rf every time | |
| 08:57:16 | lyarwood | bauzas: if it's that important people will send another email | |
| 08:57:24 | bauzas | and wait for others yelling at you that you haven't replied them ? That could work | |
| 09:01:10 | kashyap | Yeah, "selective reading" is the way to taming e-mail. | |
| 09:01:53 | kashyap | The more meandering a message, the faster it gets marked as read. | |
| 09:15:58 | openstackgerrit | huanhongda proposed openstack/nova master: Fix instance delete stuck in deleting task_state https://review.openstack.org/598084 | |
| 09:51:50 | stephenfin | bauzas: Welcome back. Here's a nice, easy (low priority) review to get you started https://review.openstack.org/#/c/530924/ | |
| 09:52:47 | sean-k-mooney | gibi: oh cool i look forward to seeing it | |
| 10:04:01 | Dinesh_Bhor | sean-k-mooney: Hi, May I have your 5 min? | |
| 10:04:22 | sean-k-mooney | Dinesh_Bhor: hi am sure what can i help with? | |
| 10:06:29 | Dinesh_Bhor | sean-k-mooney: we have a quota for instances currently. Actually we have some hypervisors which are specifically dedicated for "Rich VM" so we want to have separate quota for normal VM's and Rich VM's per project. | |
| 10:07:04 | Dinesh_Bhor | sean-k-mooney: do you think its a good idea to submit a blueprint for this? Or can it be managed somehow with metadata's? | |
| 10:07:37 | Dinesh_Bhor | Or something else may be | |
| 10:08:01 | sean-k-mooney | Dinesh_Bhor: this is similar to premptiable instances. in that case we also had the idea of two classes of instance that may have different sla's | |
| 10:09:10 | sean-k-mooney | Dinesh_Bhor: so if you were to submit a blueprint for this it may be worth trying to adress that usecase also so its a more general solution. | |
| 10:09:20 | sean-k-mooney | Dinesh_Bhor: that said the rich vms | |
| 10:09:42 | sean-k-mooney | is there richness determined by the host they land on or is it an aspect of the flavor | |
| 10:10:07 | sean-k-mooney | e.g. certin flavor you would like to have a seperate quota for. | |
| 10:11:27 | sean-k-mooney | Dinesh_Bhor: submitting a blueprint that clearly states what your usecase is and the constraits is never a bad idea but that highlevel usecase is what we would like it to capture rather then i think it should be done X way | |
| 10:12:36 | Dinesh_Bhor | sean-k-mooney: we have dedicated rich flavors which land on predefined hypervisors. Its like giving bare metal kind of experience with Rich-VMs | |
| 10:14:23 | Dinesh_Bhor | sean-k-mooney: hosts are high in memory, cpus to give bare metal kind of performance. So we want Rich-Flavors to be deployed on those hosts and want to manage quota for them. | |
| 10:14:41 | Dinesh_Bhor | sean-k-mooney: similar to normal vms per project | |
| 10:14:42 | sean-k-mooney | Dinesh_Bhor: right in that case rather then an instance quota the abblity to set a flavor qouta for a speific flavor would be enough to suit your partcalar usecase correct? | |
| 10:15:17 | sean-k-mooney | you could then use flavor to host affinity to ensure those flavor got shceduled to the correct hosts | |
| 10:15:33 | Dinesh_Bhor | A project can have normal as well as rich vms | |
| 10:16:03 | Dinesh_Bhor | sean-k-mooney: yes, we are managing that with AggregateInstanceExtraSpecFilter | |
| 10:16:26 | Dinesh_Bhor | sean-k-mooney: we are on Mitaka so can not use placement. | |
| 10:16:45 | sean-k-mooney | Dinesh_Bhor: yes when i said flavor quota i ment can have 10 instances of flavor X and any number of flavors without a qota provided they dont exceed other qoutas such as cpus | |
| 10:18:38 | sean-k-mooney | Dinesh_Bhor: the only non invasive way i can consive of to achive this i mitaka would be to write a schduler filter that would check you usage of the rich vms and fail all hosts if you exceed a qouta | |
| 10:21:38 | Dinesh_Bhor | sean-k-mooney: yes, but for that I thought of storing "no of rich-vms allowed" metadata in host-aggregate for per project but that will again degrade the performance of scheduler I think If we have 1000+ projects. | |
| 10:22:06 | Dinesh_Bhor | sean-k-mooney: okay, let me check the flavor quota thing. | |
| 10:22:11 | Dinesh_Bhor | first | |
| 10:22:49 | sean-k-mooney | Dinesh_Bhor: yes it would degrade the performance. | |
| 10:23:54 | sean-k-mooney | Dinesh_Bhor: i would speak to melwitt about this also. i belive she will be looking at how we can start to use the new keystone limmits api in nova going forward. that might help with this usecase | |
| 10:24:18 | Dinesh_Bhor | sean-k-mooney: yes, thank you so much | |
| 11:00:28 | openstackgerrit | Rong Han proposed openstack/nova master: Reset global variable after unit test is completed. https://review.openstack.org/598088 | |
| 11:17:43 | openstackgerrit | Merged openstack/nova master: Report client: Real get_allocs_for_consumer https://review.openstack.org/584599 | |
| 11:36:55 | openstackgerrit | Rong Han proposed openstack/nova master: Reset global variable after unit test is completed. https://review.openstack.org/598088 | |
| 12:04:27 | bauzas | stephenfin: https://review.openstack.org/#/c/530924/6 needs a new oslo.config version, right? | |
| 12:07:57 | bauzas | stephenfin: nevermind, I can see it's from oslo.config 5.2.0 | |
| 12:07:58 | bauzas | https://docs.openstack.org/releasenotes/oslo.config/queens.html#relnotes-5-2-0-stable-queens | |
| 12:08:18 | bauzas | and nova uses the latest https://github.com/openstack/nova/blob/master/requirements.txt#L40 | |
| 12:12:41 | zigo | bauzas: Don't worry, Rocky doesn't even build with 5.2.0 anywway ... :) | |
| 12:12:50 | zigo | Once more, requirements are just plain wrong. | |
| 12:12:56 | zigo | As usual, I'd say... | |
| 12:15:13 | zigo | Some packages are using oslo_config.sphinxconfiggen which isn't available in oslo.config 5.2.0. | |
| 12:15:28 | zigo | networking-bagpipe for example. | |
| 12:16:47 | bauzas | stephenfin: heh, I found you a new Friday nick <finucannitbacktick> :p | |
| 12:17:31 | bauzas | zigo: that's a project related issue | |
| 12:17:51 | zigo | bauzas: Ok, you need another example ... | |
| 12:17:55 | bauzas | zigo: the reviewers should look at the needed oslo version when they merge a new feature | |
| 12:18:07 | zigo | bauzas: Neutron has 1700+ unit test failures with current lower bounds ! :) | |
| 12:41:02 | openstackgerrit | Jay Pipes proposed openstack/os-traits master: clean up CUDA traits https://review.openstack.org/597170 | |
| 12:48:32 | stephenfin | bauzas: :D | |
| 12:48:51 | stephenfin | bauzas: People will eventually learn :) | |
| 12:49:08 | stephenfin | zigo: What do you mean, it doesn't build? | |
| 12:49:50 | stephenfin | bauzas: Also, we have oslo.config 6.1.0 in lower-constraints so I think we're all good there | |
| 12:50:02 | bauzas | yep | |
| 12:56:15 | mriedem | cdent: can you remind someone internally to re-propose the spec for this for stein? https://blueprints.launchpad.net/nova/+spec/vmware-live-migration | |
| 12:56:56 | mriedem | or you can if you want, it's just procedural | |
| 12:57:12 | mriedem | https://specs.openstack.org/openstack/nova-specs/readme.html#previously-approved-specifications | |
| 12:58:26 | cdent | mriedem: yeah, I think rado's gonna take care of it, he was on pto for a while | |
| 12:59:45 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Restart scheduler in TestNovaManagePlacementHealAllocations https://review.openstack.org/598152 | |
| 12:59:48 | brinzhang | mriedem: Could you please review this specs, https://review.openstack.org/#/c/591976/? If you have time :) | |
| 13:00:31 | mriedem | sure | |
| 13:00:42 | brinzhang | thanks ^^ | |
| 13:05:50 | cdent | mriedem: is there any new insight on the allocation thing yet, or has that not come round on the radar yet? | |
| 13:08:00 | mriedem | i think i got a failing xen ci result last night after i knocked off for the day, was going to investigate this morning | |
| 13:08:04 | mriedem | 1 sip into coffee ... | |
| 13:09:18 | openstackgerrit | Chen proposed openstack/nova master: Fix filter server list by multiple vm or task states https://review.openstack.org/598154 | |
| 13:16:04 | mriedem | alex_xu: ^ looks like a behavior change | |
| 13:17:43 | mriedem | cdent: oh i remember now, i had to re-run the xen ci patch last night b/c it wasn't picking up my dependency in depends-on: <url> form b/c it's still using zuul v2 | |
| 13:18:08 | cdent | mriedem: fun! | |
| 13:18:37 | mriedem | the logs are there in the latest failed run though | |
| 13:18:40 | mriedem | http://dd6b71949550285df7dc-dda4e480e005aaa13ec303551d2d8155.r49.cf1.rackcdn.com/13/597613/2/check/dsvm-tempest-neutron-network/cc81140/logs/screen-n-cpu.txt.gz | |
| 13:18:48 | mriedem | looking at req-99d9d496-6720-4837-a2ee-560605fd1afe | |
| 13:18:56 | mriedem | naichuans: efried: ^ | |
| 13:19:28 | mriedem | Aug 29 16:56:06.926641 dsvm-devstack-citrix-mia-nodepool-1379368 nova-compute[24436]: INFO nova.compute.resource_tracker [None req-99d9d496-6720-4837-a2ee-560605fd1afe None None] Using cpu_allocation_ratio 16.0 for node: 2f5a2e04-1b61-4437-ab6e-8dbbf797dc07 | |
| 13:19:41 | mriedem | Aug 29 16:56:06.926926 dsvm-devstack-citrix-mia-nodepool-1379368 nova-compute[24436]: INFO nova.compute.resource_tracker [None req-99d9d496-6720-4837-a2ee-560605fd1afe None None] RT: Sending compute node inventory changes back toplacement for node: 2f5a2e04-1b61-4437-ab6e-8dbbf797dc07 | |
| 13:19:55 | mriedem | Aug 29 16:56:06.965945 dsvm-devstack-citrix-mia-nodepool-1379368 nova-compute[24436]: INFO nova.compute.provider_tree [None req-99d9d496-6720-4837-a2ee-560605fd1afe None None] Inventory has not changed in ProviderTree for provider: 2f5a2e04-1b61-4437-ab6e-8dbbf797dc07 | |
| 13:19:55 | mriedem | WAH WAH | |
| 13:20:23 | mriedem | hmm, but then it says it does update inventory | |
| 13:20:24 | mriedem | n_unit': 1, 'max_unit': 12795}, 'DISK_GB': {'allocation_ratio': 1.0, 'total': 47, 'reserved': 0, 'step_size': 1, 'min_unit': 1, 'max_unit': 47}} | |
| 13:20:24 | mriedem | ia-nodepool-1379368 nova-compute[24436]: INFO nova.compute.provider_tree [None req-99d9d496-6720-4837-a2ee-560605fd1afe None None] Updating inventory in ProviderTree for provider 2f5a2e04-1b61-4437-ab6e-8dbbf797dc07 with inventory: {'VCPU': {'allocation_ratio': 16.0, 'total': 8, 'reserved': 0, 'step_size': 1, 'min_unit': 1, 'max_unit': 8}, 'MEMORY_MB': {'allocation_ratio': 1.5, 'total': 12795, 'reserved': 512, 'step_size': 1, | |
| 13:20:24 | mriedem | Aug 29 16:56:07.057208 dsvm-devstack-citrix-mia-nodepool-1379368 nova-compute[24436]: DEBUG nova.compute.provider_tree [None req-99d9d496-6720-4837-a2ee-560605fd1afe None None] Updating resource provider 2f5a2e04-1b61-4437-ab6e-8dbbf797dc07 generation from 0 to 1 during operation: update_inventory {{(pid=24436) _update_generation /opt/stack/new/nova/nova/compute/provider_tree.py:161}} Aug 29 16:56:07.057499 dsvm-devstack-citr | |
| 13:20:48 | mriedem | there the allocation ratios are all correct | |
| 13:21:08 | mriedem | 12, 'step_size': 1, 'min_unit': 1, 'max_unit': 12795}, 'DISK_GB': {'allocation_ratio': 1.0, 'total': 47, 'reserved': 0, 'step_size': 1, 'min_unit': 1, 'max_unit': 47}} {{(pid=24436) _update_inventory_attempt /opt/stack/new/nova/nova/scheduler/client/report.py:965}} | |
| 13:21:08 | mriedem | Aug 29 16:56:07.058213 dsvm-devstack-citrix-mia-nodepool-1379368 nova-compute[24436]: DEBUG nova.scheduler.client.report [None req-99d9d496-6720-4837-a2ee-560605fd1afe None None] Updated inventory for 2f5a2e04-1b61-4437-ab6e-8dbbf797dc07 at generation 1: {'VCPU': {'allocation_ratio': 16.0, 'total': 8, 'reserved': 0, 'step_size': 1, 'min_unit': 1, 'max_unit': 8}, 'MEMORY_MB': {'allocation_ratio': 1.5, 'total': 12795, 'reserved | |