| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-07-14 | |||
| 10:39:01 | gibi | stephenfin: OK, thanks. I have access to an SRIOV capable machine so I can try different libvirt version too | |
| 10:50:22 | gibi | I think I will be able to post some WIP code this week | |
| 11:20:19 | stephenfin | huaqiang: Okay, I prefer https://review.opendev.org/#/c/740933/ | |
| 11:21:09 | stephenfin | May be wise to get someone that knows RPC better than I do (dansmith or gibi, perhaps?) to weigh in | |
| 11:25:53 | stephenfin | lyarwood: I had my message asking you to look at the vTPM work already typed out before I checked Gerrit 😅 Thanks for the reviews | |
| 11:25:53 | stephenfin | lyarwood: I had my message asking you to look at the vTPM work already typed out before I checked Gerrit 😅 Thanks for the reviews | |
| 11:32:46 | gibi | stephenfin, huaqiang: looking at the alternatives I think I would go with the check in the api service | |
| 11:33:33 | stephenfin | gibi: so you mean prefer https://review.opendev.org/#/c/740933/ over https://review.opendev.org/716271 ? | |
| 11:34:05 | gibi | yes I prefer https://review.opendev.org/#/c/740933/ | |
| 11:37:08 | gibi | it is a bit more explicit what operations we block if the check is in the API | |
| 11:38:08 | gibi | and also understanding the code is easier for me if the check is not burried down in a long call path | |
| 11:39:50 | openstackgerrit | Merged openstack/nova stable/stein: Check cherry-pick hashes in pep8 tox target https://review.opendev.org/737922 | |
| 11:41:49 | openstackgerrit | Jinsheng Zhang proposed openstack/nova-specs master: Add nova-support-multiple-boot-volume-with-boot-order-selection spec https://review.opendev.org/737897 | |
| 11:45:35 | huaqiang | stephenfin and gibi, thanks for comments! | |
| 11:59:45 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Use better variable names for '_create_guest' https://review.opendev.org/740945 | |
| 11:59:45 | openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP: Attempt to restore some sanity to snapshot https://review.opendev.org/740336 | |
| 12:09:49 | stephenfin | lyarwood: replied at https://review.opendev.org/#/c/739209/4 | |
| 12:21:01 | lyarwood | stephenfin: just back sorry | |
| 12:21:04 | lyarwood | stephenfin: ack thanks | |
| 12:55:07 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/train: Silence amqp heartbeat warning https://review.opendev.org/728057 | |
| 12:57:23 | lyarwood | melwitt: https://review.opendev.org/#/c/732708/ - would you mind hitting this once you're online today, the stable/stein change has merged. | |
| 12:57:40 | lyarwood | and the func test below it please :) | |
| 14:06:49 | mnaser | so neutron makes 0 use of wrap_db_retry and has ssome convluted 4 year old retry mechanism which tries to avoid nested use of wrap_db_retry (well, if it detects it, it removes it, sounds like a workaround to me) | |
| 14:06:58 | mnaser | does nova have some policy on what is retried and what isn't? | |
| 14:07:44 | dansmith | mnaser: we decorate the methods we want to retry, IIRC | |
| 14:08:03 | mnaser | dansmith: ok so there isn't a "if X then decorate" policy that i can try and adopt for neutron's db stuff | |
| 14:08:19 | dansmith | mnaser: I think we also don't have a lot of nesting of things that might be called from the top, or called from within, so no nesting | |
| 14:08:29 | dansmith | mnaser: yes, the policy is "if we get a bug for a thing, decorate it" :) | |
| 14:08:55 | mnaser | ok, so the current what-seems-to-be-neutron policy is... decorate everything with a decorator that removes an extra one if it finds one to avoid nested retries | |
| 14:09:05 | mnaser | so this is going to be very fun to unwind | |
| 14:56:32 | dansmith | gmann: tres woot: https://review.opendev.org/#/c/734184/ | |
| 14:57:58 | gmann | dansmith: nice, checking | |
| 14:58:26 | dansmith | (moved to -qa) | |
| 16:09:50 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: [WIP] Support SRIOV interface attach and detach https://review.opendev.org/740995 | |
| 16:10:44 | gibi | stephenfin, sean-k-mooney: this might be intresting for you ^^ | |
| 16:12:10 | openstackgerrit | Dan Smith proposed openstack/nova master: Make our ceph job test with glance in multistore mode https://review.opendev.org/734184 | |
| 16:42:31 | openstackgerrit | Huaqiang Wang proposed openstack/nova master: objects: Introduce the 'CPUAllocationPolicy.MIXED' enum https://review.opendev.org/716267 | |
| 16:42:31 | openstackgerrit | Huaqiang Wang proposed openstack/nova master: notifications: add the pcpuset info to instance numa cell payload https://review.opendev.org/728480 | |
| 16:42:32 | openstackgerrit | Huaqiang Wang proposed openstack/nova master: libvirt: apply mixed instance CPU policy https://review.opendev.org/728481 | |
| 16:42:32 | openstackgerrit | Huaqiang Wang proposed openstack/nova master: hardware: refactor the code for CPU policy sanity check https://review.opendev.org/740925 | |
| 17:08:01 | openstackgerrit | Huaqiang Wang proposed openstack/nova master: Calculate the CPU usage for mixed instance https://review.opendev.org/716269 | |
| 17:08:02 | openstackgerrit | Huaqiang Wang proposed openstack/nova master: scheduler: mixed instance CPU resource translating https://review.opendev.org/728482 | |
| 17:25:55 | openstackgerrit | Huaqiang Wang proposed openstack/nova master: Validate CPU pinning configuration for mixed instance https://review.opendev.org/716268 | |
| 17:25:56 | openstackgerrit | Huaqiang Wang proposed openstack/nova master: scheduler: mixed instance CPU resource translating https://review.opendev.org/728482 | |
| 17:25:56 | openstackgerrit | Huaqiang Wang proposed openstack/nova master: Calculate the CPU usage for mixed instance https://review.opendev.org/716269 | |
| 18:53:20 | openstackgerrit | Merged openstack/nova master: utils: Move 'get_bdm_image_metadata' to nova.block_device https://review.opendev.org/739996 | |
| 20:30:54 | dansmith | sean-k-mooney: replied. thanks for reviewing and for your initial help in that of course | |
| 20:33:00 | sean-k-mooney | no worries | |
| 20:36:34 | sean-k-mooney | dansmith: ya makes sense to me | |
| 20:36:55 | sean-k-mooney | im glad you finally got it working by the way. | |
| 20:37:06 | sean-k-mooney | its always nice when the ci verifys something like this for us | |
| 20:37:47 | sean-k-mooney | the glance folks might even want to take a look for there own testing | |
| 21:36:25 | openstack | bug 1550919 in OpenStack Compute (nova) rocky "[Libvirt]Evacuate fail may cause disk image be deleted" [Medium,In progress] https://launchpad.net/bugs/1550919 - Assigned to Lee Yarwood (lyarwood) | |
| 21:36:25 | openstackgerrit | Merged openstack/nova stable/rocky: Add functional test for bug 1550919 https://review.opendev.org/732707 | |
| 21:54:44 | dansmith | sean-k-mooney: yeah, they'll have a lot more surface they can test once my two devstack patches land.. I hope they will make use of that, but.. who knows | |
| 22:24:03 | sean-k-mooney | dansmith: well if we are testing it its better then nothing | |
| 23:20:46 | openstackgerrit | Merged openstack/nova stable/rocky: libvirt: Don't delete disks on shared storage during evacuate https://review.opendev.org/732708 | |
| 23:37:53 | mnaser | melwitt: i know you've worked some with the quota code. does nova have any locking on quota usages? neutron currently has a row-level lock on a resource/tenant_id which is causing me worlds of pain on highly parallelized port creation | |
| 23:38:24 | mnaser | it pretty much holds a lock the whole time (with a select for update) while it calculates the usage and updates... | |
| 23:46:10 | gmann | dansmith: seems oslo policy not able to format the target data for copy_image - https://zuul.opendev.org/t/openstack/build/aa1a5cd62aaa4210b19a5877e77b87db/log/controller/logs/screen-g-api.txt#600 | |
| 23:47:21 | gmann | trying to reproduce this locally to see if oslo trying to change 'hw_rng_model' for some reason. | |
| 23:52:06 | gmann | mnaser: not sure about lock (melwitt can confirm if we do have that) but nova does recheck quota again just before the resource creation to avoid(or to minimize as much as possible) any wrong calculation or over-quota in parallel request cases. | |
| 23:52:45 | mnaser | gmann: ok so nova doesnt do any row level locking for quota usage -- also yeah, i remember that, it's even an option to recheck quota in config if i remember right | |
| 23:53:28 | gmann | * after resource creation. this config - https://github.com/openstack/nova/blob/7385adb2f526166bb174dab055d060a428d4d4ff/nova/conf/quota.py#L189 | |
| 23:56:04 | gmann | mnaser: afaik, no locks but melwitt or dansmith can confirm it. | |
| #openstack-nova - 2020-07-15 | |||
| 01:09:23 | melwitt | mnaser: I implemented a re-architecting of quotas in pike, which began counting quota usage on-the-fly by counting resources (this only does a read). prior to pike, we did do the select-for-update locking thing | |
| 02:26:53 | openstackgerrit | Jeffrey Zhang proposed openstack/nova master: Support inject-nmi action in watchdog https://review.opendev.org/741072 | |
| 03:18:04 | openstackgerrit | Merged openstack/nova master: doc: Correct typo https://review.opendev.org/740670 | |
| 03:32:41 | openstackgerrit | Keigo Noha proposed openstack/nova master: Change default num_retries for glance to 3 https://review.opendev.org/740389 | |
| 05:38:27 | openstackgerrit | Jeffrey Zhang proposed openstack/nova master: Support inject-nmi action in watchdog https://review.opendev.org/741072 | |
| 06:22:49 | openstackgerrit | Jie Li proposed openstack/nova-specs master: add volume backed server rebuild spec https://review.opendev.org/739349 | |
| 08:50:12 | openstackgerrit | Huaqiang Wang proposed openstack/nova master: hardware: Enable 'hw:cpu_dedicated_mask' for creating a mixed instance https://review.opendev.org/714706 | |
| 10:13:49 | dpaclt | Hi All I am unable to launch new vms .. http://paste.openstack.org/show/795937/ ..Anyone please suggest | |
| 10:15:50 | lyarwood | dpaclt: this isn't a support channel but grep for the req-4ce3ba68-34f8-453c-bb2a-9be9ef9adcbd id across the api/scheduler/conductor logs and you should see why all of your hosts are unable to schedule the instance | |
| 10:18:01 | lyarwood | elod: https://review.opendev.org/#/c/732717/ (and the two under it) is finally ready btw if you have time today | |
| 10:19:00 | dpaclt | Thanks lyarwood | |
| 10:44:39 | elod | lyarwood: sure, added to my TODOs :) | |
| 10:45:25 | lyarwood | thanks! | |
| 10:49:07 | openstackgerrit | Merged openstack/nova master: objects: Introduce 'pcpuset' field for InstanceNUMACell https://review.opendev.org/714658 | |
| 12:01:16 | openstack | bug 1550919 in OpenStack Compute (nova) queens "[Libvirt]Evacuate fail may cause disk image be deleted" [Medium,In progress] https://launchpad.net/bugs/1550919 - Assigned to Lee Yarwood (lyarwood) | |
| 12:01:16 | openstackgerrit | Elod Illes proposed openstack/nova stable/queens: Add functional test for bug 1550919 https://review.opendev.org/732715 | |
| 12:01:51 | openstackgerrit | Elod Illes proposed openstack/nova stable/queens: libvirt: Fix misleading debug msg "Instance is running" https://review.opendev.org/732716 | |
| 12:10:16 | openstackgerrit | Elod Illes proposed openstack/nova stable/queens: libvirt: Don't delete disks on shared storage during evacuate https://review.opendev.org/732717 | |
| 12:16:55 | jsuchome | gibi: hi, could you take another look at https://review.opendev.org/#/c/738738/ ? | |
| 12:27:08 | devfaz | Hi, is this a bug or known. If I try to disable an nova-compute service of a system terminated just seconds ago I get a trace with "An error occurred while updating the COMPUTE_STATUS_DISABLED trait on compute node resource providers managed by host XX. The trait will be synchronized automatically by the compute service when the update_available_resource periodic task runs" and rabbitmq-msgid timeout. | |
| 12:27:27 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.PY2 and six.PY3 https://review.opendev.org/726512 | |
| 12:27:41 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.add_metaclass https://review.opendev.org/726565 | |
| 12:28:01 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.reraise https://review.opendev.org/726898 | |
| 12:28:20 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.moves https://review.opendev.org/727224 | |
| 12:29:00 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.iteritems/itervalues/iterkeys https://review.opendev.org/727757 | |
| 12:29:19 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.byte2int/int2byte https://review.opendev.org/727777 | |
| 12:29:41 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.binary_type/integer_types/string_types https://review.opendev.org/728094 | |
| 12:40:28 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.text_type (1/2) https://review.opendev.org/728109 | |
| 13:00:08 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.text_type (2/2) https://review.opendev.org/728117 | |
| 13:02:23 | openstack | Launchpad bug 1858667 in OpenStack Compute (nova) "TestMultiCellMigrate.test_poll_unconfirmed_resizes_with_(no_)upcall race failing with greenlet error: Error auto-confirming resize: Reader <greenlet.greenlet object at 0x7f55c19dd048> to writer privilege escalation not allowed. Will retry later." [Undecided,Confirmed] | |
| 13:02:23 | lyarwood | dansmith: https://bugs.launchpad.net/nova/+bug/1858667 - I'm hitting this odd locking issue in some func tests, this smells like some behaviour has changed in py38 tbh but I'm having a hard time pinning it down. Would you be able to take a look? | |
| 13:21:02 | lyarwood | melwitt: ^ also if you have any time, I'm not even sure why we are wrapping target_cell if I'm honest | |
| 13:21:35 | lyarwood | CELL_CACHE is the only global thing I can see? | |
| 13:30:55 | dansmith | lyarwood: the entire DB state is global, which is what the lock is protecting | |
| 13:31:29 | dansmith | lyarwood: your python REPL examples aren't representative I think because the lock grabbing is serial in the code, yet you're nesting them (I think, without indentation) | |