| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-05-18 | |||
| 13:37:48 | sean-k-mooney | gibi shoudl i also add you greenlet state reporting | |
| 13:38:07 | gibi | you can pull the patch top of it just for data | |
| 13:38:40 | gibi | but it does not prove anything as CI is basically doing a lot of operations then stops, so there is now time for the numbers to settle to a baseline | |
| 13:39:04 | sean-k-mooney | ya but the data could be interesting to compare | |
| 13:39:18 | sean-k-mooney | shall i cherry pick your patch so? | |
| 13:39:31 | sean-k-mooney | or rebase on it | |
| 13:41:02 | gibi | just cherry pick top of yours | |
| 13:42:03 | opendevreview | sean mooney proposed openstack/nova master: DNM: log number of green(thread|let)s periodically https://review.opendev.org/c/openstack/nova/+/841040 | |
| 13:42:12 | sean-k-mooney | cool | |
| 13:42:20 | sean-k-mooney | so looking at the output of the previous run | |
| 13:42:29 | sean-k-mooney | we see both greenthreadds and greenlets | |
| 13:42:41 | sean-k-mooney | and we are not expecting to only see greenthreads with my patch | |
| 13:43:31 | gibi | yepp we expect only greenthreads and no naked greenlets | |
| 13:44:11 | gibi | greenlets are implemented in a C extension greenthreads are proper python objects implemented by eventlet | |
| 13:44:45 | sean-k-mooney | so looking at the output from the test run | |
| 13:45:00 | sean-k-mooney | teh greenthrad sayed pretty constant | |
| 13:45:13 | sean-k-mooney | but the greenlets were more or less slowly increaing over time | |
| 13:45:59 | gibi | I see same behavior locally too, but most of the time after couple of minutes of idle time the greenlets also decreased back to baseline. | |
| 13:46:04 | sean-k-mooney | https://termbin.com/dzg3 | |
| 13:46:18 | gibi | in CI there is no couple of minutes of idle time | |
| 13:46:25 | frickler | gibi: kashyap: fyi I had another fix for this in tempest recentish https://review.opendev.org/c/openstack/tempest/+/835382 | |
| 13:46:27 | gibi | but it would be interesting to see what happens there | |
| 13:46:47 | frickler | this = volume attachments | |
| 13:47:08 | sean-k-mooney | ah for tagged attachments | |
| 13:47:12 | gibi | frickler: yepp, that helps too. thanks. we need to track down all the detach scenarios | |
| 13:47:14 | sean-k-mooney | ya | |
| 13:47:41 | gibi | all volume detach operation is potenitally affected | |
| 13:48:29 | sean-k-mooney | and attach since we do detach as a cleanup action | |
| 13:48:40 | gibi | sean-k-mooney: good point, yes | |
| 13:49:10 | sean-k-mooney | you know technially this could affect nics too | |
| 13:49:25 | sean-k-mooney | i dont think we have ever seen it affect them | |
| 13:50:02 | sean-k-mooney | but both are just virtual pci devices form qemus point of view | |
| 13:50:12 | sean-k-mooney | just one is virtio-blk and the other is virtio-net | |
| 13:51:18 | gibi | yeah, I never see this in case of interfaces | |
| 13:51:28 | gibi | probably something is different down in the stack | |
| 13:51:28 | ricolin | sean-k-mooney: stephenfin | |
| 13:51:38 | gibi | either qemu dev handling or the guest OS dev handling | |
| 13:51:48 | gibi | nova today uses the same codepath | |
| 13:52:13 | sean-k-mooney | gibi: ya. i guess network attach/detach is more common and porably better tested | |
| 13:52:33 | ricolin | if you got some time, please help to review https://review.opendev.org/c/openstack/nova-specs/+/840310 as mnaser already laeve some comments would like to have your feedback:) | |
| 13:52:46 | gibi | also probably force pulling out a disk is more problematic from data consistency perspective than pulling a netdev | |
| 13:53:22 | sean-k-mooney | right the netdev is mostly stateless | |
| 13:54:14 | sean-k-mooney | ricolin: i think its stephenfin's feedback we need really | |
| 13:54:40 | sean-k-mooney | i dont like exposing the aw_bits but i understand that ye have a need for it | |
| 13:55:07 | ricolin | sean-k-mooney: cool, thanks:) | |
| 13:55:25 | sean-k-mooney | i think hw_viommu_model is the write absraction for opting in as we use the same pattern for contoling the graphic deviecs or nic models | |
| 14:03:17 | sean-k-mooney | gibi: hum https://review.opendev.org/c/openstack/tempest/+/842140 still failed the same way | |
| 14:03:23 | sean-k-mooney | frickler: ^ any idea why | |
| 14:05:13 | gibi | sean-k-mooney: yeah, this is where my tempest knowledge ends. probably something is different in the network setup of this test | |
| 14:05:20 | frickler | sean-k-mooney: I'll take a look later, in a meeting now | |
| 14:05:51 | sean-k-mooney | gibi: possible i can see in the tempest output it waited for it to go form resize verify to active | |
| 14:05:56 | sean-k-mooney | and tehn it tries to ssh in | |
| 14:06:02 | sean-k-mooney | but it didn not work | |
| 14:07:20 | sean-k-mooney | "fixed_ip_address": "10.1.0.10" | |
| 14:07:22 | sean-k-mooney | ip-route:10.1.0.0/28 dev eth0 scope link src 10.1.0.10 | |
| 14:07:54 | sean-k-mooney | ah | |
| 14:08:11 | sean-k-mooney | ## ping -c 5 10.1.0.1 | |
| 14:08:13 | sean-k-mooney | PING 10.1.0.1 (10.1.0.1): 56 data bytes | |
| 14:08:15 | sean-k-mooney | --- 10.1.0.1 ping statistics --- | |
| 14:08:17 | sean-k-mooney | 5 packets transmitted, 0 packets received, 100% packet loss | |
| 14:08:20 | sean-k-mooney | the vm cannot ping its gateway | |
| 14:10:14 | sean-k-mooney | gibi: my guess is security groups | |
| 14:12:06 | bauzas | gmann: to clarify the new release naming thingy, it would be something like "2022.2 Arbitrary" for the AA release ? | |
| 14:12:28 | bauzas | where "Arbitrary" be chosen by the Foundation folks | |
| 14:12:29 | bauzas | right? | |
| 14:13:02 | sean-k-mooney | where "Arbitrary" is aardvark so say it dansmith :) | |
| 14:13:24 | sean-k-mooney | havnt actully read the most recent status of this | |
| 14:13:33 | sean-k-mooney | so also interested | |
| 14:13:39 | dansmith | I would love them to choose such a name to make my point, but yes, it's going to be aardvark :) | |
| 14:13:57 | dansmith | the name will be the name, the version will be 2022.2 | |
| 14:17:43 | sean-k-mooney | gibi: it might be becasue the server is not being created with teh validation resouces initally | |
| 14:17:53 | bauzas | dansmith: if it was named a-ha, it could take on me | |
| 14:17:58 | sean-k-mooney | gibi: so it might not hwave an sshkey | |
| 14:18:13 | gibi | sean-k-mooney: OK, that I can fix | |
| 14:18:21 | dansmith | bauzas: you win today | |
| 14:18:23 | sean-k-mooney | gibi: we are not calling https://review.opendev.org/c/openstack/tempest/+/842140/4/tempest/api/compute/volumes/test_attach_volume.py#45= | |
| 14:18:25 | gibi | thanks for looking at it | |
| 14:18:53 | sean-k-mooney | here https://review.opendev.org/c/openstack/tempest/+/842140/4/tempest/api/compute/volumes/test_attach_volume.py#384 | |
| 14:19:07 | bauzas | dansmith: heh, thanks for the explanation anyway | |
| 14:19:26 | sean-k-mooney | so we are not waiting for the server to be sshable before we attach the multi attach volume | |
| 14:20:14 | sean-k-mooney | a is for aardvark is an amican thing we normally say ant or similar | |
| 14:20:38 | dansmith | it's perfect because it starts with AA | |
| 14:20:44 | sean-k-mooney | yep | |
| 14:20:52 | dansmith | I challenge the rest of you to pick names for the other letters that begin thusly :) | |
| 14:25:20 | sean-k-mooney | dansmith: the real question is the next AB or BB becaue one is much more of a challange | |
| 14:25:50 | dansmith | sean-k-mooney: well, I had assumed BB, but maybe AB would be appropriate for the naming | |
| 14:25:59 | dansmith | however, I think all the things would have to start with A and somewhat defeat the point | |
| 14:26:08 | dansmith | however, these are the foundations' problems now :) | |
| 14:27:03 | sean-k-mooney | this is not what i was looking for but its cute so https://www.pinterest.ie/morganlimehouse/bb-animals/ | |
| 14:28:09 | bauzas | AB are two capital letters known a lot by French | |
| 14:28:28 | bauzas | lots of french cheapy soap series | |
| 14:32:13 | opendevreview | sean mooney proposed openstack/nova master: DNM: log number of green(thread|let)s periodically https://review.opendev.org/c/openstack/nova/+/841040 | |
| 14:32:13 | opendevreview | sean mooney proposed openstack/nova master: [DNM] allow monkey patching spawn_n to spawn https://review.opendev.org/c/openstack/nova/+/842359 | |
| 14:37:37 | opendevreview | sean mooney proposed openstack/nova master: DNM: log number of green(thread|let)s periodically https://review.opendev.org/c/openstack/nova/+/841040 | |
| 14:37:37 | opendevreview | sean mooney proposed openstack/nova master: [DNM] allow monkey patching spawn_n to spawn https://review.opendev.org/c/openstack/nova/+/842359 | |
| 14:53:08 | gmann | bauzas: dansmith sean-k-mooney on release name, we still discussing how to use it in development cycle (number only or number and name both ), should be ready by this week (plan to discuss in TC meeting tomorrow ) - https://review.opendev.org/c/openstack/governance/+/841800 | |
| 14:53:19 | gmann | but yes, name will be all foundation things now | |
| 14:54:54 | bauzas | Uggla: sean-k-mooney: dansmith: gibi: gave -1 for https://review.opendev.org/c/openstack/nova-specs/+/831506 just to make sure we have a consensus | |
| 14:56:51 | gmann | bauzas: sean-k-mooney on keypair allowing @|. as it will be error -> success, it is ok for backward compatibility but we need to have microversion for interoperability | |
| 14:57:26 | Uggla | bauzas, no pb. I agree. Have you seen my proposal to add another API parameter to mange the pin/unpin az of instances ? | |