| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-06-23 | |||
| 15:01:33 | dansmith | that's a glance policy enforcement problem separate from this I think | |
| 15:01:58 | sean-k-mooney | ya i think so too | |
| 15:02:37 | sean-k-mooney | ok i better go get some work done. thanks for bring me up to speed o/ | |
| 15:03:39 | dansmith | yup | |
| 16:11:39 | gibi | public announcement: I will be off tomorrow but will be back on Thurstday | |
| 16:12:03 | sean-k-mooney | gibi_off: enjoy | |
| 16:12:42 | gibi_off | sean-k-mooney: thanks | |
| 17:23:35 | sean-k-mooney | efried: not sure if you are about but do you know if there is any reason we have not made provider trees or proviers to be iterable over there childeren? | |
| 17:24:45 | sean-k-mooney | i know that is not stirctly need but it would be nice to be able to pretty print them and it makes some operation nicer | |
| 17:25:43 | efried | Yes, there's a function to iterate over them in order. We use it in the master update_provider_tree loop. | |
| 17:26:09 | sean-k-mooney | efried: oh ok we just did not implement __iter__ | |
| 17:26:24 | efried | I would believe that. | |
| 17:26:28 | efried | It recently came up here: https://review.opendev.org/#/c/719163/18/nova/scheduler/client/report.py | |
| 17:27:17 | sean-k-mooney | that gives use the provider uuids but not the provider objects but ok i was looking at that | |
| 17:28:05 | sean-k-mooney | efried: thanks im trying to think how i will go about the transfroamtion for modeling nuam in placement and i want a nice way to visualise the tree | |
| 17:32:25 | openstackgerrit | Stephen Finucane proposed openstack/nova master: hardware: Remove '_numa_fit_instance_cell_with_pinning' https://review.opendev.org/714703 | |
| 17:32:25 | openstackgerrit | Stephen Finucane proposed openstack/nova master: hardware: Invert order of NUMA topology generation https://review.opendev.org/714701 | |
| 17:32:26 | openstackgerrit | Stephen Finucane proposed openstack/nova master: hardware: Tweak the 'cpu_realtime_mask' handling slightly https://review.opendev.org/461456 | |
| 17:32:26 | openstackgerrit | Stephen Finucane proposed openstack/nova master: hardware: Allow 'hw:cpu_realtime_mask' to be omitted https://review.opendev.org/737026 | |
| 17:33:08 | efried | I thought gibi_off wrote some kind of tool to prettyprint those trees. | |
| 17:33:17 | sean-k-mooney | maybe | |
| 17:33:37 | efried | I don't have it in front of me, but looking provider_tree.py for that iterator function. | |
| 17:33:53 | efried | It's not __iter__ but I'm pretty sure I made a getter that gives them to you in a predictable order. | |
| 17:33:57 | sean-k-mooney | it looks like its doing a depth frist traversal from the root with childern lexically sorted by uuid | |
| 17:35:05 | sean-k-mooney | efried: ya its fine ill figure it out. i might make it iterable jsut to make it spelper and immperment __repr__ for provier | |
| 17:35:24 | sean-k-mooney | *simpler and implement | |
| 17:51:49 | sean-k-mooney | efried: so ya i just need to add | |
| 17:51:50 | sean-k-mooney | def __iter__(self): | |
| 17:51:53 | sean-k-mooney | for uuid in self.get_provider_uuids(): | |
| 17:51:55 | sean-k-mooney | yield self._find_with_lock(uuid) | |
| 17:52:42 | sean-k-mooney | but i might invert the logic an make get_provier_uuids use the iterator | |
| 17:53:15 | sean-k-mooney | it would avoid the need to call fine_with_lock twice once in get_provier_uuids and then again in the iterator | |
| 17:53:29 | sean-k-mooney | thanks for the pointer | |
| 18:01:16 | efried | sean-k-mooney: feel free to add me to review if you spin up something there. | |
| 18:03:14 | efried | Here's where that thing is used https://github.com/openstack/nova/blob/f1ebc15dfc8ffb7f23b2cb9879f0ca9376931a90/nova/scheduler/client/report.py#L1348-L1349 (and IIRC the reason it was written) | |
| 18:03:50 | efried | so whatever you change, please keep in mind that that ^ is going to run every periodic -- so don't slow it down too much :P | |
| 18:04:36 | sean-k-mooney | ya it should not be any slower | |
| 18:05:17 | sean-k-mooney | but sure ill add you to the review | |
| 18:07:51 | stephenfin | melwitt: could you hit this trivial requirements patch, please https://review.opendev.org/#/c/730960/ ? | |
| 18:25:04 | openstackgerrit | sean mooney proposed openstack/nova master: [WIP] make provider tree and proviers iterable https://review.opendev.org/737594 | |
| 18:25:28 | sean-k-mooney | efried: that is the version where i have not inverted the logic ^ | |
| 18:25:34 | sean-k-mooney | which im testign with http://paste.openstack.org/show/795116/ | |
| 18:29:39 | sean-k-mooney | which if i execute it looks like this http://paste.openstack.org/show/795117/ | |
| 18:31:19 | efried | ++ | |
| 18:32:04 | sean-k-mooney | i should be able to make the provider tee actully do indenting too but that the quick hack for now | |
| 19:22:01 | melwitt | gmann: did you see the author's reply on this patch? https://review.opendev.org/#/c/730960/1//COMMIT_MSG@9 | |
| 19:22:34 | gmann | melwitt: checking | |
| 19:27:09 | gmann | melwitt: i forgot to test it locally and if any things we need to adjust from jsonschema FormatChecker. let me do it and review again. thanks for reminder | |
| 19:27:23 | melwitt | gmann: sweet thanks | |
| 19:29:27 | melwitt | stephenfin: gmann is gonna review ^ again, there was a local test he wanted to do | |
| 20:15:21 | spatel | sean-k-mooney: hey! | |
| #openstack-nova - 2020-06-24 | |||
| 01:54:34 | openstackgerrit | Merged openstack/nova master: Cap jsonschema 3.2.0 as the minimal version https://review.opendev.org/730960 | |
| 02:27:16 | openstackgerrit | Merged openstack/nova master: Guard against missing image cache directory https://review.opendev.org/736964 | |
| 03:01:43 | openstackgerrit | Merged openstack/nova master: hardware: Update and correct typing information https://review.opendev.org/714694 | |
| 03:01:55 | openstackgerrit | Merged openstack/nova master: libvirt: Add typing information https://review.opendev.org/714695 | |
| 03:16:22 | openstackgerrit | Merged openstack/nova master: hardware: Add validation for 'cpu_realtime_mask' https://review.opendev.org/468203 | |
| 03:16:30 | openstackgerrit | Merged openstack/nova master: tests: Split instance NUMA object tests https://review.opendev.org/714696 | |
| 07:23:12 | brinzhang_ | Hello guys, has anyone encountered this problem with "GLib (gthread-posix.c): Unexpected error from C library during 'pthread_key_create': Resource temporarily unavailable. Aborting" | |
| 07:24:48 | brinzhang_ | it was happened when evacuate the instances, deployed the OpenStack R release on Centos7 | |
| 07:29:28 | gibi_off | brinzhang_: is this log from a nova service? Is that service has a task limit in the output of systemctl status ? | |
| 07:30:21 | gibi_off | brinzhang_: https://github.com/systemd/systemd/blob/ddb4b0d3eb57292c38a76f9b977f73cea15448fb/NEWS#L65-L68 | |
| 07:31:34 | brinzhang_ | gibi_off: Do you mean that the number of threads has reached the upper limit? | |
| 07:33:02 | gibi_off | brinzhang_: yeah, that could be a reason | |
| 07:33:30 | brinzhang_ | it's not come from nova service log, it's print by docker | |
| 07:35:26 | gibi_off | check the output of the systemctl status for the docker service | |
| 07:42:07 | brinzhang_ | gibi_off: This problem does not occur every time you evacuate. It happens occasionally, but it has been encountered twice. | |
| 07:43:30 | brinzhang_ | gibi_off: we will try to make the TasksMax setting get bigger, then try to test | |
| 07:43:48 | brinzhang_ | gibi_off: thanks for your advice | |
| 09:24:00 | stephenfin | bauzas: think you could look at https://review.opendev.org/#/c/714700/ ? It's the last patch I've authored in that series, so I can take it from there | |
| 09:24:12 | bauzas | stephenfin: ack, clicks | |
| 10:25:17 | nightmare_unreal | is migration ( both cold and live migration) are of more I/O bound task or more CPU bound ? | |
| 10:25:44 | sean-k-mooney | nightmare_unreal: it depend on what you mean by io but io | |
| 10:26:23 | sean-k-mooney | nightmare_unreal: the more the guest dirties memeory the longer it will take and that time is bounded by the bandwidth you have to copy the data over | |
| 10:26:36 | nightmare_unreal | i mean input / ouput. What I was thinking was since we move lots of data ( essentially a whole VM) it is heavy work right ? | |
| 10:26:55 | nightmare_unreal | ah okay | |
| 10:27:25 | sean-k-mooney | copying the disk is actully relitivly simple and not needed if its boot from volume | |
| 10:27:32 | sean-k-mooney | or on shared storage | |
| 10:27:39 | sean-k-mooney | its the ram that is a problem | |
| 10:27:59 | sean-k-mooney | the virtual cpu state is pretty small so that is not hard to transfer | |
| 10:28:37 | sean-k-mooney | if a vm is under heavy load you should expect that the live migration will never finish unless you enable auto convergence and/or post copy | |
| 10:28:53 | nightmare_unreal | ah okay !! | |
| 10:29:17 | sean-k-mooney | auto convergnece after an inital timeout progressivly starts micro pausing the vm for long and longer intervals | |
| 10:29:41 | nightmare_unreal | and this is all managed by nova itself ? | |
| 10:29:53 | sean-k-mooney | no its all manage by qemu/libvirt | |
| 10:29:53 | nightmare_unreal | I mean user won't have to do these checks right | |
| 10:30:01 | nightmare_unreal | ah okay | |
| 10:30:12 | nightmare_unreal | understood | |
| 10:30:18 | sean-k-mooney | there are nova config options for auto convergancve and post copy | |
| 10:30:31 | sean-k-mooney | we just tell libvit that it can use them and it tells qemu | |
| 10:30:43 | sean-k-mooney | i think they are off by default | |
| 10:30:49 | nightmare_unreal | okay | |
| 10:31:04 | nightmare_unreal | I will read some more docs on it to get better idea | |
| 10:31:06 | nightmare_unreal | Thanks sean-k-mooney | |
| 10:31:24 | sean-k-mooney | https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.live_migration_permit_post_copy | |
| 10:31:30 | sean-k-mooney | https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.live_migration_permit_auto_converge | |
| 10:31:39 | sean-k-mooney | personally i would set both to true | |
| 10:32:00 | sean-k-mooney | just be aware if you have post copy enabled and there is a netwrok partion during the migration it will crash the vm | |
| 10:32:24 | nightmare_unreal | okay :) . I will note that | |
| 10:32:25 | sean-k-mooney | i dont think that is a signifcant enough risk to warrent disabling it but that is something you should decied | |
| 10:33:15 | nightmare_unreal | okay. mnaybe after seeral trials and error i will find out | |