| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-06-04 | |||
| 15:00:30 | dansmith | yep which? can float a job or yep, we've seen it work? | |
| 15:01:13 | sean-k-mooney | you can just do a DNM patch | |
| 15:01:30 | sean-k-mooney | and yes i was looking at patches that do that previously | |
| 15:01:44 | sean-k-mooney | i think there are still a few issue they are working through | |
| 15:01:50 | dansmith | I didn't know we could do that so easily.. now I'm curious if it works as is | |
| 15:02:41 | sean-k-mooney | you just need to refecne the correct pipeline ill see if i can find the patch i was looking at before | |
| 15:14:32 | sean-k-mooney | its 8 on 3.x kernels | |
| 15:14:37 | johnthetubaguy | sean-k-mooney: ah... | |
| 15:14:41 | sean-k-mooney | becuase that si the most that vhost kernel module supports | |
| 15:14:50 | johnthetubaguy | sean-k-mooney: guest limit, or host limit, or both? | |
| 15:14:54 | johnthetubaguy | ah | |
| 15:14:56 | sean-k-mooney | in 4.X its 256 | |
| 15:15:08 | johnthetubaguy | ah, interesting, thanks | |
| 15:15:22 | sean-k-mooney | we now have a config option so you can tell nova what it is if its different | |
| 15:15:32 | kashyap | gibi: I need to drop to run a quick errand, afraid | |
| 15:15:46 | gibi | kashyap: do you have other topics on the etherpad? | |
| 15:16:02 | kashyap | gibi: Not that I know of | |
| 15:16:02 | gibi | I don't see any | |
| 15:16:08 | gibi | OK. thanks for joining | |
| 15:16:53 | kashyap | I'm curious to hear stephenfin's topic of "documentation is awful" (line-474) - please take some notes on it on the Etherpad :) | |
| 15:17:39 | gibi | kashyap: ack | |
| 15:30:23 | sean-k-mooney | https://docs.openstack.org/nova/latest/configuration/config.html#filter_scheduler.max_io_ops_per_host | |
| 15:56:20 | melwitt | sean-k-mooney: just curious, how can you know there's not enough swap configured before any instances are booted so you know the flavor? | |
| 15:56:29 | CeeMac | hi sean-k-mooney and melwitt ; regarding or chat yesterday around shelving, we've done some testing today and it would appear that even in shelved_offloaded state an instance is accumulating usage and the resources still appear allocated in the project quota | |
| 15:57:09 | CeeMac | im back to being perplexed | |
| 15:57:24 | melwitt | CeeMac: oh... this is sounding familiar :( | |
| 15:57:43 | CeeMac | well, that doesnt sound good :/ | |
| 15:58:06 | CeeMac | i'm running rocky btw | |
| 15:58:51 | melwitt | long ago, when the shelve API was added I am recalling that we don't release quota while instance is shelved. but I didn't realize that would cause it to count as the simple tenant usage. but it makes sense that it would | |
| 15:59:27 | melwitt | the reasoning behind that decision was, not to let the user ever be in a position where they cannot unshelve for lack of quota | |
| 16:00:46 | CeeMac | yeah, i was actually having that conversation with my colleagues around implications to consider when shelving | |
| 16:01:40 | CeeMac | i guess, at least in the way I'm ingesting the data, it depends on how horizon is building its usage information for Admin | Overview | Usage summary | |
| 16:03:56 | CeeMac | let me see if I can pull some data from the openstack cli | |
| 16:16:53 | CeeMac | apparently I cant /fail | |
| 16:17:24 | melwitt | CeeMac: trying to get the same data that horizon shows you? | |
| 16:17:35 | CeeMac | yeah for the instances specifically | |
| 16:17:42 | CeeMac | i see the project usage in general | |
| 16:17:57 | CeeMac | although it wont show any details for the admin project | |
| 16:18:09 | CeeMac | which is odd (maybe) | |
| 16:18:20 | CeeMac | openstack usage show/list I've tried | |
| 16:18:45 | melwitt | ok I was about to ask, that should be the right command | |
| 16:18:52 | CeeMac | I could have sworn i'd seen something about instance usage when I was reading up yesterday, but I can't find it now if I did | |
| 16:19:02 | CeeMac | yeah, list only has some projects | |
| 16:19:10 | CeeMac | i'll maybe try with a different rc file, one sec | |
| 16:22:58 | CeeMac | yeah, thats not helped. I'll retry the test in demo project as I can see that | |
| 16:23:50 | CeeMac | although openstack usage show only gives the totals, not per instance | |
| 16:29:02 | CeeMac | ok I'm stumped | |
| 16:29:27 | melwitt | I need to look a deeper at it because obviously horizon is getting it somehow, to be able to tell you how to do it with the cli | |
| 16:31:11 | CeeMac | no worries | |
| 16:32:43 | CeeMac | i think maybe the actual api is more capable than what is implemented in the cli just now, from what I remember seeing reading through api-ref last night | |
| 16:32:55 | CeeMac | direct call api stuff is a bit beyond me at the moment though | |
| 16:35:10 | gmann | stephenfin: this one i was talking, i added index page for tempest plugins having some link on tempest do "Stable Branch * Policy" - https://docs.openstack.org/tempest/latest/plugins/index.html | |
| 17:02:39 | dansmith | stephenfin: your picture is dark so I can't tell if you were receiving my good-natured jabbing properly | |
| 17:03:02 | stephenfin | Heh, all good. See Zoom chat | |
| 17:03:24 | dansmith | ah heh yeah | |
| 17:23:30 | sean-k-mooney | melwitt: sorry didnt see that till now. total_ram * allocation ratio - reserved should <= total_ram + swap | |
| 17:24:02 | sean-k-mooney | so that would be the init_host check | |
| 17:24:24 | sean-k-mooney | the per vm check is if i add up the MEMORY_MB for all vms on this host is it larger then total ram | |
| 17:24:28 | sean-k-mooney | if so warn | |
| 17:25:00 | sean-k-mooney | the less agressive warning would be to warn on total_ram + swap | |
| 17:25:38 | sean-k-mooney | instead of > then total ram but once your are over total ram you are in OOM killer terrorty | |
| 17:26:58 | melwitt | sean-k-mooney: ah I see, thanks | |
| 17:27:23 | sean-k-mooney | melwitt: do you rememebr the nova bug for that transport thing i borught up at the end | |
| 17:27:42 | sean-k-mooney | melwitt: i remember debuging it with you at the time | |
| 17:27:52 | melwitt | yeah vaguely | |
| 17:28:01 | melwitt | it's the mandatory flag for oslo messaging | |
| 17:28:05 | sean-k-mooney | yes | |
| 17:28:28 | melwitt | I don't remember if we opened a bug about that specifically or what bug we referred to when talking about it | |
| 17:28:33 | sean-k-mooney | so it looks like there is an example of using it here | |
| 17:28:35 | sean-k-mooney | https://github.com/Gsantomaggio/rabbitmq-utils/blob/master/openstack/mandatory_test/mandatory_client_fail.py#L40-L54 | |
| 17:29:32 | sean-k-mooney | well i guess https://github.com/Gsantomaggio/rabbitmq-utils/blob/master/openstack/mandatory_test/mandatory_test.py#L35-L45 | |
| 17:29:36 | sean-k-mooney | is a better example | |
| 17:30:46 | sean-k-mooney | but what im not clear on is shoudl we just be setting it on every call or just when we do a topic send | |
| 17:31:16 | sean-k-mooney | the examples seam to be for topic sends | |
| 17:32:34 | melwitt | yeah, prob have the ask oslo peeps | |
| 17:32:40 | melwitt | cause I dunno either | |
| 17:33:35 | sean-k-mooney | i guess i could propose a patch and try adding it to things and see what breaks | |
| 17:33:42 | sean-k-mooney | https://bugs.launchpad.net/oslo.messaging/+bug/1849741 | |
| 17:33:42 | openstack | Launchpad bug 1849741 in oslo.messaging "TransportOptions are not documented" [Medium,Triaged] | |
| 17:33:44 | sean-k-mooney | that does not help however | |
| 17:34:33 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add reproducer for bug #1881455 https://review.opendev.org/733667 | |
| 17:34:33 | openstack | bug 1881455 in OpenStack Compute (nova) "migrate server reporting list index out of bound" [Undecided,New] https://launchpad.net/bugs/1881455 | |
| 17:34:33 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Use 'Exception.__traceback__' for versioned notifications https://review.opendev.org/733668 | |
| 17:39:41 | sean-k-mooney | stephenfin: by the way i don tthink you have to use Exception.__traceback__ directly anymore | |
| 17:39:51 | sean-k-mooney | infact that is an anti patteren i think | |
| 17:40:38 | melwitt | sean-k-mooney: could it have been this? https://bugs.launchpad.net/nova/+bug/1854992 | |
| 17:40:38 | openstack | Launchpad bug 1854992 in OpenStack Compute (nova) "Frequent instances stuck in BUILD with no apparent failure" [Undecided,Incomplete] | |
| 17:40:56 | sean-k-mooney | melwitt: yep that was the nova bug | |
| 17:41:15 | sean-k-mooney | well i think it was the second one | |
| 17:41:15 | melwitt | ok, can you write a note on it so we can have more hope to find it next time xD | |
| 17:41:41 | melwitt | the only reason I found that was because I thought I remembered erik reporting it | |
| 17:42:02 | sean-k-mooney | yep erik is from blizzard right | |
| 17:42:06 | melwitt | yeah | |
| 17:42:15 | sean-k-mooney | options = oslo_messaging.TransportOptions(at_least_once=True) | |
| 17:42:17 | sean-k-mooney | client = oslo_messaging.RPCClient(transport, | |
| 17:42:20 | sean-k-mooney | target, | |
| 17:42:21 | sean-k-mooney | transport_options=options) | |
| 17:42:23 | sean-k-mooney | so i think that is all we have to do | |
| 17:43:08 | melwitt | I wonder if there's a way we could test that it does something | |
| 17:43:22 | sean-k-mooney | where we do this https://github.com/openstack/nova/blob/master/nova/rpc.py#L208-L212 | |