Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-12-05
23:30:49 sean-k-mooney its in tree never mind
23:40:10 openstack Launchpad bug 1805984 in OpenStack Compute (nova) "Placement is not aware of disable compute nodes" [Undecided,New]
23:40:10 tssurya mriedem: we have filed the bug already regarding placement not knowing about disabled computes: https://bugs.launchpad.net/nova/+bug/1805984
23:41:14 sean-k-mooney tssurya: its not really a bug its a new feature that shoudl live in nova
23:41:26 sean-k-mooney well not quite
23:41:44 sean-k-mooney its not currently expected that placement would know
23:41:55 sean-k-mooney so its no a bug in that sence
23:49:44 openstack Launchpad bug 1708958 in OpenStack Compute (nova) "disabling a compute service does not disable the resource provider" [Wishlist,Won't fix]
23:49:44 tssurya sean-k-mooney: right agreed its not a bug since it was decided as a won't fix on the placement side (https://bugs.launchpad.net/nova/+bug/1708958)
23:50:19 tssurya but mriedem wanted a bug (was replying to his ML post)
23:50:25 sean-k-mooney tssurya: i set it to triage and wish list with the suggestion of converting to a spec or specless blueprint
23:50:28 tssurya to just track the issue
23:50:58 sean-k-mooney tssurya: if it was neutron i would have just set the RFE tag on the bug but nova does not use that
23:51:24 sean-k-mooney its definetly an edge case an i can see it causeing issues for larger deployments
23:52:17 sean-k-mooney did you see the conversation we had on this an hour ago
23:56:08 tssurya sean-k-mooney: was just scrolling back up to read it :D
23:56:25 tssurya I can see a couple of plausible solutions being proposed
23:56:42 sean-k-mooney yep there are 2 or 3 options
23:57:11 tssurya probably the one where reserved=total + disabling periodic updates on disabled computes to avoid overwrite is the better option
23:57:23 sean-k-mooney it looks like we should be able to come up with somethign
23:57:27 tssurya considering adding another periodic task just for this might be a overhead
23:57:32 sean-k-mooney tssurya: im not sure about that
23:58:03 sean-k-mooney i guess we will see the traits approch or using a disabled aggreate seam like a clean solution
23:58:43 tssurya oh disabled aggregates! didn't see that part yet, reading now
23:59:53 sean-k-mooney tssurya: well a disable aggreage is just an aggrage that we "nova" adds host that are down too and we jsut use the not member_of proposal to avoid
#openstack-nova - 2018-12-06
00:00:49 sean-k-mooney specifcially use this feature https://review.openstack.org/#/c/603352/
00:03:33 tssurya sean-k-mooney: yea thanks, did read the full spec yet but got the idea
00:04:19 tssurya so it would be something like adding/removing to an aggregate when we disable/enable the service I guess
00:05:44 tssurya and the fourth solution was traits I see
00:05:45 sean-k-mooney yep and we then jsut alway include member_of=!<uuid of disbaled aggreage>
00:05:51 sean-k-mooney in the placement request
00:06:06 tssurya sean-k-mooney: yep makes sense
00:06:21 sean-k-mooney we can use a uuid5 to generate a sable uuid for the aggragte but other services can create there own
00:07:58 tssurya yea, as long as the aggregate doesn't get stale (as in disruption during enable which fails to remove it from the aggregate)/we keep it in sync should work well.
00:08:12 tssurya which is a corner case
00:08:38 sean-k-mooney tssurya: well we could take care of that with a periodic task to fix it
00:09:01 sean-k-mooney e.g we udate it imendiatly form the api but heal it later if the update got lost
00:09:49 tssurya sean-k-mooney: after looking at the RT's periodic task, I am so down with periodic tasks as of now :D because of the overhead when having several computes
00:10:07 tssurya but yea as long as the interval is not so frequent, should be ok
00:10:17 sean-k-mooney haha ya but this would be on i guess the conductor
00:10:29 sean-k-mooney or supper conductor
00:11:17 sean-k-mooney the node status is in i think both the api and cell db so we dont need to run this on every compute node
00:11:42 tssurya node status is only in the cell db's
00:11:57 sean-k-mooney ok so but it could run on the cell conductor then
00:12:14 tssurya yea could be
00:12:40 sean-k-mooney an you could set the interval to a week if you wanted and this time supprot disabling it right out of the box by setting to 0
00:13:06 tssurya but not sure if its an upcall when updating the api aggregate table frm the cell conductor but yea these are implementation details
00:13:11 sean-k-mooney if you trust it to not get out of sync our you will fix it later then no need to run it
00:13:50 sean-k-mooney tssurya: oh im talking about placemetn aggreagtes not nova host aggrats by the way
00:14:45 tssurya sean-k-mooney: oh okay!
00:15:05 tssurya since you saud "nova" adds hosts, got confused
00:15:34 sean-k-mooney ya i should have said nova update the placment aggragte with the compute node RP uuid
00:16:06 tssurya right yea
01:28:47 jaypipes cfriesen: you have awoken the kraken. but unfortunately, the kraken is too tired from playing tennis to fight tonight :) so, will chat tomorrow about it.
02:23:55 openstackgerrit Jack Ding proposed openstack/nova master: Preserve UEFI NVRAM variable store https://review.openstack.org/621646
02:33:19 openstackgerrit Guo Jingyu proposed openstack/nova-specs master: Proposal for a safer noVNC console with password authentication https://review.openstack.org/623120
02:44:30 openstackgerrit Guo Jingyu proposed openstack/nova-specs master: Proposal for a safer noVNC console with password authentication https://review.openstack.org/623120
02:46:48 openstackgerrit Guo Jingyu proposed openstack/nova-specs master: Proposal for a safer noVNC console with password authentication https://review.openstack.org/623120
02:51:53 openstackgerrit Guo Jingyu proposed openstack/nova-specs master: Proposal for a safer noVNC console with password authentication https://review.openstack.org/623120
03:11:01 openstackgerrit Guo Jingyu proposed openstack/nova-specs master: Proposal for a safer noVNC console with password authentication https://review.openstack.org/623120
03:26:36 openstackgerrit Zhenyu Zheng proposed openstack/nova master: Handle tags in _bury_in_cell0 https://review.openstack.org/621856
03:43:55 openstackgerrit Takashi NATSUME proposed openstack/nova stable/rocky: Add a bug tag for nova doc https://review.openstack.org/623130
04:16:45 openstackgerrit Guo Jingyu proposed openstack/nova-specs master: Proposal for a safer noVNC console with password authentication https://review.openstack.org/623120
04:52:18 openstackgerrit Guo Jingyu proposed openstack/nova-specs master: Proposal for a safer noVNC console with password authentication https://review.openstack.org/623120
06:58:42 openstackgerrit Takashi NATSUME proposed openstack/nova master: api-ref: Body verification for the lock action https://review.openstack.org/622835
07:00:32 openstackgerrit Zhenyu Zheng proposed openstack/nova master: Handle tags in _bury_in_cell0 https://review.openstack.org/621856
07:41:25 openstackgerrit Merged openstack/nova master: Update mailinglist from dev to discuss https://review.openstack.org/621827
07:41:36 openstackgerrit Merged openstack/nova master: modify the avaliable link https://review.openstack.org/616905
09:16:53 gibi stephenfin: regarding shorted notification payload, would you like to see the samples stored in nova sorted only or also the json emitted on the message bus?
09:23:39 gibi mdbooth: in the failure in nova.tests.functional.regressions.test_bug_1550919.LibvirtFlatEvacuateTest in http://logs.openstack.org/22/606122/7/check/nova-tox-functional/1f3126b/testr_results.html.gz there are 9 seconds gap between '[nova.virt.libvirt.driver] Creating image' and the first polling of the server state. So someting is definitely slow there _before_ even the test try to wait for 5 seconds
09:23:45 gibi to see the server is ACTIVE state
09:24:06 mdbooth gibi: looking
09:24:33 mdbooth gibi: That's one of the ones which looked 'generally unhappy' I think.
09:25:08 gibi mdbooth: unfortunately we dont have such timing data from successful runs :/
09:26:14 mdbooth gibi: Or DEBUG logs :(
09:26:19 gibi does 'Creating image' means that in these test we really generate the root fs for the instance?
09:27:28 mdbooth gibi: No. The disk creation is stubbed to just touch the file.
09:27:58 mdbooth However, it does execute _resolve_driver_format in imagebackend, which executes qemu-img.
09:28:16 mdbooth I have a patch locally to mock that out, but on my unloaded system it doesn't make a massive difference.
09:28:46 mdbooth I can see that it's about 3 seconds of wall clock time, although multithreaded.
09:32:59 gibi mdbooth: I see. Honestly I don't know what could be the real problem. In the run http://logs.openstack.org/22/606122/7/check/nova-tox-functional/1f3126b/testr_results.html.gz I see missing DB table exceptions as well that reminds me the race we fixed with cdent last week in the placement db fixture. Maybe that race had other side effects. However we only fixed that in the split out placement repo.
09:34:05 mdbooth gibi: Like I said yesterday, I suspect it's just a canary: the first thing to die when conditions get bad.
09:34:30 mdbooth I could mock another couple of things out, and increase the timeout.
09:35:00 gibi mdbooth: could very well be it. Still it woul be nice to know why 5 seconds in not enough in these tests. But I know that we don't have data to figure that out
09:35:18 mdbooth gibi: Why don't we have debug enabled, btw?
09:36:09 gibi mdbooth: I'm not sure, maybe we don't want to store that much of logs.
09:37:13 gibi mdbooth: but if you can propose a patch that turns the debug log on, I can support that. and we will see if there are other oppinions
09:42:47 gibi stephenfin: I can try to propose the sorting patch and see if people start puking
09:43:03 gibi stephenfin: anyhow, thanks for the reviews on those patches
09:43:27 mdbooth gibi: Thanks for spending time on this.
09:43:50 gibi mdbooth: I mysteries :)
09:44:14 mdbooth Sometimes, anyway.
10:10:40 yan0s I have a question about nova policies
10:11:27 yan0s are variables "compute:create", "compute:get" etc deprecated?
10:12:08 yan0s I see they are not mentioned in the latest documentation
10:12:09 yan0s https://docs.openstack.org/nova/rocky/configuration/policy.html
10:13:15 yan0s also testing some of them I think they affect access rights
10:13:42 yan0s only their "os_compute_api" equivalents work
10:19:22 kashyap gibi: Heya, Zuul was "-2" on this: https://review.openstack.org/#/c/620327/. Guess if I do a 'rechek', I'll "lose" all the ACKs?
10:20:00 gibi kashyap: I don't think so

Earlier   Later