Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2024-05-14
16:55:07 andrewbonney Yeah that's it
16:55:11 dansmith gotcha
16:55:37 andrewbonney Happy to work around it for us, but I doubt we'll be the only ones to hit it
16:55:59 sean-k-mooney so just looking at our docs https://docs.openstack.org/nova/latest/cli/nova-manage.html#db-online-data-migrations
16:56:11 sean-k-mooney we state that if you dont pass --max-count
16:56:11 dansmith yeah, I'm not sure the best way to handle that in the context of how that manage command works
16:56:21 sean-k-mooney we will loop internanlly in batchs of 50
16:56:36 dansmith yeah, but we don't have marker stuff there to prevent us from looping over the same 50 over and over
16:56:42 sean-k-mooney but if you do pass --max-count you have to loop externally and look at the repsoce code
16:56:54 sean-k-mooney well we dont actully need that
16:57:13 dansmith so I mean, we could just ignore deleted instances, but that means we may have a problem adding a constraint later where this will come up again
16:57:14 sean-k-mooney since we only get instnace that dont have the compute_id set
16:57:30 sean-k-mooney but the issue is we are not settign a sentinel for those that dont match
16:57:33 dansmith right but the problem is that if the 50 are not migratable, we'll never mark them as identified and keey finding them
16:57:38 dansmith yeah
16:57:49 sean-k-mooney so we need to set it to all 0 or something like that
16:57:53 dansmith I mean, that's one way to go, but it's not really reasonable I think
16:57:58 dansmith because that makes it not an FK
16:58:18 sean-k-mooney oh right ya it would fail the FK check
16:58:47 sean-k-mooney can we return the instance uuid as a marker if we add an orderby
16:58:57 dansmith I mean, it's not marked as an FK right now, because we suck at that, but still, it's "wrong" I think
16:59:12 dansmith right, so we could add in marker stuff there, but that's kindof a bigger deal really
16:59:24 dansmith changes the interface of all of these migration routines
16:59:38 sean-k-mooney well it woudl need a new cli arg for a start and it woudl chnage the resonce so not backporable
17:00:09 dansmith one thing we could do is if we get to the bottom of this loop and couldn't migrate any of the things we found, just raise an exception and say "yo dawg, you need to archive your db"
17:00:28 dansmith it's not a CLI thing so much as the interface for these migration functions
17:00:34 dansmith they all return all,hit right now
17:01:50 dansmith I need to be reviewing the monster EE spec, so let's just table this and I
17:01:56 dansmith will do some thinking on it later
17:03:21 andrewbonney Thanks for the time
#openstack-nova - 2024-05-15
03:08:09 stan is there any way to connect the vm instance CPU and memory usage?
08:28:07 opendevreview Rajesh Tailor proposed openstack/nova master: Fix typos and punctuation issues https://review.opendev.org/c/openstack/nova/+/878779
14:19:03 opendevreview Dan Smith proposed openstack/nova master: Enable OCaaS for several nova jobs https://review.opendev.org/c/openstack/nova/+/919738
14:19:04 opendevreview Dan Smith proposed openstack/nova master: Stop using split UEC image (mostly) https://review.opendev.org/c/openstack/nova/+/919739
14:25:40 sean-k-mooney dansmith: OCasS is openstack client as a service right? creating a single cleint instnace and reusing it to make queies
14:25:48 dansmith yes
14:26:07 sean-k-mooney when did that merge cause i rememebr that beign a thing in the works for years
14:26:19 dansmith saves 5m of wasted wall clock time alone on a simple devstack run, not to mention the cpu and disk
14:26:37 dansmith sean-k-mooney: yesterday, I just freshened it up and fixed some stuff and we shoved it in
14:26:48 sean-k-mooney ok cool
14:28:15 sean-k-mooney ill review both those once the ci runs
14:28:39 sean-k-mooney dansmith: can you hit this https://review.opendev.org/c/openstack/nova/+/915562
14:28:53 sean-k-mooney there is a placement one too
14:29:09 sean-k-mooney here https://review.opendev.org/c/openstack/placement/+/915568
14:30:52 dansmith gladly
15:08:26 opendevreview mike_mp@zzzcomputing.com proposed openstack/nova master: do not use str(url) to stringify a URL for subsequent use https://review.opendev.org/c/openstack/nova/+/913910
15:08:30 chris218 Hi I'm reading about CPU topologies and pinning in the docs, from the looks of it if I configure few CPUs as dedicated I won't be able to use them as shared if I run out CPUs specified as shared?
15:10:08 gibi chris218: you understand it correctly. You need to statically split your physical CPUs between shared, dedicated, reserved for the os
15:11:45 chris218 So there isn't a way to tell nova to treat all the CPUs as shared or dedicated depending on what the users will request?
15:20:33 opendevreview Alexey Stupnikov proposed openstack/nova master: [doc] Improve description for nova-manage db purge https://review.opendev.org/c/openstack/nova/+/919746
15:24:48 opendevreview Stephen Finucane proposed openstack/nova master: do not use str(url) to stringify a URL for subsequent use https://review.opendev.org/c/openstack/nova/+/913910
15:25:32 stephenfin melwitt: Fixed the test ^
15:26:57 melwitt stephenfin: ack, will look
15:30:16 stephenfin bauzas: The OpenAPI spec has 2 +2s but I think folks are waiting on you to review it again. Could you do that or indicate if we can merge as-is? https://review.opendev.org/c/openstack/nova-specs/+/909448
15:30:33 bauzas stephenfin: I'll try for sure
16:51:18 opendevreview Merged openstack/nova master: Remove SQLAlchemy tips jobs https://review.opendev.org/c/openstack/nova/+/915562
16:51:24 opendevreview Merged openstack/placement master: Remove SQLAlchemy tips jobs https://review.opendev.org/c/openstack/placement/+/915568
17:29:22 sean-k-mooney dansmith: is there a reason we are not setting openstack-cli-server: true in the base tempest job?
17:29:39 sean-k-mooney well or base devstack job
17:29:41 dansmith sean-k-mooney: just wanted to be gradual
17:29:50 dansmith oh you mean in the other repo or ours?
17:30:02 sean-k-mooney in the ohter repo
17:30:09 sean-k-mooney but if you want to test it in ours first
17:30:13 dansmith yeah, I don't think the owners are quite ready for that
17:30:13 sean-k-mooney then im ok with that
17:30:25 dansmith I think we should lead the way here and show that it's stable and then we can argue for it
17:30:28 sean-k-mooney ok just confiming
17:31:22 sean-k-mooney i was checkign the local.conf in tempest-integrated-compute and realised it was not set
17:31:35 sean-k-mooney then i realsed you only enbaled it in some of the jobs and i was trying to figure out why
17:37:28 clarkb dansmith: nova-next's devstack runtime was just under 11 minutes with your change. Thats pretty impressive
17:38:39 dansmith clarkb: yeah, I saw a super fast nova-live migration too
17:38:53 JayF clarkb: fyi I have https://review.opendev.org/c/openstack/ironic/+/919620 up as well, harder to tell w/ironic b/c our times vary so much per cloud
17:38:55 sean-k-mooney 39 mins
17:39:03 JayF with a ty to Dan for enabling that in Ironic
17:39:11 sean-k-mooney i -1 the patch because your current patch is only setting it on the conotler nodes not the compute nodes
17:39:21 sean-k-mooney but over all it looks good
17:40:47 clarkb JayF: the devstack log file logs a bunch of devstack specific runtime info at the end
17:40:57 sean-k-mooney well no 39 mins was the full jobs i realise now when you said 9 mints that just devstack
17:42:01 sean-k-mooney JayF: https://zuul.opendev.org/t/openstack/build/265ec9ab7be145ef9925f5bfef62e2f4/log/controller/logs/devstacklog.txt#21773
17:42:04 clarkb JayF: for example https://zuul.opendev.org/t/openstack/build/68136706b3a446ecabfcee5e895f079c/log/controller/logs/devstacklog.txt#37755-37771 the osc line there is 2 minutes of osc but if you look at other jobs outisde of this change its like in the 5-10 minute range
17:42:52 sean-k-mooney clarkb: yep and the async_wait is likely reduced as well
17:43:16 sean-k-mooney we were likely hiding some of the latency isn async tasks
17:43:18 clarkb https://zuul.opendev.org/t/openstack/build/c01518a4f3a64114851dd40f7d1598e1/log/controller/logs/devstacklog.txt#31045-31061 same ironic job against a different change for comparison
17:44:29 clarkb 410 second difference between those two runs of the same job in the osc accounting. That is almost 7 minutes
17:44:32 JayF clarkb: sean-k-mooney: before: 353, after: 101 -- both on rackspace cloud
17:44:56 sean-k-mooney JayF: ya there is a lot of veriabity in the jobs however
17:45:05 sean-k-mooney but its non trivial in any case
17:45:12 JayF I'm sure, but 250 seconds is well beyond variance
17:45:19 sean-k-mooney sadly no
17:45:43 clarkb sean-k-mooney: in this case the low value is the indicator there is improvement
17:45:51 clarkb there may be that much variance but only on the high end not the low end
17:45:56 sean-k-mooney we see 20-30 mins of variance in some jobs depending on the provider. even within a single provier its sever mintues
17:46:11 clarkb particularly for osc
17:46:17 JayF yeah, that's what I'm saying, 100 seconds in OSC is well below the value I've seen for anything else
17:46:28 JayF not talking overall job run speed, just seconds in osc
17:46:33 sean-k-mooney clarkb: oh im not doubing this is an improvment
17:46:47 sean-k-mooney just that we will need more then one ci run to see how much
17:47:36 JayF I'm on the third example of that job and haven't found one >300 yet, in fact 353 is my lowest number yet, so I'm going to stop trying to kill my joy and say thank you for the extra 4 minutes :D
17:48:59 sean-k-mooney dansmith: so your change is currently only setting this on the contoler node on our multi node jobs. do we want to merge it as is and have a fix up later or fix it to apply to both. most of the destack time happens on the contoler anyway
17:49:13 sean-k-mooney so the speedup on the compute while non 0 will not be the main speedup in the job

Earlier   Later