| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-06-03 | |||
| 17:32:44 | sean-k-mooney | rouk: so currently you are just chainging [database]/connection | |
| 17:32:51 | rouk | yes. | |
| 17:32:53 | sean-k-mooney | to point to your nova db which is cell 1 | |
| 17:33:06 | sean-k-mooney | and ytou are not changin [api_database]/connection | |
| 17:33:12 | rouk | correct. | |
| 17:33:34 | sean-k-mooney | im not sure if that would implcitly be harmful | |
| 17:34:09 | sean-k-mooney | rouk: the schelder shoudl not need to be modifed by the way | |
| 17:34:14 | sean-k-mooney | it would only be nova-api | |
| 17:34:45 | rouk | yeah, i need to revert that change, kolla puts them onto the same config, i need to make my change more specific and split it up a bit | |
| 17:35:06 | sean-k-mooney | ya that is simple to do with kolla | |
| 17:35:20 | rouk | yeah, just going to do a nova-api.conf override | |
| 17:35:52 | sean-k-mooney | yep | |
| 17:36:06 | sean-k-mooney | i really like there config override system | |
| 17:36:25 | sean-k-mooney | you know htat you can use jinja varables in your config overrides too | |
| 17:37:54 | sean-k-mooney | so you can put connection = mysql+pymysql://{{ nova_cell0_database_user }}:{{ nova_cell0_database_password }}@{{ nova_cell0_database_address }}/{{ nova_cell0_database_name }} | |
| 17:37:56 | sean-k-mooney | max_pool_size = 50 | |
| 17:38:00 | sean-k-mooney | in your config | |
| 17:38:16 | sean-k-mooney | and just change teh {{nova_cell0_database_name}} bit | |
| 17:38:47 | rouk | yep, already got the template change written when i overrode the .j2 file originally, just copying my change into nova-api instead | |
| 17:39:28 | sean-k-mooney | rouk: sure just making sure you know the {{}} syntax works most people dont realise that the overrides are processed as jinja templates too | |
| 17:41:48 | rouk | i hope i dont have to write nova-api-metadata myself for victoria or something, since yeah, ill probably need per-cell metadata soon, one site will be extending past 100 hypervisors eventually. metadata will probably be fiiiine till 200 hypervisors before i start having perf issues. | |
| 17:42:10 | rouk | how bad will the fix be on the nova side to fix single metadata? | |
| 17:42:37 | sean-k-mooney | rouk: you can run multiple instance of thte metadta service today with kolla | |
| 17:43:05 | sean-k-mooney | so you should be fine well beyond 200 instance provided you configure memcache to cache between the differen metadata services | |
| 17:43:34 | rouk | yeah, i have a bunch, its just going to suck when theyre spread over geographic locations, or large networks. im talking 200+ hypervisors, 100-200 instances each | |
| 17:43:52 | rouk | right now im at 80something hypervisors. | |
| 17:43:56 | sean-k-mooney | rouk: but regarding the fix sinc e i have not looked at why we are not geting the info form the cell db im not sure but i suspect it should not be supper invaisve to fix | |
| 17:44:23 | rouk | should i open a bug for tracking? | |
| 17:44:30 | sean-k-mooney | yes definetly | |
| 17:46:07 | rouk | alright, ill have that up once i have my kolla change deployed and verify my users are happy. | |
| 18:08:04 | josephillips | hey | |
| 18:08:21 | josephillips | someone with idea of how to solve this The instance sync for host 'Computenode' did not match. Re-created its InstanceList. | |
| 18:08:21 | openstack | Launchpad bug 1881944 in OpenStack Compute (nova) "nova-api returns empty block-device-mapping in metadata queries" [Undecided,New] | |
| 18:08:21 | rouk | sean-k-mooney: https://bugs.launchpad.net/nova/+bug/1881944 | |
| 18:08:27 | josephillips | happend eventualy | |
| 18:10:17 | josephillips | i had a issue with placement api that is already resolved | |
| 18:10:25 | josephillips | but this is the one error left on scheduler | |
| 18:20:41 | artom | mordred, hey, the functional tests in sdk - they hit a real devstack-deployed cloud in some of the zuul jobs in CI, right? | |
| 18:20:58 | artom | Or am I completely misunderstanding the code? | |
| 18:32:38 | mordred | artom: yup, that's right | |
| 18:33:08 | artom | mordred, cool, thanks. That was one of the points that came out of the discussion on the nova room this morning - how to test the SDK in CI | |
| 18:33:17 | artom | Good to know there's a framework already in place | |
| 18:33:31 | artom | Though I think no multinode testing, right? So live migration for instance isn't tested | |
| 18:33:33 | mordred | artom: all of the openstacksdk-functional-* jobs (and everything in openstack/tests/functional run against devstack | |
| 18:34:01 | mordred | artom: that's right - but I imagine if there is a multi-node devstack job that sets up multinode it would not be hard to add one | |
| 18:34:24 | artom | mordred, yeah, working on that now. | |
| 18:34:32 | artom | (If it seems disorganized, that's because it is) | |
| 18:34:42 | mordred | you're describing my life :) | |
| 18:34:53 | artom | (Expect a flurry of WIP patches as I organise my thoughts by working trough them) | |
| 18:35:05 | mordred | artom: looking forward to them | |
| 18:35:50 | artom | mordred, awesome, thanks | |
| 18:35:59 | mordred | artom: fwiw - we *always* run nova tests no matter what the devstack config is but we have a bunch of different devstack configs - so with neutron advanced services, with magnum, with senlin, with masakari etc | |
| 18:36:02 | artom | I gotta take the kiddos to the park first though :P | |
| 18:36:06 | mordred | but none of those configs don't have nova | |
| 18:36:39 | artom | mordred, that sounds reasonnable | |
| 18:39:42 | dansmith | artom: mordred: sounds like that's what we're looking for, and important to note that (apparently) sdk functional is more like nova integrated | |
| 18:51:33 | mordred | dansmith: english is hard | |
| 19:23:31 | alex_xu | sean-k-mooney: gibi sorry, just saw yulong's message, I tried chrome, but I think it is network issue. With intel proxy doesn't work, without proxy even can't open the page. but anyway we already switch to zoom :) | |
| 21:39:10 | CeeMac | Evening | |
| 21:42:49 | CeeMac | I'm trying to confirm some details on what exactly 'active' means in context to (I believe) how nova-usage reports cpu-hours, gb-hours etc to horizon for Admin | Overview | Usage Summary. My original understanding was that 'Active' equated to time instances spent in 'running' state it appears that may not be the case. Is anyone able to confirm as what documentation I could find wasn't clear. | |
| 21:43:39 | CeeMac | When looking at usage over a period e.g for the previous month. | |
| 21:57:42 | CeeMac | Digging in to the api it looks like 'active' may relate to 'server_usages.state' but I can't find a list of alternative values to give that context. For example is an instance 'active' when it is shutdown, does it need to be shelved or deleted to change from 'active' state and not accumulate usage values? | |
| 22:15:34 | sean-k-mooney | artom: if you need a multi node devstack just change the nodepool nodeset | |
| 22:16:02 | sean-k-mooney | the devstack ansiable playbooks support muiltnode so it shoudl be easy to make the jobs multinode | |
| 22:18:01 | sean-k-mooney | CeeMac: i think that is correct for CPUs but i think Ram and disk might be different | |
| 22:19:03 | sean-k-mooney | shelving is intented to be the way for tenant to consume less resouces when a vm is not activly needed | |
| 22:19:53 | sean-k-mooney | in a public cloud when an instnace is shelved you are normally only changed for the disk usage in glance and not the cpu/ram/devices consumed by the flavor | |
| 22:20:25 | sean-k-mooney | since shut down vms still consume the cpu/ram/disk from a billing/usage point of view its more or less the same as if its running | |
| 22:40:13 | CeeMac | Thanks sean-k-mooney, I've just found a list of states in the api-ref parameters file. Just need to work out where the logic is queried so I can validate which of the states are 'counted', but in the interim I'll look into shelving options and document that up for our users. | |
| 22:42:34 | CeeMac | I guess while the instance is 'there' its effectively holding a reservation against the CPU and ram resources? Hence its metered for prospective billing/accounting purposes. | |
| 22:48:58 | melwitt | CeeMac: afaict, the state is not considered, it's only the launched_at/terminated_at https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/simple_tenant_usage.py#L51 so shelving would be the way to stop/resume usage | |
| 22:52:35 | melwitt | and you are correct, while the instance exists it is holding a reservation against resources. even when it's shut down | |
| 22:53:21 | melwitt | the shelve API was added to provide an easier way to remove/restore an instance from holding resources | |
| 22:53:38 | CeeMac | Thanks melwitt that seems to confirm it. OK, shelving it is then! | |
| 22:54:20 | CeeMac | I like to try and understand the logic behind things when/where I can :) | |
| 22:55:34 | melwitt | +1 :) | |
| 23:00:31 | CeeMac | Oh, I remember reading about this a while back actually. Do you happen to recall the setting for how long an instance can remain shelved before it gets offloaded / deleted and what the default is? | |
| 23:01:30 | melwitt | https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.shelved_offload_time | |
| 23:02:16 | melwitt | default is 0 for immediate offload | |
| 23:02:48 | josephillips | melwitt: taking the oportunity about shelved offload | |
| 23:03:04 | josephillips | is posible configure if a image is shelved move the storage backend too? | |
| 23:06:08 | melwitt | not completely sure I understand the question but if you shelved a volume backed instance it will snapshot the volume as well | |
| 23:06:23 | josephillips | yes | |
| 23:06:29 | josephillips | but will keep on the same backend | |
| 23:06:37 | josephillips | example if i shelved a image | |
| 23:06:41 | josephillips | a vm sorry | |
| 23:06:54 | josephillips | is a waste of resource keep on SSD | |
| 23:07:01 | josephillips | i prefer move the volumes to a HDD backend | |
| 23:07:06 | josephillips | this is posible? | |
| 23:08:18 | melwitt | oh, I see. I'm afraid I don't know if there's a way to do that | |
| 23:09:01 | CeeMac | So once the instance is offloaded there is another timer before it is completely deleted? Or it will stay shelved indefinitely? | |
| 23:09:37 | CeeMac | I seem to recall seeing a reference to that but can't find it now | |
| 23:09:50 | melwitt | josephillips: I'm thinking I wonder if you could do like a volume migrate via the cinder API to accomplish that, after it's been shelved | |
| 23:10:10 | josephillips | yeah is possible doing via cinder manually | |
| 23:10:23 | josephillips | the idea is doing automatically | |
| 23:10:42 | josephillips | via horizon | |
| 23:11:46 | melwitt | oh, ok, yeah sorry. it would "seem" that could be a relatively easy thing to add to osc/horizon as kind of a batch operation | |
| 23:11:58 | josephillips | yep | |
| 23:12:56 | melwitt | CeeMac: I think it stays shelved indefinitely | |
| 23:14:55 | CeeMac | Great, thanks. I'll experiment some tomorrow. Sleep now :) | |