| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-07-31 | |||
| 18:48:46 | mriedem | if you have a compute node with 300 GB of disk and flavors with 160 GB of disk, you're not going to get far | |
| 18:48:54 | mriedem | so you'd have to carve down your flavors to have smaller disk, | |
| 18:49:00 | mriedem | or add more storage to your compute nodes, | |
| 18:49:07 | mriedem | or use cinder to back the root disk | |
| 18:50:02 | nafiux | Lol, yeah. How servers backed with cinder volumes behave? Do they use as needed and not the actual size specified at creation time? | |
| 18:53:13 | mriedem | for volume-backed servers the root gb on the flavor doesn't really apply | |
| 18:53:34 | mriedem | the size of the volume is whatever it is when it's created | |
| 18:53:56 | sean-k-mooney | mriedem: im not sure i did a good job of explaing but it tried to explain the issue again im my reponce to your comments | |
| 18:54:00 | sean-k-mooney | https://review.opendev.org/#/c/669738/2 | |
| 18:54:01 | mriedem | the user can pre-create a volume and provide that to nova when creating the server, or the user can give some parameters to nova and nova will create the volume and attach it to the server | |
| 18:54:16 | sean-k-mooney | but yes i can factor the functional test our into there own regression module | |
| 18:54:27 | mriedem | so many words....but i started that | |
| 18:54:29 | sean-k-mooney | and add teh rebuild/shelve tests | |
| 18:55:31 | mriedem | nafiux: some people use cinder exclusively for root volumes of their servers, some people use shared storage for their compute nodes, some mix both - it takes all kinds | |
| 18:57:56 | sean-k-mooney | nafiux: cinder backed volume behvior depend on teh dirver. some cinder backends support thin provisioning allowing over commit some dont | |
| 18:58:49 | sean-k-mooney | and some time the same dirver change behavior based on your config setting which makes it all the more fun to debug | |
| 18:58:59 | nafiux | Thanks @mridem & @sean-k-mooney. I will read more about those topics, so, in general which are the different options for volumens to the instances? | |
| 19:01:11 | sean-k-mooney | there are two sets of volumes, volumes that have independent lifetime form the vm usually called data volumes | |
| 19:01:20 | sean-k-mooney | thos are create via teh cinder api. | |
| 19:01:40 | sean-k-mooney | and then there are volumes that have ther life time cupled to a vm | |
| 19:02:07 | nafiux | Ok, which is the equivalent to the root volumes in AWS? | |
| 19:02:16 | nafiux | Can I have a root volume with cinder? | |
| 19:02:20 | sean-k-mooney | yes | |
| 19:02:31 | sean-k-mooney | we call that boot form volume or bfv for short | |
| 19:03:01 | sean-k-mooney | in that case you tell nova that the root disk should not be create using the compute nodes default driver and instad should be provide by cinder | |
| 19:03:16 | nafiux | Oh, so, due my current setup for learning, I will leverage cinder for both bfv and additional volumes as needed | |
| 19:03:17 | sean-k-mooney | you can optionl specifythat that volume should be deleted when the vm is or not | |
| 19:04:05 | sean-k-mooney | i wasnt following the full conversation but you can use cinder for both | |
| 19:04:22 | nafiux | Ok, yea, is just because I have a lot of CPU & Memory but few hard disk | |
| 19:04:40 | sean-k-mooney | the default backend for non bfv guest is to create a local file on the compute node for the root disk of the instance | |
| 19:05:12 | sean-k-mooney | you can also tell the compute node to use a differnt default image backend such as ceph | |
| 19:05:46 | sean-k-mooney | in which case nova will create a remote ceph volume out of band of cinder | |
| 19:06:16 | nafiux | Ok, I need to read a lot :D cinder, ceph, and will back later with more specific questions, this is really amazing. | |
| 19:06:18 | sean-k-mooney | other peopl in a similar situation somethime mount the default location where libvirt stores its disk image on nfs | |
| 19:07:32 | sean-k-mooney | so as mriedem said there are lots of way to do it and people do many different things but its better to keep it simple when learning | |
| 19:08:13 | nafiux | Agree. Thanks for the support. | |
| 19:11:28 | mriedem | nafiux: more info on cinder in their docs https://docs.openstack.org/cinder/latest/ | |
| 19:11:46 | mriedem | there is also the ops guide which is older but relevant https://docs.openstack.org/operations-guide/ | |
| 19:12:41 | sean-k-mooney | or if you like video content there are way more videos then you will have time to watch on teh offical youtube channel | |
| 19:12:43 | sean-k-mooney | https://www.youtube.com/user/OpenStackFoundation | |
| 19:13:17 | nafiux | Sure! I will take a look on this too: https://docs.openstack.org/arch-design/design-storage/design-storage-concepts.html | |
| 19:47:42 | efried_rollin | mriedem: responded. TL;DR: a Set would be better, but a list will work just fine. | |
| 19:49:33 | efried | I guess that's not FUPable since it's an OVO/RPC thing. No takesy backseys. | |
| 19:49:46 | efried | so if you feel strongly about it you should -1 | |
| 19:52:52 | mriedem | done | |
| 19:53:16 | mriedem | i've made more comments up that series but non-voting b/c i'm a coward | |
| 19:53:27 | mriedem | i'd really like dan to review that series but he's out this week | |
| 19:54:48 | efried | mriedem: FYI Dan was involved earlier, which is how we ended up with the query and filter in their current form. | |
| 19:55:21 | mriedem | yeah i knew he blew up the single megapatch | |
| 19:56:04 | efried | but also was the one who suggested the db function as it now exists. | |
| 19:56:14 | efried | much nicer than it was before | |
| 19:56:43 | mriedem | b/c it's mostly a copy of the one he added for his earlier (first) pre-filters | |
| 19:57:08 | mriedem | fwiw most of my comments are "can you document this?" | |
| 19:57:52 | mriedem | i also feel like forbidden is the wrong word for the user-facing side of this feature, it's really about host isolation | |
| 19:57:57 | mriedem | but it's probably too late to change that now | |
| 19:58:25 | mriedem | unless i'm just too slow to connect the dots, which is possible | |
| 19:58:42 | mriedem | lots of negative logic in this feature makes it hard for me to reason about | |
| 19:58:56 | mriedem | don't not give me the hosts i don't not want | |
| 19:59:04 | efried | I'll have another look at the documentation, but I thought it was ultimately explained the right way for users. | |
| 19:59:17 | efried | The logic is necessarily hairy and compound-negative-confusing | |
| 19:59:25 | efried | but the ux winds up being really crisp and clean imo | |
| 20:02:58 | efried | ...you're right, the doc https://review.opendev.org/#/c/667952/10/doc/source/reference/forbidden-aggregates.rst still talks about forbidden aggregates. Though really, the explanation is very good, and "forbidden aggregates" is really just used as the name-of-feature. | |
| 20:16:30 | mriedem | yeah i'm not -1 on it, it's just a bit confusing | |
| 20:16:33 | mriedem | some of the wording | |
| 20:18:15 | efried | I think you've got a valid point that we could s/forbidden/isolated/ and it would be more comprehensible. The docs could be fixed later, but if we want to rename the conf opt we should do it now. | |
| 20:21:25 | prometheanfire | sean-k-mooney: ya, looks like it's lxml https://review.opendev.org/673848 | |
| 20:23:06 | mriedem | efried: question in https://review.opendev.org/#/c/670182/1 | |
| 20:24:23 | efried | mriedem: responded | |
| 20:29:24 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Fix python3 compatibility of rbd get_fsid https://review.opendev.org/664513 | |
| 20:31:07 | mriedem | amorin: are you still working on https://review.opendev.org/#/c/667294/ ? | |
| 20:45:14 | openstackgerrit | Merged openstack/nova master: neutron: log something more useful in _get_instance_nw_info https://review.opendev.org/673694 | |
| 21:43:21 | efried | mriedem: RequestContext question for ya | |
| 21:43:56 | efried | I've been tracking where nova-compute talks to Placement to do things like creating the compute node RP. | |
| 21:44:42 | efried | IIUC those interactions use a RequestContext that gets created when the service starts up, because they're done in a periodic, not based on a request coming from e.g. conductor or whatever. | |
| 21:45:00 | efried | which would be fine, except... | |
| 21:45:13 | efried | it doesn't seem to be the same context that's being used for logging. | |
| 21:45:25 | mriedem | b/c the request id changes etc | |
| 21:45:34 | mriedem | which is because of a thread local store for the context way up in oslo.context | |
| 21:45:36 | mriedem | and it's mega confusing | |
| 21:45:53 | efried | rightright, but it's a *different* context? | |
| 21:46:02 | mriedem | as mentioned https://bugs.launchpad.net/nova/+bug/1835958/comments/7 | |
| 21:46:02 | openstack | Launchpad bug 1835958 in OpenStack Compute (nova) "Nova sync power state on large clusters causes poor performance" [Undecided,New] | |
| 21:46:21 | efried | viz: http://logs.openstack.org/86/672986/2/check/nova-live-migration/0171fa9/logs/screen-n-cpu.txt.gz#_Jul_30_22_09_35_334853 | |
| 21:46:22 | efried | here ^ you can see that the log line is showing one request_id, but the context we're using for the placement request has a different request_id. | |
| 21:47:58 | mriedem | req-428963d8-bc71-4823-8e35-97db13886690 is just some admin context created when the compute service starts up right? | |
| 21:48:20 | mriedem | https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L1321 | |
| 21:48:33 | mriedem | so you're using a random context there with a random request_id | |
| 21:48:50 | efried | req-42... is the placement-side local request_id, just ignore that one. | |
| 21:48:51 | mriedem | because https://github.com/openstack/oslo.context/blob/master/oslo_context/context.py#L273 | |
| 21:49:20 | mriedem | i was looking at that one because | |
| 21:49:20 | mriedem | Jul 30 22:09:35.334349 ubuntu-bionic-rax-ord-0009499845 nova-compute[28986]: INFO nova.scheduler.client.report [None req-fb42d5e6-1c0d-41d0-88c5-a4f1ac127a41 None None] [req-428963d8-bc71-4823-8e35-97db13886690] Created resource provider record via placement API for resource provider with UUID 5dcb9522-0513-440a-80db-d985b5d45704 and name ubuntu-bionic-rax-ord-0009499845. | |
| 21:49:35 | efried | req-fb42d5e6-1c0d-41d0-88c5-a4f1ac127a41 is the oslo.context one that oslo.log is paying attention to | |
| 21:49:35 | efried | req-8a7e2cc5-3870-4118-b55f-4533d7a4de80 is the one in the RequestContext being used for the placement communication. | |
| 21:50:07 | efried | yeah, I know, I had to track that one down as well. I started this whole thing because I was wondering why tf we needed to print that reqid at all. | |
| 21:50:37 | efried | it's the one being sent *back* to us in the placement response headers, which turns out to be the (not-global!) request_id on the placement side | |
| 21:50:51 | efried | and because of this whole snafu, it's currently the only way for us to correlate the requests on the nova & placement sides | |
| 21:51:04 | efried | but that flies in the face of the entire global_request_id concept | |
| 21:51:38 | efried | if we get global_request_id working, we don't need that at all because we'll be able to correlate with global_request_id, which is the whole friggin point. | |
| 21:51:48 | efried | so my question is this | |
| 21:52:21 | mriedem | idk what oslo.log is doing really but i know things get wonky with this https://github.com/openstack/oslo.context/blob/master/oslo_context/context.py#L300 | |