Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-03-04
16:31:17 kukacz_ dansmith: hmm, that's bad. then it seems I don't have a method to safely prevent non-ssd instances being scheduled on ssd node - thinking the classic example used in Nova docs for ages: https://docs.openstack.org/nova/queens/admin/configuration/schedulers.html
16:31:54 dansmith kukacz_: nova is optimized for cattle, not pets
16:32:48 dansmith if you're on something new enough you could potentially use the new isolated aggregates thing for your ssd aggregate
16:34:20 kukacz_ dansmith: just for my curiosity - how does the pets/cattle apply in this case?
16:34:52 dansmith kukacz: it's a pet if you can't shoot it in the head and boot another one in its place
16:35:20 dansmith https://docs.openstack.org/nova/latest/reference/isolate-aggregates.html
16:37:37 dansmith also if you're willing to shelve/unshelve, you could just resize those instances to a flavor that has the new key in it
16:38:56 kukacz_ dansmith: unfortunately, this is queens. isolated aggregates seem to be from rocky
16:39:13 dansmith train actually
16:39:53 kukacz_ dansmith: yes, the resize method is something I was just thinking of. probably a way to go. but it's not my workload. I'll have to ask many customers to follow that rule
16:40:03 dansmith yup
16:50:48 kukacz_ dansmith: one more detail - how is the original set of flavor parameters stored for each instance? in nova.instances table I see instance_type_id pointing at the current flavor. it must be something else, I guess ...
16:51:23 dansmith kukacz_: it's a serialized json blob in instance_extra.flavor
16:51:52 kukacz_ dansmith: aha!
16:51:59 kukacz_ dansmith: thank you a lot for all the clarification!
16:52:18 dansmith kukacz_: how come you're willing to shelve something but not resize it? resize should be faster and less disturbing to the workload
16:54:24 kukacz_ dansmith: it's not problem of willingness to resize. it's just that our customers use shelving to pause billing on temporarily stopped instances. now we'll have to explain them, that they'll also need to issue resize call to avoid their instance end up running on hosts where we don't want them to be
16:55:09 dansmith oh okay I thought maybe you were shelving proactively
16:56:19 openstackgerrit Lee Yarwood proposed openstack/nova stable/train: libvirt: Ignore DiskNotFound during update_available_resource https://review.opendev.org/711276
16:57:06 openstackgerrit Lee Yarwood proposed openstack/nova stable/stein: libvirt: Ignore DiskNotFound during update_available_resource https://review.opendev.org/711277
16:57:32 openstackgerrit Lee Yarwood proposed openstack/nova stable/rocky: libvirt: Ignore DiskNotFound during update_available_resource https://review.opendev.org/711278
16:58:04 openstackgerrit Lee Yarwood proposed openstack/nova stable/queens: libvirt: Ignore DiskNotFound during update_available_resource https://review.opendev.org/711279
16:58:06 melwitt stephenfin: what made you think https://review.opendev.org/711093 is blocking the gate?
16:58:54 stephenfin melwitt: We're referencing a module that's been deleted so we're broken with the latest version of oslo.db, no?
16:58:54 melwitt it was just a heads up about an upcoming requirements update and should get normal review
16:59:09 stephenfin ah, crap. I didn't realize it was pre-emptive
16:59:25 stephenfin I'll pull it out of the queue
16:59:31 melwitt yeah but I don't think the change to the latest happened yet, it's proposed. this is a standard thing that happens when requirements team looks to bump things
16:59:48 openstackgerrit Stephen Finucane proposed openstack/nova master: Remove oslo_db.sqlalchemy.compat reference https://review.opendev.org/711093
16:59:55 kukacz_ dansmith: no, as a service provider I mostly cannot touch customer instances, they manipulate them themselves
17:00:26 stephenfin melwitt: Gotcha. Yeah, sorry, missed that. Dragged it out ^
17:41:08 openstackgerrit Stephen Finucane proposed openstack/nova master: libvirt: Use domain capabilities to get supported device models https://review.opendev.org/666915
17:41:09 openstackgerrit Stephen Finucane proposed openstack/nova master: Add transform_image_metadata request filter https://review.opendev.org/665775
18:56:50 ozzzo can anyone help me set set pointer_model to None in nova.conf?
18:57:02 openstack Launchpad bug 1866106 in OpenStack Compute (nova) "Can't set "pointer_model = None" in nova.conf" [Undecided,New]
18:57:02 ozzzo it appears to be a bug; I opened https://bugs.launchpad.net/nova/+bug/1866106
18:57:20 ozzzo the help text in nova.conf says to use None or <None> but nova errors on both
18:57:32 ozzzo ERROR nova ConfigFileValueError: Value for option pointer_model is not valid: Valid values are [None, ps2mouse, usbtablet], but found 'None'
18:57:57 ozzzo I tried leaving it blank but that fails also
18:58:29 ozzzo the underlying issue is that I need to not load the USB and PS2 drivers to build large-RAM Centos VMs, because Centos chokes while loading drivers on VMs with over 1064304G RAM
19:06:58 melwitt ozzzo: I agree that looks like a bug. the choice should instead be 'none' (I see several other examples, especially in nova/conf/libvirt.py) and handled accordingly in the code. I think the StrOpt won't ever recognize None as a non-string 'None'
19:07:49 ozzzo ok trying none
19:08:08 melwitt ozzzo: I mean I think you'll need a patch to do it
19:08:15 ozzzo oic ok
19:08:38 melwitt it needs to be fixed in order for you to choose none
19:10:08 melwitt lemme see if I can propose one real quick for you
19:10:23 ozzzo ok cool thank you!
19:10:44 melwitt thanks for opening the bug and letting us know
19:11:02 ozzzo no problem, my users are screaming for giant memory Centos VMs
19:11:03 mriedem the config is coercing None to 'None'
19:11:14 ozzzo I'm trying to get them to use Ironic instead but they don't want to
19:11:45 mriedem seems like that might be a bug in oslo.config rather than nova
19:13:07 ozzzo should I ask in their IRC channel?
19:13:48 bnemec The only way for a config opt to have a None value is for that to be the default and for the opt to be unset.
19:14:14 bnemec So completely absent from the file, not something like "opt="
19:14:18 mriedem bingo https://github.com/openstack/oslo.config/blob/master/oslo_config/types.py#L142
19:14:29 mriedem yeah the default is usbtablet
19:14:34 mriedem so you can't unset it to avoid the default
19:14:53 ozzzo am I on the right track; will setting this to none stop the USB driver from loading?
19:15:12 mriedem so you're looking at what? default being None, but then checking to see if the option is explicitly set to None or default to 'usbtablet' if not set to None explicitly?
19:15:39 ozzzo or is there a better way to not load USB?
19:16:11 mriedem bnemec: couldn't oslo.config just fix this line? https://github.com/openstack/oslo.config/blob/master/oslo_config/types.py#L142
19:16:23 mriedem if value is None and None is a valid choice and not the default, then don't coerce to 'None'
19:18:19 bnemec I don't think it would help. I'm pretty sure the argument parser will never return None, it will return a string "None" anyway.
19:18:41 bnemec I actually think the cast is unnecessary because I'm pretty sure the raw values we get are all strings to begin with.
19:19:01 bnemec I could be wrong though. It's been a while since I've dug into the internals of oslo.config.
19:19:53 mriedem yeah maybe
19:20:44 mriedem guess you could hack a new choice "unset" or something to mean None
19:22:28 ozzzo my actual goal is to not load the USB and PS2 drivers. Is there a better way?
19:23:23 mriedem no, that's why i'm saying it sounds like you need a new choice on that option
19:26:47 mriedem i think that would basically take care of itself here https://github.com/openstack/nova/blob/757fc03b78d542e7262343b65eacea02ce11dd04/nova/virt/libvirt/driver.py#L5949
19:27:13 mriedem that method would return None so a pointer device wouldn't be set in the guest xml https://github.com/openstack/nova/blob/757fc03b78d542e7262343b65eacea02ce11dd04/nova/virt/libvirt/driver.py#L5700
19:27:55 mriedem a choice of 'none' like melwitt suggested would do it
19:28:17 openstackgerrit Merged openstack/nova master: Add test coverage of existing os-console-auth-tokens policies https://review.opendev.org/706687
19:29:02 mordred mriedem: but if you don't pick usb or ps2, how will your keyboard work?
19:29:26 melwitt I'm working on a patch now. writing up the commit message
19:31:23 ozzzo these giant-memory VMs are running batch jobs; I may be wrong but I think we can live without a keyboard
19:31:23 dansmith mordred: ten digit rotary phone dial
19:31:34 dansmith mriedem: hack in the characters via their decimal ascii equivalent... duh.
19:32:01 dansmith er, mordred ^ :P
19:32:36 mriedem cripes
19:34:57 openstackgerrit melanie witt proposed openstack/nova master: Fix CONF.pointer_model choice None to use 'none' for StrOpt https://review.opendev.org/711315
19:38:10 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Introduce scope_types in os-console-auth-tokens https://review.opendev.org/706688
19:39:42 melwitt gmann: hey I was wondering if you saw johnthetubaguy comment on this patch https://review.opendev.org/705127 I couldn't really follow it
19:41:36 gmann melwitt: yeah, i can move those to deprecated also, i did not do initially as those were ~3 years old and users might have switched to new one.
19:41:40 gmann updating..
19:42:16 melwitt gmann: ok. since I didn't understand it I didn't agree or disagree, but wondered what you thought about it
19:48:14 gmann while new policy were introduced, we did not remove those old policy from enforcement from our API so operators might be relying on those old policies default or override.
19:48:31 gmann keeping them for deprecation phase does not harm much.
20:01:18 melwitt ozzzo, mriedem: I'm weirdly noticing in the code that setting pointer_model='ps2mouse' looks like it would have the same effect (nothing added to libvirt xml) as setting None if setting None were possible
20:02:19 mriedem i was wondering about that too
20:02:43 ozzzo interesting; I'll try it and then look at the XML
20:02:52 mriedem that pointer_model option was meant to replace use_usb_tablet back in newton but it was never fully removed because the upgrade compat path was really complicated (i tried and abandoned the attempt)
20:03:07 mriedem i wonder if there was a bad refactor at some point that broke the ps2mouse support
20:03:28 melwitt I was wondering the same thing
20:03:53 mriedem note that devstack sets it to ps2mouse explicitly https://github.com/openstack/devstack/blob/93d22d82988ee781feddf84535d65adf44472379/lib/nova_plugins/hypervisor-libvirt#L44
20:04:05 mriedem so you could just check one of the gate job logs for a guest xml to see if the pointer device is not set
20:04:46 melwitt oh nice
20:07:17 mriedem code looks basically the same since newton https://github.com/openstack/nova/blob/newton-eol/nova/virt/libvirt/driver.py#L4675
20:07:50 melwitt here's a guest xml https://zuul.opendev.org/t/openstack/build/fcc8380b76924d0ca8acf158f7571ad8/log/controller/logs/screen-n-cpu.txt#1888

Earlier   Later