Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-05-23
17:19:06 efried one of them is. the other could be swapped out as it stands.
17:20:52 efried You could juice up patch_exists so the path arg can be a str or a regex :)
17:21:03 efried that would satisfy the other one.
17:22:22 efried oh, no it wouldn't.
17:22:36 efried aspiers: You could make it a str or a callable...
17:22:47 efried starts to smell a lot like side_effect at that point
17:27:50 aspiers efried: yeah, I was thinking of supporting callables, but ... scope creep, I dunno
17:31:41 efried aspiers: Feel free to push it to the wishlist for later. Add a `TODO(aspiers): figure out how to use patch_exists here` and move on.
17:35:38 openstackgerrit Merged openstack/nova stable/stein: Handle placement error during re-schedule https://review.opendev.org/657602
17:35:48 openstackgerrit Merged openstack/nova stable/stein: Only call _fill_provider_mapping if claim succeeds https://review.opendev.org/657603
17:36:53 aspiers efried: good idea
17:42:53 openstackgerrit Merged openstack/nova stable/stein: Stop logging traceback when skipping quiesce https://review.opendev.org/660325
18:32:16 ganso lyarwood: Hi! Could you please take a look at this patch? https://review.opendev.org/#/c/659338 Thanks in advance!
18:43:46 openstackgerrit melanie witt proposed openstack/nova master: Count instances from mappings and cores/ram from placement https://review.opendev.org/638073
18:43:46 openstackgerrit melanie witt proposed openstack/nova master: Set [quota]count_usage_from_placement = True in nova-next https://review.opendev.org/653146
18:43:47 openstackgerrit melanie witt proposed openstack/nova master: Use instance mappings to count server group members https://review.opendev.org/638324
18:43:47 openstackgerrit melanie witt proposed openstack/nova master: Add documentation for counting quota usage from placement https://review.opendev.org/653845
19:04:02 tasker I'm running Pike and trying to turn the logging level down to WARNING ( instead of the default INFO ), and "verbose = False" no longer works. The documentation at pike/admin/manage-logs.html and pike/configuration/config.html provide non-matching information.
19:04:45 tasker one ( admin ) says to use "log-config" while the other ( configuration ) says "log-config" is deprecated and that all other log items will be ignored if the new "log_config_append" is used.
19:04:57 tasker what's the easiest way to turn Nova's output down to WARNING?
19:34:10 sean-k-mooney efried: do you know what the policy on deprecating parts of config vaules is
19:34:39 sean-k-mooney efried: i want to deprecate and remove one of the ways of whitelisting device in the pci_white_list this cycle
19:34:51 sean-k-mooney efried: specificly the devicename parmater
19:35:09 efried that's interesting.
19:35:41 sean-k-mooney would it need a blueprint and or spec? or jsut a bug ?
19:35:53 efried For something like that, you would just have to note the deprecation in the help text of the opt and in a reno; and add a warning log to the code that parses it.
19:35:58 efried oh, you mean *paperwork*
19:36:23 efried depends, is there a bug? :)
19:36:28 sean-k-mooney ya so we woudl need to keep it around for train and log a warning if its used
19:36:47 sean-k-mooney not for this but there are bugs caused by that parmaater
19:36:50 efried I would think so, yes.
19:37:32 sean-k-mooney the issue is if your kernel is configured not to bind back the PFs/VFs to the nic driver e.g. they stay boudn to vfio
19:37:52 sean-k-mooney then if you restart the compute agent it cant dicover the VFs or PFs again
19:38:21 sean-k-mooney because the PF/VF does not have a netdev and therfore you cant find it by name
19:38:22 efried And this is the best way to "fix" the bug? Deprecating/removing the ability to specify a device by name?
19:38:33 sean-k-mooney but also the name can somethime change after it rebound
19:38:42 sean-k-mooney efried: ya basically
19:38:46 efried are there not scenarios where identifying a device by name *doesn't* break?
19:39:02 efried (s/not// for same meaning with fewer negatives)
19:39:21 sean-k-mooney it only works if the device is reliable bound back to the nic driver when nolonger used by the guest and the name does not change whne it rebound to the host
19:39:37 sean-k-mooney so it can work but its really fragile
19:40:28 sean-k-mooney using vendor id and product id or useing the pci address are both more robost as they dont depned on teh driver or the semanitc of the nic vendor
19:40:39 efried sean-k-mooney: I guess what I would do in this case is write the patch and start a thread on the ML with your intent, soliciting opinions on whether this is okay to kill. Link the patch and ML thread from each other. IMO that's sufficient paperwork. And if someone thinks otherwise, you can always write a bug/bp later.
19:41:21 efried though it sounds like what you're talking about actually *is* a bug, and this is just one possible remedy.
19:41:33 sean-k-mooney efried: ok sound good.
19:41:54 efried In my ignorance, another remedy could be "blow up if devname is used for devices it has the potential to break on"
19:42:02 sean-k-mooney it depend on how you configre your driver at the kernel levle
19:42:17 sean-k-mooney and to some extent the nic vendor
19:42:20 efried no way to detect it, other than going all the way through until you hit the bug??
19:42:47 sean-k-mooney i belive intel nics prefer the networkign driver over vfio-pci but i think it depend on the model to some extend and your udev rules
19:43:57 sean-k-mooney efried: the main issue comes down to will udev/systemd chose to bind the driver to i40e or leave it bound to vfio-pci
19:44:27 sean-k-mooney so its not really a nova issue
19:44:43 sean-k-mooney its more a distro/packaging/install/confituion thing on the host
19:45:13 sean-k-mooney which is what make it hard for nova to dectet and or warn about
19:45:15 efried tasker: Does
19:45:15 efried [DEFAULT]
19:45:15 efried default_log_levels = nova=WARNING
19:45:15 efried not work?
19:46:00 efried sean-k-mooney: ack, and shrug. I hope we can soon deprecate *all* of [pci]passthrough_whitelist and [pci]alias.
19:46:14 sean-k-mooney efried: maybe in U
19:46:37 sean-k-mooney of course then we woudl have a cyborg whitelist and device profile
19:46:46 sean-k-mooney but then its there problem :)
19:46:50 efried heh
19:47:24 sean-k-mooney if we get the cyborg integration done fully this cycle that might be a nice thing to port over early next cycle
19:48:09 efried sean-k-mooney: cyborg/fpga + providers.yaml we might have all we need.
19:48:41 sean-k-mooney well i was thinig more generic pci passhtough rather then smart nic
19:49:29 sean-k-mooney e.g. replaceing all the uses of the pci tracker in nova with a very basic generic pci driver in cyborg
19:50:13 sean-k-mooney but making that work with hardare offload ovs and all the other legacy things we have would be a lot of work
19:50:31 sean-k-mooney manyly on the nova side to make it transparent
19:51:28 sean-k-mooney efried: anyway thanks ill send an email to the list
19:51:49 efried coo
20:02:03 tasker efried: did not. I still see a bunch of "HEAD / => generated 0 bytes"
20:02:40 efried tasker: Coming from nova?
20:03:21 tasker yes. specifically the "nova-api-os-compute" container ( if that distinction matters. )
20:04:03 efried tasker: Can you paste me the entire log line please?
20:04:32 efried I'm thinking that might be coming from the wsgi layer
20:04:47 tasker http://dpaste.com/2C5P2PY
20:05:31 tasker I concur. to complicate things, I don't know how to turn that thing down, either.
20:05:33 efried in which case you may wish to try adding ,eventlet.wsgi.server=WARNING
20:06:59 efried I think the left-hand-side of that '=' is a substring match. So if you say foo.bar=WARNING then log lines coming from foo.bar and foo.bar.baz and foo.bar.etc will all get filtered at a WARNING level.
20:08:00 efried hmph, no indication in that line where it's coming from. Is there a log line template that will add that? Looking...
20:09:49 efried tasker: Looks like you might be able to add %(pathname)s to your logging_default_format_string to get a clue where that line is being generated from; then use the prefix trick above to suppress it.
20:17:54 tasker efried: did not work. "%(pathname)s" was applied to another set of messages, but not the one I'm trying to filter out.
20:18:20 efried interesting
20:18:56 tasker wait .. scratch that. it wasn't applied at all.
20:19:12 efried assume you're restarting services each time
20:19:31 tasker yup.
20:19:58 tasker http://dpaste.com/1ZSB78D -- what the "wsgi" logs look like from "nova-api-wsgi.log" when all testing options are removed. this is the original behaviour.
20:20:14 efried guess you could try adding the pathname to logging_debug_format_suffix
20:20:29 tasker when I set anything to the config file, the lines in nova-api-wsgi.log stop, but not to nova-api-os-compute.log
20:20:53 tasker the latter file contains lines of the first paste I sent you.
20:21:52 efried hmph. I'm afraid I'm out of ideas.
20:22:01 efried Course, I'm not what you'd call an "operator".
20:22:12 tasker hehe. can we bring back "verbose = False"?
20:23:34 tasker efried: I appreciate you taking the time to help me out. You've shed some light on some things I did not know and given me a lot of good information to play around with.
20:23:59 efried you're welcome, sorry I couldn't be more help.
20:30:29 tasker I did match the "pid" in the log file to the PIDs of the uWSGI helpers that spawn whenever the nova service starts.
20:30:40 tasker that lends weight to it being the WSGI module(s).
20:31:03 tasker are the modules using the same file independent of Nova?
20:32:08 efried tasker: You're out of my wheelhouse at this point. Throwing it around earlier like I knew what I was talking about, but I can barely spell WSGI.

Earlier   Later