| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-12-17 | |||
| 14:10:01 | spatel | hmm! how do i check that? | |
| 14:10:10 | spatel | sean-k-mooney: ^ | |
| 14:10:10 | sean-k-mooney | as a result there is not netdev created for the VF and you get that warning | |
| 14:10:53 | sean-k-mooney | you coudl check in sysfs what driver one of the vfs e.g. 03:09.4 is boudn too | |
| 14:11:10 | spatel | let me check | |
| 14:12:25 | spatel | /sys/bus/pci/devices/0000\:03\:09.4/ which dir i should check here for that? | |
| 14:13:14 | spatel | I think this one? cat /sys/bus/pci/devices/0000\:03\:09.4/driver/bind | |
| 14:13:26 | sean-k-mooney | ls -al /sys/devices/pci0000\:00/0000\:03\:09.4/ | |
| 14:13:58 | sean-k-mooney | the driver director should be a symlink | |
| 14:14:07 | sean-k-mooney | it will look something like this driver -> ../../../bus/pci/drivers/e1000e | |
| 14:14:31 | spatel | oh! yes its /sys/bus/pci/devices/0000:03:09.4/driver -> ../../../../bus/pci/drivers/vfio-pci | |
| 14:15:06 | sean-k-mooney | if you checked ls -al /sys/devices/pci0000\:00/0000\:03\:09.4/net it either would not exist or be empty is my guess | |
| 14:15:16 | sean-k-mooney | so ya the warning is expected | |
| 14:16:51 | spatel | yes /sys/bus/pci/devices/0000:03:09.4/net: No such file or directory | |
| 14:17:34 | sean-k-mooney | right so the PF has virtual functions that are bount to vfio-pci which does not create netdevs for the VFs | |
| 14:18:07 | spatel | hmm! | |
| 14:18:28 | spatel | so no way we reduce that logging except filter in syslog right? | |
| 14:18:38 | spatel | its just filling my disk with that warning :) | |
| 14:18:46 | sean-k-mooney | you can filter in nova | |
| 14:18:56 | spatel | how? | |
| 14:19:16 | sean-k-mooney | ill get the config option one sec | |
| 14:20:06 | sean-k-mooney | https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.default_log_levels | |
| 14:20:12 | spatel | reading... | |
| 14:21:02 | sean-k-mooney | you should be able to add nova.pci.utils=ERROR | |
| 14:21:27 | spatel | wow! that would be amazing :) | |
| 14:21:37 | spatel | i didn't know you can filter log by nova applicatoon | |
| 14:21:44 | sean-k-mooney | not you will have to set all the default too | |
| 14:21:44 | spatel | application* | |
| 14:21:49 | sean-k-mooney | *note | |
| 14:22:21 | sean-k-mooney | we use pythongs loggin framework so you can filter logs by module | |
| 14:22:34 | spatel | so i have to set all default list in nova.conf including nova.pci.utils=ERROR right? | |
| 14:23:21 | sean-k-mooney | i think so that said i have never tried to limit logs from nova its self but i think that will work | |
| 14:23:36 | spatel | i will give it a try on lab :) | |
| 14:23:44 | spatel | thanks for the tip :) | |
| 14:24:11 | spatel | sean-k-mooney: check this out https://bugs.launchpad.net/horizon/+bug/1807251 | |
| 14:24:11 | openstack | Launchpad bug 1807251 in OpenStack Dashboard (Horizon) "Horizon Overview summary showing wrong numbers " [Undecided,Incomplete] | |
| 14:26:44 | spatel | sean-k-mooney: you can close this issue with your comments https://bugs.launchpad.net/nova/+bug/1808738 | |
| 14:26:44 | openstack | Launchpad bug 1808738 in OpenStack Compute (nova) "No net device was found for VF" [Undecided,New] | |
| 14:27:29 | openstackgerrit | Jack Ding proposed openstack/nova master: Improve libvirt image and snapshot handling https://review.openstack.org/616692 | |
| 14:28:07 | sean-k-mooney | im triaging it currently as low as the wraning is noisy as you said and does not explain why it happens. | |
| 14:28:26 | sean-k-mooney | but it can fill your disk which is annoying. | |
| 14:29:17 | spatel | sean-k-mooney: it would be good if adjust code to understand behavior and don't sent WARNING | |
| 14:29:24 | openstackgerrit | Jack Ding proposed openstack/nova master: Correct instance port binding for rebuilds/reboots https://review.openstack.org/603844 | |
| 14:31:27 | sean-k-mooney | spatel: i was thinking of bumping it down to debug | |
| 14:31:54 | sean-k-mooney | spatel: ther freatre that need the netdev to exist but none that are critical | |
| 14:32:25 | sean-k-mooney | the main one being nic feature based schduling which is only half implemnted | |
| 14:32:46 | sean-k-mooney | so its not very usfull currently since it does not work | |
| 14:33:59 | sean-k-mooney | you may also be unable to use vnic_type=macvtap | |
| 14:37:11 | mriedem | gibi: we have a runways queue spot open https://etherpad.openstack.org/p/nova-runways-stein but i'm not sure if we should put your 2nd mention of https://blueprints.launchpad.net/nova/+spec/bandwidth-resource-provider in there or just assume it will be covered by the existing entry? | |
| 14:37:23 | mriedem | they are all in a series so i'm not sure why you had that in the etherpad twice | |
| 14:49:17 | lyarwood | mriedem: https://review.openstack.org/#/c/579004/6/nova/virt/block_device.py - I'm trying to follow up on your comments here, quick question if you have a second. In the case where the BDM doesn't have an attachment_id can we just create one or do we still need to use the v2 APIs? | |
| 14:50:23 | lyarwood | cinderv2 is deprecated but I'm not sure if the intention here was to always assume that we had access to cinderv3 when updating the connection_info here or not. | |
| 14:50:47 | mriedem | cinder v3.0 is backward compatible with v2 apis | |
| 14:51:07 | mriedem | so if we have a legacy attachment (no attachment_id in the bdm) we can use the old initialize_connection flow | |
| 14:52:08 | mriedem | converting legacy attachments to "new style" attachments in refresh_connection_info is an interesting thought, | |
| 14:52:18 | mriedem | but since this is something that needs to be backported, i would avoid that in this change | |
| 14:53:09 | mriedem | we definitely do need to get code in to start migrating the old style attachments ( https://review.openstack.org/#/c/541420/ https://review.openstack.org/#/c/549130/ ) | |
| 14:53:17 | mriedem | otherwise we'll never be able to drop those old flows | |
| 14:54:38 | lyarwood | ack yeah thanks just had to remind myself about the v2/v3 compat, that somehow slipped my mind. | |
| 14:54:53 | lyarwood | and yeah this is getting backported if it ever lands | |
| 14:55:01 | lyarwood | so converting isn't helpful here yet | |
| 14:55:15 | lyarwood | maybe in a follow up, unless you're already doing that in the above changes | |
| 15:01:42 | mriedem | i'm not | |
| 15:02:10 | mriedem | we really have two things that need to be done: 1. bfv where nova-compute creates new volumes needs to use the new flow, that's the first patch above, | |
| 15:02:23 | mriedem | 2. migrate existing attachments, which can only be done on the compute b/c we need the host connector | |
| 15:02:28 | mriedem | that's the 2nd patch | |
| 15:02:54 | mriedem | it does that on n-cpu startup in a separate thread on each compute using a specialized db query to try and make this as targeted as possible, because i think that's what we talked about doing in dublin, | |
| 15:03:37 | mriedem | but in berlin there were suggestions of just migrating on first access of a legacy attachment, like maybe during move operations or reboot (like refresh_connection_info), | |
| 15:03:44 | mriedem | so idk where we are on migrating existing attachments | |
| 15:03:53 | gibi | mriedem: regarding runways. You are correct, everything related to bandwidth is in a single chain of commits. I just finished the second set of commits _after_ the first set was already in the queue | |
| 15:04:06 | mriedem | we need to have some kind of data migration though otherwise we can't ever drop the old code paths | |
| 15:04:09 | gibi | mriedem: and I did not want to change an item in the queue | |
| 15:04:25 | spatel | sean-k-mooney: sorry i was in meeting my IRC client kick me out | |
| 15:04:41 | mriedem | gibi: ok. fwiw when i'm looking at the runways queues i'm just going through the series until i can't go anymore | |
| 15:04:52 | mriedem | but maybe you're just trying to chunk these up since it's a lot of changes | |
| 15:04:55 | lyarwood | mriedem: yeah I'd be in favor of the original 2 tbh, otherwise who knows when the attachments would be migrated if ever. | |
| 15:05:12 | sean-k-mooney | spatel: i triaged https://bugs.launchpad.net/nova/+bug/1808738 | |
| 15:05:12 | openstack | Launchpad bug 1808738 in OpenStack Compute (nova) "No net device was found for VF" [Low,Triaged] | |
| 15:05:27 | gibi | mriedem: I'm OK to merge the two bwm entry into a single one | |
| 15:05:35 | mriedem | gibi: also, ftr, one of my goals this week (before i'm out next week) is to get through your bw provider series; i'm re-reading the spec now | |
| 15:05:36 | sean-k-mooney | spatel: the horizon bug seams to be waiting on there responce after you followed up | |
| 15:05:37 | gibi | mriedem: it will be a looong one | |
| 15:05:58 | mriedem | gibi: ok i merged them in | |
| 15:06:03 | gibi | mriedem: thanks | |
| 15:06:06 | mriedem | gibi: are you around this week? | |
| 15:06:23 | gibi | mriedem: I'm available until end of Thursday | |
| 15:06:45 | brandor5 | hello everyone: Is there a cli command to list the # of instances on a host? | |
| 15:06:54 | mriedem | ok, that works. i have to get my end of year appraisal stuff done by eow too. :( | |
| 15:07:08 | sean-k-mooney | brandor5: yes i belive so one sec | |
| 15:07:20 | gibi | mriedem: on Friday I start my vacation and I will be back on 7th | |
| 15:07:24 | brandor5 | sean-k-mooney: cool, thank you very much | |
| 15:07:38 | sean-k-mooney | brandor5: openstack server list --host <hostname> | |
| 15:08:01 | sean-k-mooney | brandor5: you will need to be an admin i think to run that | |
| 15:08:15 | mriedem | brandor5: i assume you're more looking for something like this https://docs.openstack.org/python-novaclient/latest/cli/nova.html#nova-hypervisor-show | |
| 15:08:21 | gibi | mriedem: tomorrow I will be on and off during my afternoon but I will prioritize the bandwidth stuff if any | |
| 15:08:45 | brandor5 | sean-k-mooney: that one I knew about, sorry I should have been more clear... print list of hypervisors with the # of instacnes they have running on them | |
| 15:09:00 | sean-k-mooney | brandor5:actully you will need to add --all-projects "openstack server list --all-projects --host <hostname>" | |
| 15:09:09 | mriedem | brandor5: https://docs.openstack.org/python-novaclient/latest/cli/nova.html#nova-hypervisor-stats | |
| 15:09:31 | sean-k-mooney | oh yes there is ^ | |
| 15:09:59 | sean-k-mooney | you can run "openstack hypervisor stats show" | |