Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-09-04
13:44:42 sean-k-mooney pf affinity is for bonding. if you bound 2 sriov nics in the guest you want vf to have anti affinity at the pf level
13:45:04 jaypipes sean-k-mooney: NUMA is definitely not a trait...
13:45:12 jaypipes sean-k-mooney: you don't "have NUMA" or not.
13:45:36 sean-k-mooney jaypipes: well actully if your plathform only has one memory controler then you dont have numa
13:46:13 jaypipes sean-k-mooney: you know what I meant :)
13:46:42 sean-k-mooney :) yes i do
13:47:05 sean-k-mooney that should read unfortunetly i do :)
13:47:11 jaypipes hehe
13:47:43 jaypipes sean-k-mooney: before we get to Denver, I'd appreciate if you and ralonsoh can prioritize these placement features.
13:48:11 jaypipes sean-k-mooney: we're def not going to get to them all, and I'd like to work on the top priority items that actually have a chance of being completed in Queens
13:48:36 sean-k-mooney yep understood.
13:48:55 jaypipes sean-k-mooney: and remember that we already have two higher priorities in placement-land: getting all the move operations straight (migrations, etc) and fully supporting shared storage
13:48:55 sean-k-mooney ill try and see what once have the most bang for there buck
13:49:12 jaypipes sean-k-mooney: and integrating traits into the virt driver and scheduling layers./
13:49:21 jaypipes sean-k-mooney: cool, thanks man
13:50:05 sean-k-mooney i mainly added the vf selection topic becase fo the pci passthrough overhaul section.
13:50:33 jaypipes sean-k-mooney: you mean the generic device manager thing? :)
13:50:35 openstackgerrit Dmitry Tantsur proposed openstack/nova master: Correct examples in "Manage Compute services" documentation https://review.openstack.org/500551
13:50:57 dtantsur just saw a person hit by this ^^^
13:51:58 jaypipes dtantsur: +2
13:52:19 dtantsur thnx
13:55:00 stephenfin alex_xu: Just wait til you get the two of them in a room together :P
13:58:20 stephenfin gibi: Easy "kill dead code" patch here, if you fancied taking a look https://review.openstack.org/#/c/483030/
14:07:29 gibi stephenfin: looking...
14:09:37 sean-k-mooney jaypipes: yes the generic device manager thing + there are a few other spects related to sriov in general such as attach/detach support.
14:09:49 jaypipes yup
14:22:45 sbezverk sean-k-mooney : ping
14:24:20 sean-k-mooney sbezverk: hi
14:25:21 sbezverk sean-k-mooney hi, quick question for you if you have a second. Are you aware of any in depth libvirt debugging/troubleshooting guide?
14:25:29 openstackgerrit Merged openstack/os-traits master: Add a new parameter ``suffix`` to function ``get_traits`` https://review.openstack.org/483451
14:26:08 sean-k-mooney sbezverk: not as such no. are you trying to find a guide to debug problems using libvirt or developing libvirt
14:27:54 sbezverk sean-k-mooney I see "x-files" type of problem with one VM and need to dig way deeper in libvirt to understand what is going on
14:29:01 sean-k-mooney well libvirt iteslef does almost nothing excepth take an xml and convert it into a qemu ars list
14:29:38 sean-k-mooney perhaps thats a bit unfair to it but usually the issue is not in libvirt but in the thing its calling of the thing that is driving it
14:30:25 sean-k-mooney sbezverk: what is the problem you are having?
14:30:47 sbezverk sean-k-mooney you asked yourself ;)
14:31:33 sbezverk sean-k-mooney one vm gets paused without any particular reason, I checked everything I could no issue with storage, plenty in vm and on the host
14:32:04 sbezverk run extensive diagnostic on the host memory/cpu/disk subsystem comes clean
14:32:45 sbezverk I wanted to see what process/even signals to libvirt to pause that vm
14:34:55 sean-k-mooney ok is there anyting in the instance log in /var/log/libvirt/qemu/instance-<id>.log
14:34:56 sbezverk even answering whether it is triggered outside on VM or some internal thing in the VM is problematic as no logs indicate anything abnormal
14:35:34 sean-k-mooney if libvirt recived a api call to pause the vm it should be in the libvirtd log in /var/log/libvirtd.log
14:36:45 sbezverk sean-k-mooney ok thanks I will double check, I enabled debug level of log for libvirt
14:52:03 sbezverk sean-k-mooney what do you think about this http://paste.openstack.org/show/620346/
14:56:06 sean-k-mooney sbezverk: you would not happen to be trying to use a new intel purly plathform or new rayzen server would you
14:56:51 sean-k-mooney sbezverk: i have seen this before but only when you are trying to use a gust cpu model that uses feature not availabel on the host
14:57:09 sbezverk sean-k-mooney nope it is cisco ucs box with 2600 something v4
14:57:09 stephenfin sean-k-mooney, sbezverk
14:57:26 sean-k-mooney such as when you are using an old kvm/qemu that does not understand the plathform your are on
14:57:30 stephenfin sean-k-mooney, sbezverk: That does look very similar. I'm pretty sure there was a patch up somewhere to do with this
14:57:32 sean-k-mooney stephenfin: yes?
14:57:41 stephenfin sean-k-mooney: Sorry - git return too early :P
14:57:45 stephenfin *hit
14:59:24 sean-k-mooney sbezverk: one thing you could do to rule out that the cpu model was the issue is enablle host passhtrough by seting cpu_mode = host-passthrough in the libvirt section of /etc/nova/nova.conf
14:59:25 sbezverk stephenfin : do you have a link by any chance?
14:59:44 sbezverk it is pass through
14:59:50 sbezverk sean-k-mooney ^^^
15:00:05 sean-k-mooney really? huh that is strange indeed
15:00:59 sean-k-mooney sbezverk: well according to that log the host does not suport vmx or smx so you have not hadware virtualiation support
15:01:28 sean-k-mooney can you show me the output of cat /proc/cpuinfo and also check that /dev/kvm is presnet
15:01:34 sbezverk sean-k-mooney : :) there are 5 other vms happily running on the host
15:01:51 sean-k-mooney haha of course there is :)
15:02:45 sbezverk sean-k-mooney that is why I call it the x-files case ;)
15:03:10 sbezverk I need Mulder asap ;)
15:05:13 sbezverk sean-k-mooney cpu_mode=host-model and passthrough what is the difference?
15:06:33 sean-k-mooney host-model is baseically emulate the commen set of fetures for the generation of cpu the host has e.g. haswell host-passthrough mean dont fake the cpu at all and let the vm see exactly what the host has
15:07:01 sbezverk sean-k-mooney hmm, maybe it is worth a shot then
15:07:06 sbezverk http://paste.openstack.org/show/620348/
15:07:19 sbezverk here is cpuinfo from the host
15:07:58 sean-k-mooney when i ran an old qemu/libvirt on a skylake xeon host-model kept instinted it was a skylake-client(e.g. desktop) cpu but that cause a simmilar error to what you have with the missing featuers. host-passthrough fixed that
15:10:55 sbezverk sean-k-mooney cool, thanks a lot at least it is one step further!
15:12:43 sean-k-mooney sbezverk: virsh cpu-models x86_64 should tell you if it has a model for broadwell http://paste.openstack.org/show/620351/
15:14:22 openstackgerrit Merged openstack/nova master: Replaces uuid.uuid4 with uuidutils.generate_uuid() https://review.openstack.org/481007
15:14:32 sean-k-mooney you can see on that older system it does not konow about skylake at all but with newer libvirt/qemu it does.
15:19:06 sbezverk sean-k-mooney and 2690v4 is which architecture?
15:19:22 stephenfin sahid: Does your -1 apply to PS3 or PS2 here? https://review.openstack.org/#/c/465160/
15:19:51 sean-k-mooney sbezverk: broadwell http://ark.intel.com/products/91770/Intel-Xeon-Processor-E5-2690-v4-35M-Cache-2_60-GHz
15:20:17 sbezverk sean-k-mooney thanks a lot
15:21:01 sean-k-mooney oh 2000 grand a piece thats not a cheap cpu.
15:21:23 sean-k-mooney sorry 2 grand or 2000 not 2000 grand lol
15:21:59 sean-k-mooney that would be a very pricy cpu indeed
15:22:54 stephenfin dansmith: Could you take a look at https://review.openstack.org/#/q/topic:clean-numa-objects+status:open this week?
15:23:16 stephenfin sahid: nvm - think I get you now
15:25:51 openstackgerrit Merged openstack/nova master: Stop handling differences in registerCloseCallback https://review.openstack.org/416297
15:25:53 bauzas stephenfin: remember that US folks are in holiday today
15:26:12 stephenfin bauzas: Aha, right. My bad :)
15:26:25 openstackgerrit Merged openstack/nova master: Correct examples in "Manage Compute services" documentation https://review.openstack.org/500551
15:27:29 stephenfin bauzas: vocal?
15:27:51 stephenfin I should hope you're don't plan to be verbose. That'd be rather dull to listen to :P
15:27:59 stephenfin *you don't
15:30:50 abhi89 melwitt: Hi.. wanted to clarify some doubts on cells in openstack.. specifically _instances_cores_ram_count() which is added..
15:32:29 abhi89 melwitt: nova_context.scatter_gather_all_cells gives this {'00000000-0000-0000-0000-000000000000': {'project': {'instances': 0, 'ram': 0, 'cores': 0}}, '5a761e7e-a5ba-46af-a312-a9bc725991de': {'project': {'instances': 0, 'ram': 0, 'cores': 0}}} when no compute host is registered. what does "00000000-0000-0000-0000-000000000000" indicate here
15:32:53 abhi89 melwitt: cells are disable on my setup
15:33:55 sean-k-mooney bauzas: i have been putting off updateing my laptop for 2 years because i only use it at summits and eveytime i go i think i should update this but its too close to start now
15:55:48 openstackgerrit Jiri Suchomel proposed openstack/nova master: Retry few times when creating the initial connection to libvirtd. https://review.openstack.org/416928
16:04:23 openstackgerrit Stephen Finucane proposed openstack/nova master: hardware: Flatten functions https://review.openstack.org/367470
16:04:24 openstackgerrit Stephen Finucane proposed openstack/nova master: Standardize '_get_XXX_constraints' functions https://review.openstack.org/385071
16:04:24 openstackgerrit Stephen Finucane proposed openstack/nova master: Rename '_numa_get_constraints_XXX' functions https://review.openstack.org/385072
16:04:25 openstackgerrit Stephen Finucane proposed openstack/nova master: De-duplicate _numa_get_flavor_XXX_map_list https://review.openstack.org/385074
16:06:51 openstackgerrit Chris Dent proposed openstack/nova master: Update RT aggregate map less frequently https://review.openstack.org/489633

Earlier   Later