| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-09-20 | |||
| 12:55:41 | efried | sdague Reverted, anyway. It's possible an actual fix is in the works, based on jamespage comments in the bug. | |
| 12:56:35 | kashyap | Can someon remind, in what context are we using Ubuntu PPA (Personal Package Archives) | |
| 12:57:02 | sdague | kashyap: to get newer libvirt on xenial | |
| 12:57:03 | cdent | efried: thanks, I’m still relocating my landmarks | |
| 12:57:31 | kashyap | sdague: Is PPA considered 'official' - as in upstream Ubuntu maintainers maintain it? | |
| 12:57:37 | sdague | kashyap: yes | |
| 12:57:45 | sdague | this is the ocata ppa we're working with | |
| 12:57:52 | sdague | it's ubuntu supported | |
| 12:58:51 | kashyap | Ah, okay. For a moment, I assumed PPA in this context meant - "random unsupported packages done by a contributor, which is used only for personal testing" | |
| 13:02:14 | stephenfin | sahid: RE: https://review.openstack.org/#/c/501132/, could you add a summary of the comments with sean-k-mooney to the commit message? After that, it's an easy +2 | |
| 13:05:45 | alex_xu | nova api meeting is running at #openstack-meeting-4 | |
| 13:14:15 | openstackgerrit | Merged openstack/nova-specs master: Spec to remove the hide server address config options https://review.openstack.org/502516 | |
| 13:28:23 | openstackgerrit | Eric Fried proposed openstack/nova master: Use ksa adapter for placement conf & requests https://review.openstack.org/492247 | |
| 13:46:48 | openstackgerrit | Merged openstack/nova master: Fix a typo https://review.openstack.org/505062 | |
| 13:49:34 | openstackgerrit | Stephen Finucane proposed openstack/nova master: doc: Split flavors docs into admin and user guides https://review.openstack.org/501342 | |
| 13:49:35 | openstackgerrit | Stephen Finucane proposed openstack/nova master: doc: Add documentation for emulator_thread_policy https://review.openstack.org/501721 | |
| 13:49:35 | openstackgerrit | Stephen Finucane proposed openstack/nova master: doc: Add documentation for cpu_realtime, cpu_realtime_mask https://review.openstack.org/502056 | |
| 13:50:10 | stephenfin | sdague: Could you take a look at the first of those? gibi has reviewed it a few times but it keeps getting into merge conflicts :( | |
| 13:52:49 | sdague | +2 | |
| 13:52:55 | openstackgerrit | sahid proposed openstack/nova master: libvirt: bandwidth param should be set in guest migrate https://review.openstack.org/497455 | |
| 13:52:56 | openstackgerrit | sahid proposed openstack/nova master: libvirt: add method to configure migration speed https://review.openstack.org/497456 | |
| 13:52:56 | openstackgerrit | sahid proposed openstack/nova master: libvirt: slow live-migration to ensure network is ready https://review.openstack.org/497457 | |
| 13:52:56 | sdague | that all looks very reasonable | |
| 13:53:19 | stephenfin | sdague: Thank you, sir | |
| 13:53:20 | gibi | stephenfin, sdague: I'm also checking that rebase with an intent to approve it | |
| 13:53:34 | stephenfin | gibi: and you too :) | |
| 13:55:33 | stephenfin | sahid: Could you take a look at https://review.openstack.org/#/c/502056/ again? | |
| 13:56:05 | stephenfin | I get that we need to do more configuration that what's there, but there's a lot of stuff to do. I'd prefer to add an admin guide doc for that in the future | |
| 13:56:16 | stephenfin | ...which I should do sometime in the next few weeks | |
| 13:57:04 | mdbooth | 2017-09-14 15:54:39.689 120626 ERROR nova.s2017-09-14 15:54:39.690 120562 ERROR nova.servicegroup.drivers.db [-] Unexpected error while reporting service status | |
| 13:57:16 | gibi | stephenfin: +2+W | |
| 13:57:20 | openstackgerrit | OpenStack Proposal Bot proposed openstack/os-traits master: Updated from global requirements https://review.openstack.org/503646 | |
| 13:57:23 | openstackgerrit | OpenStack Proposal Bot proposed openstack/os-vif master: Updated from global requirements https://review.openstack.org/502708 | |
| 13:57:25 | stephenfin | gibi: Yay. Thanks :) | |
| 13:57:35 | mdbooth | With the continuation of the first log later in the file without its timestamp | |
| 13:57:43 | mdbooth | Are we not locking in the logger? | |
| 13:58:22 | sahid | stephenfin: the admin is going to configure the flavor, it seems reasonable to me to add a note saying what i do have mentioned on the review | |
| 13:58:42 | mdbooth | Unfortunately, it also means the logs are not correctly sorted :/ | |
| 13:58:58 | stephenfin | sahid: Right, but vcpu_pin_set is not a flavor property. I've mentioned pinned CPUs because it is | |
| 13:58:58 | openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: [WIP] CLI for aggregates https://review.openstack.org/505643 | |
| 13:59:03 | mdbooth | This is not conducive to merge sorting | |
| 13:59:37 | stephenfin | You also need to configure things like isolate the CPUs and use a properly configured guest, but I don't mention those there because they're nothing to do with configuring flavor properties | |
| 13:59:55 | sahid | isolate the CPUs? | |
| 14:00:13 | stephenfin | sahid: From the host? | |
| 14:00:29 | stephenfin | Whatever it is that replace the isolcpus boot parameter | |
| 14:00:31 | sahid | that is not related to Nova, i'm talking about an option which is related to Nova, that is why i think a small note is important | |
| 14:01:25 | stephenfin | and vcpu_pin_set is not related to flavors. This is an flavor (extra_spec) overview doc | |
| 14:01:35 | sahid | if you don't mention that an admin could just think that after to have configured the host, enabling cpu_realtime=yes in flavor is enough | |
| 14:02:41 | stephenfin | They could also boot an standard Linux guest kernel or forget to configure isolcpus | |
| 14:03:00 | stephenfin | The point is that this is just an overview of the flavor extra specs available and the interactions between them | |
| 14:03:18 | stephenfin | We should add a real-time doc but it should be separate, like this: | |
| 14:03:55 | stephenfin | https://docs.openstack.org/nova/latest/admin/cpu-topologies.html | |
| 14:04:18 | sahid | stephenfin: ok, i just gave to you my point, i would have added that note but feel free to not mention it | |
| 14:05:07 | stephenfin | sahid: Yup, and I appreciate it :) I'm countering that I don't think it's necessary here, and would make more sense in the upcoming larger doc | |
| 14:05:16 | stephenfin | ...where I'll definitely mention it | |
| 14:06:02 | mriedem | mdbooth: it's likely a problem in the customers log config | |
| 14:06:13 | mriedem | mdbooth: i saw something like that with our new super conductor logs in devstack, the fix for that was in devstack https://review.openstack.org/#/c/497944/1/lib/nova | |
| 14:06:22 | mriedem | but you should checkout what devstack does for log config | |
| 14:06:22 | mdbooth | mriedem: Looking. | |
| 14:08:03 | mdbooth | mriedem: Are you sure that's the same? It looks like 2 threads are writing simultaneously to the same log file. | |
| 14:08:36 | mdbooth | Hence a new log starts in the middle of the previous one, rather than on a separate line | |
| 14:08:39 | sdague | mdbooth: the python logger should handle that | |
| 14:09:51 | sdague | callers of the logger should not be locking around it, that's all supposed to be handled within the logger itself | |
| 14:10:07 | sdague | all oslo.log does is setup some common python logger patterns | |
| 14:10:08 | mdbooth | sdague: Yeah, that's what I'd have thought... | |
| 14:10:33 | mdbooth | I wonder how this has happened, though | |
| 14:10:51 | mdbooth | There are a ton of examples of it in these logs | |
| 14:11:18 | sdague | mdbooth: going through syslog? | |
| 14:11:30 | sdague | because syslog has some challenges | |
| 14:11:36 | mdbooth | Looks like it was generating DB errors continuously for a period of time, so lots of opportunity for overlap | |
| 14:11:49 | mdbooth | Do we normally log through syslog? | |
| 14:11:53 | toabctl | mriedem, hey. could you please have another look at https://review.openstack.org/#/c/398308/ ? | |
| 14:11:58 | sdague | no, we normally log to a file or stdout | |
| 14:12:37 | sdague | now, that being said, I expect if you are blowing through the output buffer regularly by putting giant stack traces all the time, you might end up with the tails of those going weird | |
| 14:12:51 | jamespage | sdague, efried: just to be clear, I don't think there is a fix to make in qemu - afaict its behaving as intended for the 2.10 release | |
| 14:12:51 | sdague | but that's probably a more latent python logging issue | |
| 14:13:07 | sdague | jamespage: yeh, it being another flag seems to indicate that | |
| 14:13:19 | sdague | jamespage: do you all have a patch already for it for your pike ppa on nova? | |
| 14:13:20 | mdbooth | Hmm, this is conductor not compute | |
| 14:13:34 | mdbooth | Are the workers independent? | |
| 14:13:48 | mdbooth | i.e. might they be separately opening the same log file? | |
| 14:14:26 | sdague | mdbooth: yes, the workers are processes | |
| 14:14:28 | jamespage | sdague: I have a simple patch to fix the nova package for Pike in Ubuntu and the UCA; that's not good for direct submission to nova as its not conditional i.e. its a blind add the flag (cause we know which qemu version will be in use for artful and xenial+ Pike UCA) | |
| 14:14:37 | mdbooth | sdague: I'll bet that's it... | |
| 14:14:41 | sdague | jamespage: gotcha | |
| 14:14:48 | mriedem | toabctl: done | |
| 14:15:02 | toabctl | mriedem, thx | |
| 14:15:06 | sdague | jamespage: link for where you injected it might be good regardless | |
| 14:15:17 | sdague | jamespage: then we can figure out the right conditional | |
| 14:15:30 | jamespage | sdague: looking the libvirt driver + images module to figure out the best way to pass that in conditionally - most version checking is done in driver, not in images... | |
| 14:15:34 | jamespage | sdague: | |
| 14:15:35 | jamespage | sure | |
| 14:15:56 | mriedem | jamespage: i was having the same problem when thinking about how to make that conditional | |
| 14:16:04 | mriedem | the driver knows the version, but way down in the bowels of the image code it doesn't | |
| 14:16:20 | jamespage | mriedem: yeah its awkward from that perspective | |
| 14:16:32 | jamespage | lemme attach my patch to the bug report | |
| 14:18:57 | mriedem | tasker: updated https://review.openstack.org/#/c/504260/ | |
| 14:20:09 | jamespage | mriedem: my thinking was to pass that down from driver to the image code as an optional param | |
| 14:20:46 | jamespage | testing that patch shortly | |
| 14:22:32 | mriedem | jamespage: yeah that is probably what i'd do | |