| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-08-16 | |||
| 12:36:03 | mdbooth | Anything exercising that in py3 will hit it, because it's basically a syntax error. Seems nothing was previously. | |
| 12:36:16 | efried | Right, my point. | |
| 12:36:26 | kosamara | Eg PowerVM -> \X{4}, libvirt -> \x{4} | |
| 12:37:24 | kosamara | I would like to say that the value format is the driver's responsibility as well, but there needs to be a common schema. | |
| 12:37:27 | efried | kosamara: Those values come from the hardware and their form (32-bit integer) is ordained by a standard. | |
| 12:37:55 | mdbooth | Ooh, timestamps are clicky: http://logs.openstack.org/33/591733/2/check/nova-tox-functional-py35/c3d298a/job-output.txt.gz#_2018-08-15_14_21_38_077216 | |
| 12:38:19 | kosamara | Let me double check | |
| 12:39:38 | efried | kosamara: we could ask for it to be represented as an integer via the schema, so you could do 0xC001, 0xc001, 0140001 (octal), 49153 (decimal), etc. | |
| 12:40:18 | kosamara | efried: libvirt reports \x{4} | |
| 12:41:39 | kosamara | efried: solves the problem, but sounds confusing for the operator | |
| 12:42:11 | efried | kosamara: I agree. | |
| 12:42:18 | kosamara | efried: I would prefer to dictate an arbitrary format \X{4} and then it will be the driver's responsibility to normalize it. | |
| 12:43:35 | efried | kosamara: I'm fine with that. Note that yaml tries to outsmart you on this, though. It interprets e.g. C001 (without quotes) as a string, but 4321 (without quotes) as an integer base 10. | |
| 12:44:05 | efried | kosamara: So we should document that these hex values be quoted. | |
| 12:46:25 | efried | mdbooth: Is your test case really valid if it's trying to fetch a nonexistent image? Maybe we're better off fixing the fake by taking off the default to .get() to force the test author to set it up. | |
| 12:47:11 | efried | kosamara: Hm, which makes me wonder whether I did my schema validation correctly in https://review.openstack.org/#/c/579289/ | |
| 12:47:34 | efried | because I remember running across this problem. | |
| 12:47:59 | efried | but now I can't remember if the test failed because 4321 got interpreted as an int or because it failed schema validation because not a string. | |
| 12:48:10 | efried | I would hope the latter. Need to add a test. | |
| 13:04:14 | kosamara | efried: nice, thanks for the tip! | |
| 13:07:09 | mdbooth | efried: My test case isn't concerned with images. | |
| 13:07:20 | mdbooth | efried: Except that it needs to boot an instance which has disks. | |
| 13:17:19 | efried | mdbooth: I can fix the commit message and +2. | |
| 13:19:44 | openstackgerrit | Eric Fried proposed openstack/nova master: Py3 fix in fake image service https://review.openstack.org/592056 | |
| 13:20:46 | efried | mdbooth: Done ^ | |
| 13:21:29 | tobias-urdin | have a weird issue, i have some instances stuck in building (scheduling) state which show up in dashboard and with api, but server show throws NotFound exception so I cant remove them, anyubody? i've restarted nova-api, scheduler and conductors. | |
| 13:24:24 | mdbooth | efried: Thanks | |
| 13:24:43 | openstack | bug 1550919 in OpenStack Compute (nova) "[Libvirt]Evacuate fail may cause disk image be deleted" [Medium,In progress] https://launchpad.net/bugs/1550919 - Assigned to Matthew Booth (mbooth-9) | |
| 13:24:43 | openstackgerrit | Matthew Booth proposed openstack/nova master: Add regression test for bug 1550919 https://review.openstack.org/591733 | |
| 13:25:27 | openstackgerrit | Matthew Booth proposed openstack/nova master: Don't delete disks on shared storage during evacuate https://review.openstack.org/578846 | |
| 13:53:35 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: Making instance listing skipping down cells configurable https://review.openstack.org/592428 | |
| 13:55:52 | tssurya | mriedem: ^ | |
| 13:59:40 | kosamara | efried: What does PowerVM want to do, if the YAML it parses includes identification keys it doesn't understand? | |
| 14:00:31 | efried | kosamara: Not sure. | |
| 14:00:39 | efried | kosamara: Options: ignore, log, error. | |
| 14:00:50 | kosamara | efried: I'm tending towards making it an error in libvirt, with the assumption that the operator has made an oversight and will be baffled to see his devices not recognized if it's just ignored. | |
| 14:00:58 | efried | Right | |
| 14:01:06 | efried | that's a reasonable argument. | |
| 14:01:24 | kosamara | Should it be enforced by the spec for all drivers though? | |
| 14:01:30 | efried | Because the conf is going to have to be host-specific, which means driver-specific. | |
| 14:01:53 | efried | kosamara: But my hesitation is this: | |
| 14:02:43 | efried | At some point in the future, we might like to support a global conf on the conductor, whose values percolate down to the computes, which can override (optionally) via local configs of their own. | |
| 14:03:03 | efried | In that scenario, the global conf would presumably want to include the union of all the relevant keys. | |
| 14:03:13 | efried | and the computes would have to ignore the ones it doesn't understand. | |
| 14:03:19 | efried | But maybe we can solve that when we get there. | |
| 14:04:40 | tssurya | dansmith, melwitt: FYI, mriedem has some stuff on the cells meeting agenda | |
| 14:04:58 | dansmith | tssurya: for next week? | |
| 14:05:20 | tssurya | heh not really, just up for open discussion/opinions I guess | |
| 14:05:27 | efried | kosamara: As for attempting to define the same rule for all drivers, I think that's a good idea for consistency. | |
| 14:16:05 | sean-k-mooney | kosamara: i would generally try to minimise divergnce options in dirver | |
| 14:17:16 | sean-k-mooney | efried: host-specfic doen not inherintly mean driver-specific | |
| 14:17:58 | efried | sean-k-mooney: Can we have more than one nova-compute service running on a single host? Or more than one driver loaded in a single nova-compute process? | |
| 14:20:34 | sean-k-mooney | efried: yes we can have more then one nova compute on the same host | |
| 14:20:46 | sean-k-mooney | efried: normally only done with ironic | |
| 14:21:29 | sean-k-mooney | we can only have 1 driver loaded as far as i am aware | |
| 14:21:37 | efried | That sounds really confusing. Are they reporting different host resources? | |
| 14:22:03 | efried | or the compute services each "own" a different subset of the ironic nodes? | |
| 14:22:25 | sean-k-mooney | the compute serveice each own a subset of the ironic nodes | |
| 14:22:50 | efried | okay, so still, is there an example where multiple different drivers could exist on a single host? | |
| 14:23:16 | sean-k-mooney | efried: not really. no. | |
| 14:23:27 | sean-k-mooney | you could have ironic and libvirt on the same node | |
| 14:23:34 | sean-k-mooney | i have done that but only for dev | |
| 14:23:45 | efried | okay. So host <=> driver for all intents and purposes. | |
| 14:24:05 | sean-k-mooney | you not really going to run libvirt and say nova-docker on the same host that said you could | |
| 14:24:09 | efried | sean-k-mooney: If you did have that kind of dual setup, would they be pointing to different configs? | |
| 14:24:22 | sean-k-mooney | efried: yes | |
| 14:24:27 | efried | okay, then the model holds. | |
| 14:25:17 | sean-k-mooney | efried: the point i was more trying to make is that the config them selve are not virt driver specic in the content they contain | |
| 14:25:29 | efried | wtf, of course they are | |
| 14:25:47 | sean-k-mooney | what is different? | |
| 14:25:50 | efried | you try to put a libvirt nova.conf onto a powervm compute node, it won't work at all. | |
| 14:26:07 | sean-k-mooney | efried: only the libvirt section would be different | |
| 14:26:23 | sean-k-mooney | anything outside the virt driver section should be identical | |
| 14:26:32 | efried | That sounds like a theoretical statement, not a practical one. | |
| 14:26:50 | sean-k-mooney | efried: if it not we have serious bugs | |
| 14:27:38 | sean-k-mooney | the only section that should be virt driver sepecific is the virt driver section. | |
| 14:27:50 | efried | Same config *format* I'll grant you, but the idea of being able to compose one config file and then copy it verbatim around a heterogeneous cloud - I have a hard time believing that anybody actually does that. | |
| 14:28:47 | sean-k-mooney | efried: alot of the installers bacsically do that | |
| 14:29:10 | sean-k-mooney | you are right here may be some hardware specific changes such as the vcpu_pin_set | |
| 14:29:35 | sean-k-mooney | but that dose not depend on the virt driver virt driver | |
| 14:29:54 | dansmith | sean-k-mooney: to have more than one compute per host you have to tell them each that they are a different hostname, and thus a different config, which means they might as well be separate hosts | |
| 14:31:20 | efried | kosamara: Based on this discussion, how do you feel about recommending that virt drivers log a warning, but not raise an exception, when they encounter a field they don't recognize/support/use? | |
| 14:31:49 | kosamara | efried: That's what I was converging to, I think that's best | |
| 14:32:05 | efried | this would give the admin the right clue as to why their device doesn't show up, if they were in fact expecting that field to be processed. | |
| 14:32:20 | kosamara | which satisfies my scenario | |
| 14:32:36 | sean-k-mooney | dansmith: that is true. | |
| 14:32:52 | efried | "WARNING: Device identification field XXX in inventory.yaml not recognized by the YYY driver; ignoring" | |
| 14:33:48 | sean-k-mooney | efried: do you need to raise an exception? | |
| 14:34:03 | efried | no, that's what kosamara and I are talking about. | |
| 14:34:11 | sean-k-mooney | sorry you said not raise an exception | |
| 14:34:41 | efried | If it's a common thing for configs to be copied around a cloud, then we should log and ignore. | |
| 14:35:36 | sean-k-mooney | efried: ya if we raise an exception from config parsing we normall hard crash the agent. but that seam wrong in this case | |
| 14:37:05 | efried | sean-k-mooney: This processing is happening in the compute service, in the virt driver's update_provider_tree method, on startup and/or SIGHUP, so I don't think it crashes the compute service; but it would make lots of things not work, depending where the exception was raised in update_provider_tree. | |
| 14:40:29 | sean-k-mooney | efried: sorry i was not being clear. it not where the parsing is done. we have two classes of config options. 1 where if its invalid we stop the agent form running and a second where we log a warning and continue on | |
| 14:40:46 | efried | sean-k-mooney: Note that this isn't oslo.config | |
| 14:40:49 | sean-k-mooney | this feels more like a warning to me then stop everything | |
| 14:41:31 | sean-k-mooney | efried: yes. the impentation is not what im concerned about its the semantics im thinking about | |
| 14:43:36 | efried | But, to avoid baffling the operator, we should log a warning that she'll see when she goes to investigate why her device isn't showing up. | |
| 14:43:36 | efried | For this reason, as well as in anticipation of future support for a global inventory.yaml on the conductor, it makes most sense *not* to treat an unrecognized field as an error. | |
| 14:43:36 | efried | Right. And semantically we converged on the fact that it's (apparently) fairly common for operators to write a config file once and then copy it around the cloud. | |