| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-07-25 | |||
| 20:51:39 | mriedem | and so we'll create the compute node record here https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L531 | |
| 20:51:58 | mriedem | http://logs.openstack.org/79/487179/1/check/gate-tempest-dsvm-neutron-multinode-full-ubuntu-xenial-nv/1dd9db5/logs/subnode-2/screen-n-cpu.txt.gz#_Jul_25_19_30_44_459686 | |
| 20:52:27 | mriedem | sdague: but does systemd wait or does it just launch off the service start and not block on it? | |
| 20:52:50 | sdague | mriedem: define wait | |
| 20:55:20 | sdague | mriedem: we're starting in the foreground, it's tracking the parent process, but the issue is it's much later that things are ready | |
| 20:56:21 | sdague | anyway, I need to work on dinner, I've got this half assed patch running locally, if it works I'll push it | |
| 20:56:24 | mriedem | right when we start the n-cpu service, and the service is launched, that's all async | |
| 20:56:31 | sdague | mriedem: it's not async | |
| 20:56:38 | jangutter | mriedem, jaypipes: should I assert on exception.NovaException or exception.InternalError at https://review.openstack.org/#/c/486426/6/nova/tests/unit/virt/libvirt/test_vif.py@1616 | |
| 20:56:49 | sdague | it's that it's not ready for 25 - 30 seconds after start | |
| 20:57:00 | sdague | and, there is no /health to know that | |
| 20:57:19 | sdague | we poll api processes that we start to know they are ready before we move on | |
| 20:57:26 | sdague | but there isn't a direct interface for that | |
| 20:57:34 | mriedem | right i meant https://github.com/openstack/nova/blob/master/nova/service.py#L138 | |
| 20:57:47 | mriedem | which is what calls compute manager pre_start_hook that sets this all up | |
| 20:58:09 | sdague | mriedem: ok, before I leave, I want to make sure we get this question clear :) | |
| 20:58:30 | sdague | systemd is starting things, and it's running as parent process as soon as python exec happens | |
| 20:59:00 | mriedem | i guess i was thinking about like sysv init scripts and services, | |
| 20:59:05 | mriedem | where you can run service nova-compute status | |
| 20:59:08 | mriedem | and see if it's started or not | |
| 20:59:19 | sdague | sure, but all that tells you is if the process is running | |
| 20:59:25 | sdague | the process is running | |
| 20:59:38 | sdague | eventually the process is ready | |
| 20:59:42 | mriedem | but can't the start routine block until it's actually started or crashed? | |
| 20:59:51 | sdague | it is started | |
| 20:59:58 | sdague | the process is running | |
| 21:00:27 | sdague | how does anything external know if a process is ready other than pid existing? | |
| 21:00:31 | mriedem | ok, well this is all latent stuff and shouldn't block https://review.openstack.org/#/c/477556/ | |
| 21:00:35 | mriedem | so can we get that in? | |
| 21:00:54 | mriedem | like, this behavior goes back to ocata | |
| 21:01:04 | sdague | https://www.freedesktop.org/software/systemd/man/systemd-notify.html if you want deeper state interaction between process and systemd | |
| 21:01:34 | sdague | mriedem: the neutron folks are currently borked? | |
| 21:01:39 | mriedem | no | |
| 21:01:49 | mriedem | the dvr-ha multinode job is non-voting and in the experimental queue | |
| 21:01:54 | sdague | ok | |
| 21:01:55 | mriedem | i've already talked to haleyb about it | |
| 21:02:07 | sdague | if they are cool with it, that's fine | |
| 21:02:44 | sdague | I'll try to get this wait call in place | |
| 21:02:52 | sdague | I just appoved the fleet patch | |
| 21:03:01 | mriedem | ok | |
| 21:03:13 | sdague | this other thing takes a while to run, so off for the night, we'll see what it looks like in the morning | |
| 21:06:21 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove redundant free_vcpus logging in _report_hypervisor_resource_view https://review.openstack.org/487216 | |
| 21:11:28 | jangutter | mriedem, jaypipes: should I assert on exception.NovaException or exception.InternalError at https://review.openstack.org/#/c/486426/6/nova/tests/unit/virt/libvirt/test_vif.py@1616 | |
| 21:14:01 | mriedem | internal error | |
| 21:14:17 | mriedem | you should assert the thing being raised | |
| 21:14:33 | jangutter | roger, tightening the noose. | |
| 21:14:49 | mriedem | try/catch Throwable | |
| 21:15:38 | openstackgerrit | Jan Gutter proposed openstack/nova master: Add VIFHostDevice support to libvirt driver https://review.openstack.org/486426 | |
| 21:17:23 | jaypipes | mriedem: raisable. :P | |
| 21:22:52 | openstackgerrit | Jan Gutter proposed openstack/nova master: Netronome SmartNIC Enablement https://review.openstack.org/483459 | |
| 21:26:29 | mriedem | dansmith: fyi about fleet week to the dev list | |
| 21:26:31 | mriedem | sent | |
| 21:28:06 | dansmith | mriedem: ack, I'll read when I return in a bit | |
| 21:35:20 | mriedem | jaypipes: your claims in the scheduler change | |
| 21:35:30 | mriedem | http://logs.openstack.org/66/483566/10/check/gate-tempest-dsvm-ironic-ipa-wholedisk-bios-agent_ipmitool-tinyipa-ubuntu-xenial-nv/3a475d1/logs/screen-n-sch.txt.gz#_Jul_24_16_26_25_372045 | |
| 21:35:30 | mriedem | the ironic nv job failed on a scheduling novalidhost | |
| 21:35:44 | mriedem | Jul 24 16:26:25.372045 ubuntu-xenial-osic-cloud1-disk-10046831 nova-scheduler[11679]: DEBUG nova.scheduler.filter_scheduler [None req-76fb384b-64f6-4ed7-86fd-dbb616497e43 tempest-BaremetalBasicOps-2030059131 tempest-BaremetalBasicOps-2030059131] Found host state None that wasn't in allocation requests. Skipping. {{(pid=11679) _schedule /opt/stack/new/nova/nova/scheduler/filter_scheduler.py:196}} | |
| 21:38:29 | mriedem | aha | |
| 21:38:31 | mriedem | YES! | |
| 21:40:51 | openstackgerrit | Ildiko Vancsa proposed openstack/nova master: Implement new attach Cinder flow https://review.openstack.org/330285 | |
| 21:43:54 | ildikov | mriedem: I fixed the translate patch and the attach patch too should have a clean test run now | |
| 21:44:08 | ildikov | mriedem: let me know if there's anything else I should do | |
| 21:44:13 | ildikov | mriedem: tnx | |
| 21:53:24 | openstackgerrit | Matt Riedemann proposed openstack/nova master: claim resources in placement API during schedule() https://review.openstack.org/483566 | |
| 21:53:25 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Set IronicNodeState.uuid in _update_from_compute_node https://review.openstack.org/487227 | |
| 21:53:26 | mriedem | jaypipes: ^ | |
| 22:04:33 | jackie-truong | Hi @dansmith, are you available? I have some questions on nova/objects/instance.py that I think you can answer | |
| 22:04:46 | dansmith | jackie-truong: for a few minutes yeah | |
| 22:05:22 | jackie-truong | dansmith: I'm helping out with this patch https://review.openstack.org/#/c/457711/ | |
| 22:06:10 | jackie-truong | dansmith: And we're having some issues with properly initializing the list of strings (trusted_certs) so that it is actually an Instance field | |
| 22:07:01 | dansmith | jackie-truong: well, it's defined as a field, which means it's actually a field, but go on... | |
| 22:07:38 | jackie-truong | We're seeing the error "Instance object has no attribute 'trusted_cert'" | |
| 22:08:03 | dansmith | jackie-truong: well, it's defined as "trusted_certs" (note the plural), so maybe that's why? | |
| 22:08:23 | jackie-truong | dansmith: Sorry, that was a typo on my part | |
| 22:08:35 | jackie-truong | "Instance object has no attribute 'trusted_certs' " | |
| 22:08:37 | dansmith | damn, thought that was going to be quick :P | |
| 22:09:00 | jackie-truong | dansmith: lol | |
| 22:09:07 | dansmith | well, I'd need to see where/how it's used then I guess, | |
| 22:09:08 | jackie-truong | dansmith: If you need to head out, I can hop on later this week | |
| 22:09:25 | dansmith | but you really shouldn't get an AttributeError the way you have it defined here | |
| 22:09:25 | jackie-truong | dansmith: We thought this would be sufficient: "instance['trusted_certs'] = db_inst['trusted_certs']" | |
| 22:09:42 | dansmith | L457? | |
| 22:10:00 | jackie-truong | dansmith: Yes, L457 | |
| 22:10:22 | dansmith | db_inst is an "Instance" as well although it's from the model.. maybe it's actually complaining about that one? | |
| 22:10:34 | jackie-truong | dansmith: Would we need to add some lines to load the trusted_certs attribute in obj_load_attr? | |
| 22:11:09 | dansmith | jackie-truong: I don't see trusted_certs on the db model | |
| 22:11:24 | dansmith | so I expect that's the problem. is there some other patch that is supposed to be under this? | |
| 22:12:31 | jackie-truong | Yes, this is part of the Nova certificate validation blueprint https://blueprints.launchpad.net/openstack/?searchtext=nova-validate-certificates | |
| 22:12:56 | jackie-truong | dansmith: But no, we didn't add anything to nova/db/sqlalchemy/models.py | |
| 22:13:12 | dansmith | jackie-truong: well, uh, where do you think it's getting stored then? :P | |
| 22:13:19 | jackie-truong | Good point >< | |
| 22:13:26 | dansmith | jackie-truong: but yeah, I bet that's what you're hitting | |
| 22:13:56 | jackie-truong | Cool, thanks! So there's no need to add anything to instance.py's obj_load_attr? | |
| 22:14:26 | jackie-truong | In other words, once we put trusted_certs in the db models, then L457 should be good? | |
| 22:15:37 | dansmith | L457 will be yeah.. you only need obj_load_attr() changes if you want to be able to lazy-load this field, which I think I said on that review a while back | |
| 22:15:59 | dansmith | you'll need a sql migration to add the column, then support on the model, then this patch and it should load | |
| 22:16:20 | dansmith | although a list is a list, not a column, so you need to figure out how you're going to store that and bring it in as a list of things, but I'm sure you know that | |
| 22:16:43 | jackie-truong | Got it. Thanks a lot, dansmith! | |
| 22:24:33 | openstackgerrit | Vipin Balachandran proposed openstack/nova master: VMware:Use power_state const for power state check https://review.openstack.org/483122 | |
| 22:26:44 | dansmith | jackie-truong: np | |