Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-03-07
15:03:08 mriedem tempest run: error: argument --regex/-r: expected one argument
15:03:10 dansmith jaypipes: yeah, but seems worth it.. i
15:03:16 dansmith jaypipes: I'll tack it on the end
15:05:40 mtreinish mriedem: hmm, that looks like a job not setting a regex and it was expecting one
15:06:04 mtreinish mriedem: I have no idea how/where any of the job definitions are setup now, maybe andreaf can help
15:06:05 mriedem mtreinish: so likely this http://logs.openstack.org/80/549880/1/experimental/legacy-tempest-dsvm-py35-full-devstack-plugin-ceph/7923b02/job-output.txt.gz#_2018-03-05_22_47_04_710732
15:06:13 mriedem mtreinish: it's in openstack-zuul-jobs
15:06:37 mriedem https://github.com/openstack-infra/openstack-zuul-jobs/blob/master/playbooks/legacy/tempest-dsvm-py35-full-devstack-plugin-ceph/run.yaml#L61
15:06:43 mriedem ^ doesn't exist in nova,
15:06:47 mriedem i think jbernard was working on adding it
15:07:05 mtreinish that probably is the cause then
15:07:25 mtreinish especially if that's being expected to set the regex things run with
15:07:26 andreaf mriedem I defined a new style job for ceph some time ago but the patch has been waiting for reviews for long time https://review.openstack.org/#/c/543048/
15:07:29 mriedem heh https://review.openstack.org/#/c/459774/
15:07:47 mriedem andreaf: is that py3?
15:07:55 mriedem andreaf: i can't read the new zuulv3 voodoo
15:07:56 andreaf mriedem both I think
15:08:08 andreaf yeah both
15:09:07 andreaf mriedem it defines the job and also a job template that runs py27 on check and py3 on experimental
15:09:27 andreaf the project stanzas are pretty much the same as before
15:10:11 andreaf mriedem mtreinish anyways my point was rather than fixing the legacy job perhaps we should run the new style one
15:11:19 jaypipes dansmith: coolio.
15:11:42 mriedem andreaf: yeah maybe - you should have the same regex in the py3 job in there
15:11:52 mriedem andreaf: given https://review.openstack.org/#/c/459774/4/tempest-conf.sh
15:13:40 idlemind presenting svm (nested virtualization - amd) to guests. i've set my cpu_mode to host-passthrough. when i look at the /proc/cpuinfo i see my amd cpu details but it's missing the svm property ... i'm pretty sure i had this working at one point but i can't seem to think of anything else i did
15:14:08 andreaf mriedem the regex wasn't in the legacy py3 job - I was just migrating it not enhancing it
15:14:19 andreaf mriedem but ok I can add it in there
15:15:19 Spazmotic mriedem wanted to just thank you again for your testing ideas. Was great getting more experience with Mock after wanting to pull my hair out for a couple of hours :)
15:17:14 openstackgerrit Rodolfo Alonso Hernandez proposed openstack/os-vif master: Add abstract OVSDB API https://review.openstack.org/476612
15:17:14 mriedem \o/
15:17:40 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: Save admin password to sysmeta in libvirt driver https://review.openstack.org/550498
15:18:21 mriedem andreaf: yeah i think the thing was jbernard was trying to slim down the set of tests that get run in the ceph job to stabilize it
15:18:30 mriedem andreaf: and then start expanding the test bucket
15:19:29 mriedem although the job should probably be running the scenario tests in serial
15:19:42 openstackgerrit Rodolfo Alonso Hernandez proposed openstack/os-vif master: Add native implementation OVSDB API https://review.openstack.org/482226
15:22:10 mriedem stephenfin: http://logs.openstack.org/80/549880/1/experimental/legacy-tempest-dsvm-py35-full-devstack-plugin-ceph/7923b02/logs/dpkg-l.txt.gz
15:22:11 mriedem ii ceph 12.2.1-0ubuntu0.17.10.1~cloud0
15:22:18 mriedem so we should be ok version wise
15:23:11 openstackgerrit Dan Smith proposed openstack/nova master: Add --purge helper flag to archive_deleted_rows https://review.openstack.org/550182
15:23:12 openstackgerrit Dan Smith proposed openstack/nova master: Make nova-manage db purge take --all-cells https://review.openstack.org/550502
15:24:32 stephenfin mriedem: You mean we should be OK now? Looks like we were using an older one when that bug was reported https://bugs.launchpad.net/glance-store/+bug/1706405/comments/3
15:24:33 openstack Launchpad bug 1706405 in glance_store "ceph jobs failing to upload images in pike due to "AttributeError: 'NoneType' object has no attribute 'Rados'" on py35" [Undecided,Confirmed]
15:24:49 mriedem stephenfin: that's a pretty old bug,
15:24:57 mriedem i'm watching https://review.openstack.org/#/c/543048/ to see if it passes now
15:25:03 stephenfin Right. Just making sure
15:25:55 mriedem andreaf: what is your plan to drop the legacy ceph job? are you just going to restrict that to the stable branches in openstack-zuul-jobs?
15:27:38 andreaf mriedem: that's what I'm doing at first for integration gate jobs, start with master only and then go back and replace for stable branches
15:28:42 andreaf mriedem: but that takes some time, so you're right you may want to fix the legacy job as well meanwhile
15:29:03 andreaf mriedem tbh I wasn't sure whether that job was important to anyone anymore - so I just put up my patch, asked for reviews and then I let it be
15:29:07 idlemind grr my issue might be from a hypervisor that wasn't rebooted (or kvm_amd wasn't removed/inserted) after enabling nested ... we'll see
15:29:23 mriedem andreaf: ceph ci is important about once per quarter :)
15:29:46 mriedem which reminds me that http://grafana.openstack.org/dashboard/db/ceph-failure-rate still doesn't work
15:34:09 openstackgerrit Dan Smith proposed openstack/nova master: Add --purge helper flag to archive_deleted_rows https://review.openstack.org/550182
15:34:10 openstackgerrit Dan Smith proposed openstack/nova master: Make nova-manage db purge take --all-cells https://review.openstack.org/550502
15:37:06 dansmith mriedem: I think it'd be easiest to rebase the archive --all-cells patch on top of my series instead of the other way around, just based on where the touch points are
15:37:12 dansmith which I will do if mine lands first
15:37:31 dansmith but he was first, so no worries either way
15:38:53 mriedem is there a dependency?
15:40:29 dansmith no just a conflict
15:40:53 dansmith but the conflict is way smaller if he goes on top of me, because of where in manage.py he's touching I think
15:41:27 mriedem hongbin: https://bugs.launchpad.net/nova/+bug/1754071 if you're interested
15:41:27 openstack Launchpad bug 1754071 in OpenStack Compute (nova) "image not found warning in logs when instance is deleted during snapshot" [Low,Triaged]
15:41:39 mriedem hongbin: as a follow up to https://review.openstack.org/#/c/511074/
15:42:19 hongbin mriedem: ack, will look into that
15:43:42 openstackgerrit sahid proposed openstack/nova-specs master: update: introducing isolate emulthreads on host https://review.openstack.org/511188
15:44:07 Spazmotic Sorry for the nickspam.. VPN shenanigans.
15:46:18 mriedem dansmith: artom: this is pretty nifty https://review.openstack.org/#/c/539584/ - we should consider doing that in the libvirt driver too
15:46:30 mriedem it's not obvious so i had to look at the paste that claudiub|2 put into the review comments
15:47:00 mriedem when port binding fails with the libvirt driver, i think we eventually just get the virtual interface exception or whatever, which logs a generic warning
15:47:02 dansmith ooh, blame
15:47:07 dansmith love it
15:47:33 mriedem and i always have to dig back through the compute logs to find the port binding failed error
15:54:28 mriedem andreaf: looks like the tempest regex changes are now making https://review.openstack.org/#/c/543048/ fail
15:54:47 mriedem don't think you can put comments in that yaml file for the regex can you? it looks like those comments are literally going into the regex intput
15:54:49 mriedem *input
15:58:31 idlemind hmm still can't get svm (amd nested) presented to a guest instance ... it's enabled on both hosts (cat /sys/module/kvm_amd/parameters/nested = 1). cpu_mode is host-passthrough and i see the model name correctly matches the underlying hardware directly but svm isn't picked up by the guest os
16:02:02 cdent mriedem: when you have a moment can you let me know what you'd like to have in place (spec, blueprint, whatever) to get the -2 lifted from https://review.openstack.org/#/c/362766/ (that's the optional placement db stuff). thanks.
16:06:57 mriedem cdent: reading the notes on the -2, it sounds like we talked about it at one point in a nova meeting and asked for a spec to cover the details of the change, how it gets rolled into CI (maybe the nova-next job?), and how to avoid whatever issue we had when it was merged and then reverted in newton (which i wasn't involved with at the time)
16:07:22 mriedem as dansmith mentioned at the ptg, you could run a separate placement db today,
16:07:39 mriedem if you point the nova.conf placement is using at a different placement database using the nova_api schema
16:07:46 cdent mriedem: aye
16:08:03 mriedem which in a spec i guess just goes under 'alternatives'
16:08:52 cdent now that that stuff is within a suite of other changes would you suggest that the spec covers all of it (in one spec) or something else?
16:09:31 mriedem suite of other changes == moving imports around?
16:10:31 cdent moving imports, moving objects into placemnet hierarchy, changing db config to use its own code (instead of the nova one which does more than needed)
16:11:01 mriedem i wouldn't lump that into the 'be able to run nova-api and placement-api on the same host with a single config but different dbs' thing
16:11:11 mriedem that other stuff is more general 'extract placement'
16:11:21 mriedem of which the separate db is a part
16:11:32 cdent ✔
16:11:54 mriedem another alternative to this,
16:12:24 mriedem is you could run nova-api and placement-api on the same host, not venv/containers, but if you had a config file strictly for placement, then you'd just run the placement service using that config file
16:13:16 mriedem nova-api --config-file /etc/nova/nova.conf && placement-api --config-file /etc/nova/placement.conf ?
16:14:50 cdent the way I did it in the current change was done that way mostly to make "doing stuff in devstack (and thus CI)" relatively easy: add a single config setting, set it, done
16:15:14 mriedem which might be how everyone else deploys everything today,
16:15:30 cdent the container experiments I'm doing use a custom config file for the container, which is a severely curtailed nova.conf
16:15:50 mriedem so unless i'm missing something, it seems to be a trade off between ease of deployment for nova + placement with a single config file, vs nova not doing this and just leaving it up to packagers/deployment tooling to handle the split if they want a split
16:16:22 mriedem eventually once placement is split out and has it's own placement.conf, it would just have a single [database] option group right?
16:16:50 cdent My feeling is that the optional config thing is just a convenience to make life easier (for us and other people) during whatever length of transition we have.
16:17:08 cdent It, uh, leaves options open...
16:17:18 mriedem pun intended

Earlier   Later