| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-07-27 | |||
| 09:37:17 | sean-k-mooney | so you shoudl always be abel to set it indepenently of the host | |
| 09:37:52 | CeeMac | it appears to be a known issue around how windows handles time from the bios clock, which i believe is the entire premise on why the os_type=windows patch was ported from hyper-v hypervisor to kvm hypervisor in nova | |
| 09:37:59 | sean-k-mooney | CeeMac: for what its worth i hate daylight saving time so im glad ill by on utc permently next year | |
| 09:38:06 | CeeMac | sean-k-mooney: yeah, thats the dream! | |
| 09:38:28 | CeeMac | yeah, can't say I'm a big fan either. its those pesky customers who are complaining :) | |
| 09:38:35 | sean-k-mooney | well os_type=windows does other things too | |
| 09:38:40 | sean-k-mooney | but that patch yes | |
| 09:39:31 | sean-k-mooney | although when we use os_type=windows it shoudl enabel the hyperv clock | |
| 09:39:42 | CeeMac | from what I gather, there is a reg hack that resolves the problem, but microsoft wont support it and say it is "buggy", whatever that means | |
| 09:39:50 | sean-k-mooney | which clock is this changing | |
| 09:39:58 | sean-k-mooney | we have multipel clock source i think in the vm | |
| 09:40:28 | sean-k-mooney | CeeMac: why not set the windows vms to utc | |
| 09:40:43 | sean-k-mooney | you can disable DST in them | |
| 09:40:43 | CeeMac | get added to the clock offset stanza for instances with os_type=windows | |
| 09:40:43 | CeeMac | i've seen the <timer name='hypervclock' present='yes'/> | |
| 09:41:11 | sean-k-mooney | CeeMac: ill pretend your not altering the xml :) | |
| 09:41:27 | CeeMac | sean-k-mooney: its ok, i'm not, i'm just looking at it with dumpxml | |
| 09:41:31 | CeeMac | :) | |
| 09:42:07 | sean-k-mooney | so we have offset and timezone https://github.com/openstack/nova/blob/master/nova/virt/libvirt/config.py#L703-L705 | |
| 09:42:35 | CeeMac | so, the issue we're looking at currently, is a hosted platform for a customer clocking / time management systems. Presumably they want/need the clock in / clock out times to register against the correct regional time for accurate reporting | |
| 09:44:28 | sean-k-mooney | and then we add the hyperv clock source https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L5840-L5844 in addation to the pit, rtc and optionally the hpet | |
| 09:45:13 | sean-k-mooney | CeeMac: right but the normal way to do that is to use UTC and then do the conversion client side | |
| 09:45:40 | sean-k-mooney | you should never store data in local time | |
| 09:46:05 | CeeMac | tbh, until we came across this issues it was always my belief that the windows kernel ran in utc natively and just adusted 'user' time based on the regional settings | |
| 09:46:11 | sean-k-mooney | you enither use unix time or some other utc reference time like isotime format | |
| 09:46:23 | CeeMac | but windows | |
| 09:46:29 | CeeMac | appears to be the issues | |
| 09:46:30 | sean-k-mooney | CeeMac: i belive it does with a different epoc | |
| 09:46:46 | CeeMac | as its chose to use the 'localtime' standard, if you can call it a standard | |
| 09:46:50 | CeeMac | stupid windows | |
| 09:46:51 | sean-k-mooney | CeeMac: i think windows expect the hardware time to be in utc | |
| 09:47:22 | CeeMac | i think more stick poking is required | |
| 09:48:23 | CeeMac | thanks sean-k-mooney we'll have a ponder and see if we can see whats happening, or not happening as the case may be | |
| 09:49:38 | sean-k-mooney | https://libvirt.org/formatdomain.html#time-keeping | |
| 09:51:06 | CeeMac | thanks, thats an interesting read. | |
| 09:52:12 | sean-k-mooney | so reading that localtime should give you what you wanted i think | |
| 09:52:36 | CeeMac | yeah, that was our theory. Perhaps we're missing something in the os | |
| 09:52:52 | sean-k-mooney | we can use timezone since we can only update that on reboot so you woudl have to reboot your vms evey 6 months | |
| 09:53:11 | CeeMac | i presume there is no easy way to insert the additional track and catchup information? | |
| 09:53:15 | sean-k-mooney | we could maybe use variable | |
| 09:53:36 | sean-k-mooney | its not currently exposed in nova no | |
| 09:54:05 | sean-k-mooney | this is getting more speicic then im comfortable exposing in a cloud env by the way | |
| 09:54:10 | CeeMac | all the data suggests that it should work, as you say | |
| 09:54:15 | sean-k-mooney | we might expsoe it but im not sure we should in general | |
| 09:54:48 | CeeMac | no thats fine, i get that. It must be working for other people, otherwise I'm sure there would be more chat out there for it not working | |
| 09:54:58 | sean-k-mooney | <clock offset='varible' basis='localtime'> | |
| 09:55:04 | sean-k-mooney | we cold maybe try that | |
| 09:55:09 | sean-k-mooney | *could | |
| 09:57:12 | sean-k-mooney | CeeMac: if we were to expose the timer settign i think it woudl ahve to be at teh image level rghat then host by the way | |
| 09:57:42 | CeeMac | sean-k-mooney: yes, that would make sense | |
| 09:57:55 | sean-k-mooney | CeeMac: host level config that modify the xml are basicaly terribel form a live migration standpoint as we havne to schdule on it or pass the data or both | |
| 09:58:36 | sean-k-mooney | so this type of info really needt to live with the instnace for its lifetime hence flavor or image | |
| 09:59:08 | sean-k-mooney | with that said i woudl be tempeted to start using server metadata for this personally | |
| 09:59:29 | sean-k-mooney | since this is a very pet like tuning | |
| 09:59:40 | CeeMac | it makes sense that it should follow the instance | |
| 09:59:48 | CeeMac | windows is all about the pet sadly | |
| 10:01:26 | CeeMac | server metadata would be useful i think | |
| 10:02:55 | sean-k-mooney | we dont currently allow it to modify xml exctra but i have always wondered if we should allow it to set anything that is setabel via image metadata | |
| 10:03:46 | sean-k-mooney | that would require a spec and some semi invasive changes to parts of the libvirt driver | |
| 10:03:49 | CeeMac | in some scenarios it could be beneficial, especially if its discovered you need to retrofit a value that would normally only be doable through an image | |
| 10:03:54 | sean-k-mooney | so not sure its worth it | |
| 10:03:59 | CeeMac | its still a very pet mentality granted | |
| 10:04:20 | CeeMac | hmm, cost/benefit are skewed i guess | |
| 10:04:30 | sean-k-mooney | CeeMac: ya we are adding a nova-manage command that operators can use for a limit set of image properties | |
| 10:04:57 | CeeMac | oh, that sounds interesting | |
| 10:05:25 | sean-k-mooney | the intent is for operators taht need to change things for upgrades | |
| 10:05:39 | sean-k-mooney | e.g. move to q35 which means you have to remove use of ide | |
| 10:05:48 | sean-k-mooney | ectra | |
| 10:06:15 | sean-k-mooney | we approved https://github.com/openstack/nova-specs/blob/master/specs/newton/approved/virt-image-props-boot-override.rst in the past but then decieded to no porceed with it when we came to implemenation | |
| 10:07:21 | sean-k-mooney | im still somewhat open to that idea but i dont fully recal what the main objects were | |
| 10:07:32 | CeeMac | i imagine its tricky finding a decent balance point between stability and the ability to make dynamic changes | |
| 10:08:15 | sean-k-mooney | ya and stricking a blance between upstream and downstream | |
| 10:08:35 | sean-k-mooney | downstream our hands are forced a bit by change made by other teams | |
| 10:09:21 | sean-k-mooney | for example qxl graphic is going away downstream so we have to provide a way to move instances off it which is what propmeted the change | |
| 10:25:26 | sean-k-mooney | CeeMac: this is the new nova-magage command by the way https://github.com/openstack/nova-specs/blob/master/specs/xena/approved/nova-manage-commands-to-update-libvirt-device-models.rst | |
| 10:25:42 | sean-k-mooney | CeeMac: lyarwood is currently workign on it for xena | |
| 10:28:49 | CeeMac | looks good | |
| 10:46:47 | opendevreview | Merged openstack/python-novaclient master: Use Block Storage API v3 instead of API v2 https://review.opendev.org/c/openstack/python-novaclient/+/802417 | |
| 11:13:51 | opendevreview | Merged openstack/nova master: Bump os-resource-classes to 1.1.0 https://review.opendev.org/c/openstack/nova/+/800976 | |
| 11:50:53 | opendevreview | Merged openstack/os-vif master: add configurable per port bridges https://review.opendev.org/c/openstack/os-vif/+/798055 | |
| 12:34:33 | brinzhang | gibi,sean-k-moonkey,stephenfin,alex_xu: we submited the PoC code for allowing PMEM's data wihle migrate vm, but it is not completely implemented in accordance with the spec, because some irrationality in the spec was discovered during the implementation process, so we hope you can review the PoC code. | |
| 12:35:18 | brinzhang | ah, gibi has a holday(pto), gibi_pto | |
| 12:36:08 | brinzhang | the PoC code link: https://review.opendev.org/c/openstack/nova/+/802225 | |
| 12:36:44 | sean-k-mooney | brinzhang: do you have a patch to update the spec whith what diverged | |
| 12:37:28 | sean-k-mooney | brinzhang: what exactly was teh "irrationality in the spec" it may have been stated that way for a reason which the impleation misses | |
| 12:40:46 | brinzhang | sean-k-mooney: in spec we want to ask ensure which pmem we can copy in the target host when run migtate_disk_and_power_off, but this time we cannt know the target pmem's path in the context | |
| 12:41:04 | sean-k-mooney | yes we can | |
| 12:41:12 | brinzhang | https://review.opendev.org/c/openstack/nova-specs/+/785563/14/specs/xena/approved/allow-migrate-pmem-data.rst#65 | |
| 12:41:41 | sean-k-mooney | its includeded in the instance claim if i recall correctly | |
| 12:41:44 | brinzhang | we test and write the logs, and we cannt get the target PMEM's path from the context | |
| 12:43:31 | sean-k-mooney | ill have to review but i was pretty sure i pointed to the code where you could get it at one point | |
| 12:43:43 | brinzhang | not exectly, it's cannot claim it now, now we just get the pmem's path when we were started if I wasn's missed some key info | |
| 12:43:53 | sean-k-mooney | no | |
| 12:44:03 | sean-k-mooney | it must claim it before you do the data copy | |
| 12:44:25 | sean-k-mooney | you cannot pass back paths to unclaimed devices | |
| 12:44:33 | sean-k-mooney | that is a potential securrity bug | |
| 12:46:19 | brinzhang | yes, hope we were wrong, you can review the PoC code and give some points, thanks | |
| 12:46:27 | sean-k-mooney | prep_resize should have claimed the remote pmemdevices and store them in the migration context | |
| 12:46:53 | sean-k-mooney | brinzhang: i can but since it deviates form the spec im -1 untill you can show why the spec wont work | |
| 12:47:18 | sean-k-mooney | why did you not save the pmem device in the migration context in prep_resize | |